2024-09-21T02:55:47.995 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T02:55:48.030 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:55:48.169 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913450' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCF6ZAEBepPVcGjVSFnzKA3RFKTucB2sC44AvwtQNfgjiu1M2NmP5icyII/Do+vACU8KsfW+cDfmUpJvItD6i8w= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgJoUu9vDqIH5zwLIPywBxI1AU/VNVDNccg6jXIRssZU4egmoQKV8F4ZuHb0lA20aWwWE18RfhRmUhhyOQmXRY= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJDPE+VQMUghYj6yqbJ5Odf274ttDWILeUEhX5sCBb3hii8csM08jfuFABUOv297ygg3dR2Q51olkQ4HtjXva0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T02:55:48.169 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T02:55:48.170 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T02:55:48.171 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T02:55:48.172 INFO:teuthology.task.internal:Checking packages... 2024-09-21T02:55:48.194 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T02:55:48.195 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T02:55:48.195 INFO:teuthology.packaging:ref: None 2024-09-21T02:55:48.195 INFO:teuthology.packaging:tag: None 2024-09-21T02:55:48.195 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:55:48.195 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:55:48.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:55:48.336 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T02:55:48.337 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T02:55:48.353 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T02:55:48.353 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T02:55:48.413 INFO:teuthology.task.internal:Saving configuration 2024-09-21T02:55:48.433 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T02:55:48.473 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T02:55:48.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:50:50.133144', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCF6ZAEBepPVcGjVSFnzKA3RFKTucB2sC44AvwtQNfgjiu1M2NmP5icyII/Do+vACU8KsfW+cDfmUpJvItD6i8w='} 2024-09-21T02:55:48.523 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:50:50.131860', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgJoUu9vDqIH5zwLIPywBxI1AU/VNVDNccg6jXIRssZU4egmoQKV8F4ZuHb0lA20aWwWE18RfhRmUhhyOQmXRY='} 2024-09-21T02:55:48.544 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:50:50.134505', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJDPE+VQMUghYj6yqbJ5Odf274ttDWILeUEhX5sCBb3hii8csM08jfuFABUOv297ygg3dR2Q51olkQ4HtjXva0='} 2024-09-21T02:55:48.544 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T02:55:48.550 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T02:55:48.550 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T02:55:48.550 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T02:55:48.550 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T02:55:48.648 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9f188ec160>, signals=[15]) 2024-09-21T02:55:48.649 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T02:55:48.654 INFO:teuthology.task.internal:Opening connections... 2024-09-21T02:55:48.655 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T02:55:48.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:55:48.735 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-09-21T02:55:48.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:55:48.809 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T02:55:48.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:55:48.884 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T02:55:48.890 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-09-21T02:55:48.908 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-09-21T02:55:48.909 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-09-21T02:55:48.966 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-09-21T02:55:48.966 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-09-21T02:55:48.966 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-09-21T02:55:48.966 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:55:48.967 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:55:48.968 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-09-21T02:55:48.990 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-09-21T02:55:49.008 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-09-21T02:55:49.008 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-09-21T02:55:49.066 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:55:49.067 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:55:49.067 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:55:49.067 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:55:49.067 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:55:49.067 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:55:49.067 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-09-21T02:55:49.089 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-21T02:55:49.108 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-21T02:55:49.109 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:55:49.167 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:55:49.168 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:55:49.168 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:55:49.168 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:55:49.168 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-21T02:55:49.190 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T02:55:49.201 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T02:55:49.230 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T02:55:49.231 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:55:49.233 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:55:49.235 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:55:49.252 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T02:55:49.259 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T02:55:49.259 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:55:49.291 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:55:49.293 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:55:49.394 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T02:55:49.469 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T02:55:49.484 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T02:55:49.484 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T02:55:49.484 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T02:55:49.484 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:55:49.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:55:49.501 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-21T02:55:49.559 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-503.el9.x86_64 2024-09-21T02:55:49.559 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-503.el9.x86_64 2024-09-21T02:55:49.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-09-21T02:55:52.473 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.3 MB 00:01 2024-09-21T02:55:56.783 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-21T02:56:02.050 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-21T02:56:03.526 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-21T02:56:04.655 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T02:56:10.052 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T02:56:11.917 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T02:56:11.917 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T02:56:12.013 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-21T02:56:12.028 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-09-21T02:56:12.029 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:56:12.030 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 79 M 2024-09-21T02:56:12.030 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 126 M 2024-09-21T02:56:12.030 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-21T02:56:12.212 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 477 kB/s | 36 kB 00:00 2024-09-21T02:56:12.737 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T02:56:13.246 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-21T02:56:13.671 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-21T02:56:13.671 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:56:13.671 INFO:teuthology.orchestra.run.smithi083.stdout:Total 48 MB/s | 79 MB 00:01 2024-09-21T02:56:14.350 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-21T02:56:14.709 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-21T02:56:14.709 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-21T02:56:15.648 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-21T02:56:15.648 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-21T02:56:18.069 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-21T02:56:18.758 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:56:18.841 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:56:21.579 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:56:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:56:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:56:26.686 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:56:31.512 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:57:10.055 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:57:10.095 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:57:10.476 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:57:10.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:57:10.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:57:10.476 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:57:11.290 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:57:11.290 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:57:11.291 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-21T02:57:11.637 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:57:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:57:11.729 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T02:57:11.729 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T02:57:11.729 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T02:57:11.729 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T02:57:11.730 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T02:57:11.730 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:57:11.788 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:57:11.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:57:11.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 82.el9 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8274670 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-09-21T02:57:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T02:57:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T02:57:11.829 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T02:57:11.829 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 5.14.0-508.el9.x86_64 2024-09-21T02:57:11.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T02:57:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-09-21T02:57:13.792 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T02:57:13.814 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-09-21T02:57:13.816 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-21T02:57:13.831 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.LL0vmTSQTz 2024-09-21T02:57:13.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LL0vmTSQTz 2024-09-21T02:57:13.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.LL0vmTSQTz 2024-09-21T02:57:14.031 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.LL0vmTSQTz is 6KB 2024-09-21T02:57:14.080 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.LL0vmTSQTz 2024-09-21T02:57:14.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T02:57:14.162 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T02:57:14.162 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T02:57:14.162 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T02:57:14.162 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T02:57:14.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T02:57:14.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-09-21T02:57:44.333 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T02:57:44.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T02:57:44.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:58:18.991 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-21T02:58:27.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T02:58:27.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:58:46.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-21T02:58:58.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T02:58:58.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:58:58.744 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-21T02:58:59.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T02:58:59.103 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T02:58:59.103 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-21T02:58:59.118 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-508.el9.x86_64 2024-09-21T02:58:59.118 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T02:58:59.118 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T02:58:59.118 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T02:59:00.119 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:00.120 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T02:59:00.130 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T02:59:00.130 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:59:00.132 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:59:00.135 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:59:00.153 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T02:59:00.187 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T02:59:00.229 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T02:59:00.229 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:59:00.231 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:59:00.233 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:59:00.333 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T02:59:00.341 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T02:59:00.341 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:59:00.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:59:00.358 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:59:00.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:59:00.376 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:59:00.378 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:59:00.401 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:59:00.455 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.467 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.483 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.489 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.497 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.584 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:59:00.585 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T02:59:00.593 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T02:59:00.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:59:00.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:59:00.597 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:59:00.625 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T02:59:00.637 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T02:59:00.637 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:59:00.664 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:59:00.667 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:59:00.683 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:00.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:00.827 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:00.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:00.979 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T02:59:00.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:59:01.062 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:01.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:01.159 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:01.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:01.288 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T02:59:01.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:59:01.355 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:01.400 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:59:01.459 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:01.543 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:59:01.587 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T02:59:01.587 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:59:01.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-09-21T02:59:01.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-09-21T02:59:01.659 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-21T02:59:01.693 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:59:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:59:01.738 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:59:02.214 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T02:59:02.220 INFO:teuthology.task.internal:Starting timer... 2024-09-21T02:59:02.221 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T02:59:02.275 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T02:59:02.323 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T02:59:02.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-09-21T02:59:02.450 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-09-21T02:59:02.452 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-09-21T02:59:02.500 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-09-21T02:59:02.502 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-21T02:59:02.576 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-21T02:59:02.578 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T02:59:02.578 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-09-21T02:59:02.604 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-09-21T02:59:02.604 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-09-21T02:59:02.632 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-09-21T02:59:02.633 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-21T02:59:02.655 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-21T02:59:02.655 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-21T02:59:02.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:59:02.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:59:02.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:59:02.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:59:02.711 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:59:02.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:59:02.737 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T02:59:02.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:59:02.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:59:02.815 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:59:02.848 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T02:59:02.856 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T02:59:02.867 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T02:59:02.868 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:03:04.190 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-21T03:03:04.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T03:03:04.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:03:04.269 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-21T03:03:04.341 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-21T03:03:04.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-21T03:03:04.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:03:04.417 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-21T03:03:04.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-21T03:03:04.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T03:03:04.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:03:04.570 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-21T03:03:04.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T03:03:04.647 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:03:04.664 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:03:04.664 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:03:04.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:03:04.667 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:03:04.667 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:03:04.669 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:03:04.669 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:03:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:03:04.697 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:03:04.709 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:03:04.712 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:03:04.718 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:03:04.733 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:03:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-09-21T03:03:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-09-21T03:03:04.746 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-09-21T03:03:04.748 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-09-21T03:03:04.758 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-21T03:03:04.760 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:03:04.762 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:03:04.770 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-21T03:03:04.774 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:03:04.776 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:03:04.784 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:03:04.798 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:03:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.831 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.836 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:03:04.839 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:03:04.839 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-21T03:03:04.839 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.839 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.839 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.840 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.850 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.854 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:03:04.855 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:03:04.863 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:03:04.863 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:03:04.863 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:03:04.863 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:03:04.868 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:03:04.868 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:03:04.869 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:03:04.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:03:04.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:03:04.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:03:04.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:03:04.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:03:05.023 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:03:05.023 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:03:05.027 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:03:05.027 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:03:05.030 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:03:05.030 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:03:05.132 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:03:05.132 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:03:05.132 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:03:05.138 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:03:05.139 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:03:05.139 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:03:05.144 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:03:05.145 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:03:05.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:03:05.158 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:03:05.158 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:03:05.164 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:03:05.164 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:03:05.171 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:03:05.171 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:03:05.224 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:03:05.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:03:05.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:03:05.299 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:03:05.304 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:03:05.311 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:03:05.363 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-09-21T03:03:05.365 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-09-21T03:03:05.368 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-09-21T03:03:05.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-09-21T03:03:05.376 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-09-21T03:03:05.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-09-21T03:03:05.590 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-09-21T03:03:05.594 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-09-21T03:03:05.617 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:03:05.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:03:05.624 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-09-21T03:03:05.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:03:06.158 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-21T03:03:06.186 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 287 kB/s | 84 kB 00:00 2024-09-21T03:03:06.224 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-21T03:03:06.441 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T03:03:06.486 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 93 kB/s | 24 kB 00:00 2024-09-21T03:03:06.525 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T03:03:06.718 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:03:06.762 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T03:03:06.801 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:03:07.375 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T03:03:07.971 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.3 MB 00:01 2024-09-21T03:03:08.684 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-21T03:03:09.326 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-21T03:03:12.889 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 20 MB 00:03 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-21T03:03:16.278 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-21T03:03:17.215 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T03:03:17.946 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-21T03:03:18.581 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-21T03:03:20.265 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-21T03:03:21.178 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T03:03:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.5 MB 00:00 2024-09-21T03:03:22.588 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-21T03:03:24.070 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-21T03:03:24.748 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:24.749 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:24.760 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:03:24.761 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:03:24.762 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:03:24.763 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:03:24.764 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:03:24.767 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:03:24.768 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:03:24.831 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:03:24.855 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:03:24.856 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:03:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:03:24.858 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:03:24.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:03:24.860 INFO:teuthology.orchestra.run.smithi142.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:03:24.861 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:03:24.862 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:03:24.863 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:03:24.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:03:24.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:03:24.866 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-21T03:03:24.867 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-21T03:03:24.868 INFO:teuthology.orchestra.run.smithi142.stdout:Install 149 Packages 2024-09-21T03:03:24.868 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 8 Packages 2024-09-21T03:03:24.868 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:03:24.868 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 228 M 2024-09-21T03:03:24.868 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-21T03:03:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T03:03:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T03:03:25.989 INFO:teuthology.orchestra.run.smithi142.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.1 MB/s | 873 kB 00:00 2024-09-21T03:03:26.056 INFO:teuthology.orchestra.run.smithi142.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T03:03:26.306 INFO:teuthology.orchestra.run.smithi142.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.2 MB/s | 2.3 MB 00:00 2024-09-21T03:03:26.406 INFO:teuthology.orchestra.run.smithi142.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.3 MB/s | 5.5 MB 00:00 2024-09-21T03:03:26.507 INFO:teuthology.orchestra.run.smithi142.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-09-21T03:03:26.609 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T03:03:26.765 INFO:teuthology.orchestra.run.smithi142.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.0 MB 00:00 2024-09-21T03:03:27.282 INFO:teuthology.orchestra.run.smithi142.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 12 MB/s | 20 MB 00:01 2024-09-21T03:03:27.332 INFO:teuthology.orchestra.run.smithi142.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T03:03:27.449 INFO:teuthology.orchestra.run.smithi142.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 15 MB/s | 10 MB 00:00 2024-09-21T03:03:27.518 INFO:teuthology.orchestra.run.smithi142.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.2 MB/s | 226 kB 00:00 2024-09-21T03:03:27.685 INFO:teuthology.orchestra.run.smithi142.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:03:27.711 INFO:teuthology.orchestra.run.smithi142.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 141 kB/s | 27 kB 00:00 2024-09-21T03:03:27.736 INFO:teuthology.orchestra.run.smithi142.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T03:03:27.778 INFO:teuthology.orchestra.run.smithi142.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T03:03:27.803 INFO:teuthology.orchestra.run.smithi142.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T03:03:27.828 INFO:teuthology.orchestra.run.smithi142.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:03:27.870 INFO:teuthology.orchestra.run.smithi142.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 7.4 MB/s | 507 kB 00:00 2024-09-21T03:03:27.937 INFO:teuthology.orchestra.run.smithi142.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 679 kB/s | 45 kB 00:00 2024-09-21T03:03:28.012 INFO:teuthology.orchestra.run.smithi142.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T03:03:28.112 INFO:teuthology.orchestra.run.smithi142.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 19 MB/s | 5.3 MB 00:00 2024-09-21T03:03:28.138 INFO:teuthology.orchestra.run.smithi142.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T03:03:28.163 INFO:teuthology.orchestra.run.smithi142.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T03:03:28.189 INFO:teuthology.orchestra.run.smithi142.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.9 MB/s | 307 kB 00:00 2024-09-21T03:03:28.214 INFO:teuthology.orchestra.run.smithi142.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:03:28.239 INFO:teuthology.orchestra.run.smithi142.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:03:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 170 kB 00:00 2024-09-21T03:03:28.364 INFO:teuthology.orchestra.run.smithi142.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T03:03:28.390 INFO:teuthology.orchestra.run.smithi142.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 373 kB/s | 34 kB 00:00 2024-09-21T03:03:28.415 INFO:teuthology.orchestra.run.smithi142.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:03:28.779 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:28.780 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:28.793 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:03:28.794 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:03:28.795 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:03:28.796 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:03:28.798 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:03:28.801 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:03:28.802 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:03:28.865 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T03:03:28.887 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T03:03:28.887 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:03:28.888 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:03:28.889 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:03:28.890 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:03:28.891 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:03:28.892 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:03:28.893 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:03:28.894 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:03:28.895 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:03:28.896 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:03:28.897 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:03:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:03:28.899 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:Install 149 Packages 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:03:28.900 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 228 M 2024-09-21T03:03:28.901 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T03:03:28.932 INFO:teuthology.orchestra.run.smithi142.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T03:03:28.990 INFO:teuthology.orchestra.run.smithi142.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T03:03:29.149 INFO:teuthology.orchestra.run.smithi142.stdout:(33/157): ceph-mgr-dashboard-19.3.0-5120.g18432 19 MB/s | 14 MB 00:00 2024-09-21T03:03:29.174 INFO:teuthology.orchestra.run.smithi142.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 268 kB/s | 49 kB 00:00 2024-09-21T03:03:29.199 INFO:teuthology.orchestra.run.smithi142.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T03:03:29.225 INFO:teuthology.orchestra.run.smithi142.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.5 MB/s | 285 kB 00:00 2024-09-21T03:03:29.258 INFO:teuthology.orchestra.run.smithi142.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T03:03:29.284 INFO:teuthology.orchestra.run.smithi142.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 314 kB/s | 18 kB 00:00 2024-09-21T03:03:29.626 INFO:teuthology.orchestra.run.smithi142.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T03:03:29.627 INFO:teuthology.orchestra.run.smithi193.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-21T03:03:29.734 INFO:teuthology.orchestra.run.smithi142.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-21T03:03:29.826 INFO:teuthology.orchestra.run.smithi142.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T03:03:29.852 INFO:teuthology.orchestra.run.smithi142.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-21T03:03:29.852 INFO:teuthology.orchestra.run.smithi193.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T03:03:29.877 INFO:teuthology.orchestra.run.smithi142.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 762 kB/s | 191 kB 00:00 2024-09-21T03:03:29.902 INFO:teuthology.orchestra.run.smithi142.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T03:03:29.903 INFO:teuthology.orchestra.run.smithi193.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.9 MB/s | 148 kB 00:00 2024-09-21T03:03:29.936 INFO:teuthology.orchestra.run.smithi142.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T03:03:29.961 INFO:teuthology.orchestra.run.smithi142.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T03:03:30.003 INFO:teuthology.orchestra.run.smithi142.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-09-21T03:03:30.028 INFO:teuthology.orchestra.run.smithi142.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-21T03:03:30.154 INFO:teuthology.orchestra.run.smithi142.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-09-21T03:03:30.161 INFO:teuthology.orchestra.run.smithi193.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.8 MB/s | 2.3 MB 00:00 2024-09-21T03:03:30.179 INFO:teuthology.orchestra.run.smithi142.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T03:03:30.204 INFO:teuthology.orchestra.run.smithi142.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-21T03:03:30.229 INFO:teuthology.orchestra.run.smithi142.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T03:03:30.236 INFO:teuthology.orchestra.run.smithi193.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.4 MB/s | 5.5 MB 00:00 2024-09-21T03:03:30.255 INFO:teuthology.orchestra.run.smithi142.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-21T03:03:30.288 INFO:teuthology.orchestra.run.smithi142.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-21T03:03:30.314 INFO:teuthology.orchestra.run.smithi142.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T03:03:30.347 INFO:teuthology.orchestra.run.smithi142.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-09-21T03:03:30.353 INFO:teuthology.orchestra.run.smithi193.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-09-21T03:03:30.373 INFO:teuthology.orchestra.run.smithi142.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-21T03:03:30.415 INFO:teuthology.orchestra.run.smithi142.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T03:03:30.430 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-21T03:03:30.440 INFO:teuthology.orchestra.run.smithi142.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-21T03:03:30.474 INFO:teuthology.orchestra.run.smithi142.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T03:03:30.532 INFO:teuthology.orchestra.run.smithi142.stdout:(61/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-09-21T03:03:30.557 INFO:teuthology.orchestra.run.smithi142.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 294 kB/s | 24 kB 00:00 2024-09-21T03:03:30.583 INFO:teuthology.orchestra.run.smithi142.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-21T03:03:30.587 INFO:teuthology.orchestra.run.smithi193.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.0 MB 00:00 2024-09-21T03:03:30.608 INFO:teuthology.orchestra.run.smithi142.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-21T03:03:30.633 INFO:teuthology.orchestra.run.smithi142.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T03:03:30.658 INFO:teuthology.orchestra.run.smithi142.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T03:03:30.684 INFO:teuthology.orchestra.run.smithi142.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi142.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-21T03:03:30.801 INFO:teuthology.orchestra.run.smithi142.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-21T03:03:30.835 INFO:teuthology.orchestra.run.smithi142.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-21T03:03:30.860 INFO:teuthology.orchestra.run.smithi142.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:03:30.927 INFO:teuthology.orchestra.run.smithi142.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-21T03:03:30.969 INFO:teuthology.orchestra.run.smithi142.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T03:03:31.011 INFO:teuthology.orchestra.run.smithi142.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-21T03:03:31.053 INFO:teuthology.orchestra.run.smithi142.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T03:03:31.088 INFO:teuthology.orchestra.run.smithi193.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 13 MB/s | 20 MB 00:01 2024-09-21T03:03:31.136 INFO:teuthology.orchestra.run.smithi142.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-21T03:03:31.137 INFO:teuthology.orchestra.run.smithi193.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T03:03:31.162 INFO:teuthology.orchestra.run.smithi142.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T03:03:31.187 INFO:teuthology.orchestra.run.smithi142.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-21T03:03:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T03:03:31.229 INFO:teuthology.orchestra.run.smithi142.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-09-21T03:03:31.254 INFO:teuthology.orchestra.run.smithi142.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-21T03:03:31.271 INFO:teuthology.orchestra.run.smithi142.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-09-21T03:03:31.296 INFO:teuthology.orchestra.run.smithi193.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 15 MB/s | 10 MB 00:00 2024-09-21T03:03:31.347 INFO:teuthology.orchestra.run.smithi142.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-21T03:03:31.363 INFO:teuthology.orchestra.run.smithi193.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T03:03:31.372 INFO:teuthology.orchestra.run.smithi142.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T03:03:31.397 INFO:teuthology.orchestra.run.smithi142.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-21T03:03:31.422 INFO:teuthology.orchestra.run.smithi142.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:03:31.447 INFO:teuthology.orchestra.run.smithi142.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-21T03:03:31.473 INFO:teuthology.orchestra.run.smithi142.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T03:03:31.498 INFO:teuthology.orchestra.run.smithi142.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:03:31.523 INFO:teuthology.orchestra.run.smithi142.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-09-21T03:03:31.547 INFO:teuthology.orchestra.run.smithi193.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T03:03:31.548 INFO:teuthology.orchestra.run.smithi142.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:03:31.572 INFO:teuthology.orchestra.run.smithi193.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 129 kB/s | 27 kB 00:00 2024-09-21T03:03:31.574 INFO:teuthology.orchestra.run.smithi142.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:03:31.597 INFO:teuthology.orchestra.run.smithi193.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T03:03:31.639 INFO:teuthology.orchestra.run.smithi193.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T03:03:31.657 INFO:teuthology.orchestra.run.smithi142.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-21T03:03:31.665 INFO:teuthology.orchestra.run.smithi193.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T03:03:31.682 INFO:teuthology.orchestra.run.smithi142.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-21T03:03:31.690 INFO:teuthology.orchestra.run.smithi193.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:03:31.708 INFO:teuthology.orchestra.run.smithi142.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-21T03:03:31.724 INFO:teuthology.orchestra.run.smithi193.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T03:03:31.733 INFO:teuthology.orchestra.run.smithi142.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T03:03:31.758 INFO:teuthology.orchestra.run.smithi142.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-21T03:03:31.784 INFO:teuthology.orchestra.run.smithi142.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-21T03:03:31.791 INFO:teuthology.orchestra.run.smithi193.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 678 kB/s | 45 kB 00:00 2024-09-21T03:03:31.809 INFO:teuthology.orchestra.run.smithi142.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T03:03:31.834 INFO:teuthology.orchestra.run.smithi142.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:03:31.859 INFO:teuthology.orchestra.run.smithi142.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:03:31.866 INFO:teuthology.orchestra.run.smithi193.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T03:03:31.884 INFO:teuthology.orchestra.run.smithi142.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:03:31.910 INFO:teuthology.orchestra.run.smithi142.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T03:03:31.950 INFO:teuthology.orchestra.run.smithi193.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T03:03:31.983 INFO:teuthology.orchestra.run.smithi193.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T03:03:32.017 INFO:teuthology.orchestra.run.smithi193.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T03:03:32.042 INFO:teuthology.orchestra.run.smithi193.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T03:03:32.068 INFO:teuthology.orchestra.run.smithi193.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:03:32.097 INFO:teuthology.orchestra.run.smithi193.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.6 MB/s | 86 kB 00:00 2024-09-21T03:03:32.152 INFO:teuthology.orchestra.run.smithi142.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T03:03:32.156 INFO:teuthology.orchestra.run.smithi193.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T03:03:32.177 INFO:teuthology.orchestra.run.smithi142.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-21T03:03:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-09-21T03:03:32.239 INFO:teuthology.orchestra.run.smithi193.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 18 MB/s | 3.1 MB 00:00 2024-09-21T03:03:32.265 INFO:teuthology.orchestra.run.smithi193.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 315 kB/s | 34 kB 00:00 2024-09-21T03:03:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-09-21T03:03:32.290 INFO:teuthology.orchestra.run.smithi193.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:03:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi142.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-21T03:03:32.379 INFO:teuthology.orchestra.run.smithi142.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-21T03:03:32.404 INFO:teuthology.orchestra.run.smithi142.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T03:03:32.429 INFO:teuthology.orchestra.run.smithi142.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-21T03:03:32.471 INFO:teuthology.orchestra.run.smithi142.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T03:03:32.497 INFO:teuthology.orchestra.run.smithi142.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:03:32.522 INFO:teuthology.orchestra.run.smithi142.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-21T03:03:32.547 INFO:teuthology.orchestra.run.smithi142.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T03:03:32.572 INFO:teuthology.orchestra.run.smithi142.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-21T03:03:32.598 INFO:teuthology.orchestra.run.smithi142.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T03:03:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T03:03:32.632 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:32.633 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:03:32.646 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:03:32.647 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:03:32.648 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:03:32.649 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:03:32.651 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:03:32.654 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:03:32.655 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:03:32.657 INFO:teuthology.orchestra.run.smithi142.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:03:32.682 INFO:teuthology.orchestra.run.smithi142.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:03:32.717 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-21T03:03:32.754 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:03:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:03:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:03:32.757 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:03:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:03:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:03:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:03:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:03:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:03:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:03:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:03:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:03:32.766 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:Install 149 Packages 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 8 Packages 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 228 M 2024-09-21T03:03:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-21T03:03:32.808 INFO:teuthology.orchestra.run.smithi193.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T03:03:32.875 INFO:teuthology.orchestra.run.smithi193.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T03:03:32.876 INFO:teuthology.orchestra.run.smithi142.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 745 kB/s | 144 kB 00:00 2024-09-21T03:03:32.899 INFO:teuthology.orchestra.run.smithi142.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 478 kB/s | 11 kB 00:00 2024-09-21T03:03:32.916 INFO:teuthology.orchestra.run.smithi142.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T03:03:32.933 INFO:teuthology.orchestra.run.smithi142.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T03:03:32.941 INFO:teuthology.orchestra.run.smithi193.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 744 kB/s | 49 kB 00:00 2024-09-21T03:03:32.958 INFO:teuthology.orchestra.run.smithi142.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-21T03:03:32.984 INFO:teuthology.orchestra.run.smithi142.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-21T03:03:33.051 INFO:teuthology.orchestra.run.smithi142.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-09-21T03:03:33.083 INFO:teuthology.orchestra.run.smithi193.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T03:03:33.101 INFO:teuthology.orchestra.run.smithi142.stdout:(130/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-09-21T03:03:33.108 INFO:teuthology.orchestra.run.smithi193.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 98 kB/s | 16 kB 00:00 2024-09-21T03:03:33.134 INFO:teuthology.orchestra.run.smithi193.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:03:33.134 INFO:teuthology.orchestra.run.smithi142.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-09-21T03:03:33.160 INFO:teuthology.orchestra.run.smithi142.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-21T03:03:33.185 INFO:teuthology.orchestra.run.smithi142.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-21T03:03:33.210 INFO:teuthology.orchestra.run.smithi142.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-21T03:03:33.243 INFO:teuthology.orchestra.run.smithi142.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-09-21T03:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-21T03:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-21T03:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 419 kB/s | 28 kB 00:00 2024-09-21T03:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-21T03:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T03:03:33.387 INFO:teuthology.orchestra.run.smithi142.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-21T03:03:33.389 INFO:teuthology.orchestra.run.smithi083.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-09-21T03:03:33.412 INFO:teuthology.orchestra.run.smithi142.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T03:03:33.438 INFO:teuthology.orchestra.run.smithi142.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T03:03:33.463 INFO:teuthology.orchestra.run.smithi142.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:03:33.488 INFO:teuthology.orchestra.run.smithi142.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T03:03:33.514 INFO:teuthology.orchestra.run.smithi142.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T03:03:33.518 INFO:teuthology.orchestra.run.smithi193.stdout:(37/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T03:03:33.539 INFO:teuthology.orchestra.run.smithi142.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:03:33.543 INFO:teuthology.orchestra.run.smithi193.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 45 kB/s | 18 kB 00:00 2024-09-21T03:03:33.572 INFO:teuthology.orchestra.run.smithi142.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T03:03:33.585 INFO:teuthology.orchestra.run.smithi193.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 1.6 MB/s | 784 kB 00:00 2024-09-21T03:03:33.623 INFO:teuthology.orchestra.run.smithi142.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:03:33.715 INFO:teuthology.orchestra.run.smithi083.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.6 MB/s | 873 kB 00:00 2024-09-21T03:03:33.790 INFO:teuthology.orchestra.run.smithi083.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 10 MB/s | 5.5 MB 00:00 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi083.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T03:03:33.898 INFO:teuthology.orchestra.run.smithi142.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 9.4 MB/s | 3.4 MB 00:00 2024-09-21T03:03:33.932 INFO:teuthology.orchestra.run.smithi083.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 16 MB/s | 2.3 MB 00:00 2024-09-21T03:03:33.948 INFO:teuthology.orchestra.run.smithi142.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 7.3 MB/s | 2.3 MB 00:00 2024-09-21T03:03:33.998 INFO:teuthology.orchestra.run.smithi142.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 819 kB/s | 41 kB 00:00 2024-09-21T03:03:34.049 INFO:teuthology.orchestra.run.smithi142.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T03:03:34.132 INFO:teuthology.orchestra.run.smithi142.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-21T03:03:34.177 INFO:teuthology.orchestra.run.smithi193.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 121 kB/s | 80 kB 00:00 2024-09-21T03:03:34.224 INFO:teuthology.orchestra.run.smithi083.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-21T03:03:34.242 INFO:teuthology.orchestra.run.smithi142.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.3 MB/s | 1.8 MB 00:00 2024-09-21T03:03:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:(7/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-09-21T03:03:34.291 INFO:teuthology.orchestra.run.smithi142.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-21T03:03:34.369 INFO:teuthology.orchestra.run.smithi193.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-21T03:03:34.416 INFO:teuthology.orchestra.run.smithi142.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 3.2 MB 00:00 2024-09-21T03:03:34.419 INFO:teuthology.orchestra.run.smithi193.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 915 kB/s | 801 kB 00:00 2024-09-21T03:03:34.422 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:03:34.423 INFO:teuthology.orchestra.run.smithi142.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T03:03:34.467 INFO:teuthology.orchestra.run.smithi083.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 16 MB/s | 20 MB 00:01 2024-09-21T03:03:34.500 INFO:teuthology.orchestra.run.smithi083.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 759 kB/s | 25 kB 00:00 2024-09-21T03:03:35.019 INFO:teuthology.orchestra.run.smithi193.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 301 kB/s | 253 kB 00:00 2024-09-21T03:03:35.034 INFO:teuthology.orchestra.run.smithi083.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 22 MB/s | 18 MB 00:00 2024-09-21T03:03:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T03:03:35.086 INFO:teuthology.orchestra.run.smithi193.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 162 kB/s | 106 kB 00:00 2024-09-21T03:03:35.160 INFO:teuthology.orchestra.run.smithi083.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 361 kB/s | 27 kB 00:00 2024-09-21T03:03:35.235 INFO:teuthology.orchestra.run.smithi083.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T03:03:35.311 INFO:teuthology.orchestra.run.smithi193.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-21T03:03:35.335 INFO:teuthology.orchestra.run.smithi083.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T03:03:35.477 INFO:teuthology.orchestra.run.smithi083.stdout:(15/157): ceph-radosgw-19.3.0-5120.g18432218.el 8.5 MB/s | 10 MB 00:01 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi083.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 950 kB/s | 158 kB 00:00 2024-09-21T03:03:35.528 INFO:teuthology.orchestra.run.smithi083.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T03:03:35.561 INFO:teuthology.orchestra.run.smithi083.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T03:03:35.620 INFO:teuthology.orchestra.run.smithi083.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 775 kB/s | 45 kB 00:00 2024-09-21T03:03:35.661 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-21T03:03:35.703 INFO:teuthology.orchestra.run.smithi083.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T03:03:35.712 INFO:teuthology.orchestra.run.smithi193.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 195 kB/s | 135 kB 00:00 2024-09-21T03:03:35.779 INFO:teuthology.orchestra.run.smithi193.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-21T03:03:35.787 INFO:teuthology.orchestra.run.smithi083.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T03:03:35.811 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-21T03:03:35.811 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-21T03:03:35.929 INFO:teuthology.orchestra.run.smithi083.stdout:(22/157): librgw2-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.3 MB 00:00 2024-09-21T03:03:35.954 INFO:teuthology.orchestra.run.smithi083.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 1.9 MB/s | 321 kB 00:00 2024-09-21T03:03:35.979 INFO:teuthology.orchestra.run.smithi083.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:03:35.995 INFO:teuthology.orchestra.run.smithi193.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T03:03:36.005 INFO:teuthology.orchestra.run.smithi083.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:03:36.030 INFO:teuthology.orchestra.run.smithi083.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:03:36.046 INFO:teuthology.orchestra.run.smithi193.stdout:(49/157): python3-urllib3-1.26.5-6.el9.noarch.r 293 kB/s | 215 kB 00:00 2024-09-21T03:03:36.071 INFO:teuthology.orchestra.run.smithi193.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-21T03:03:36.130 INFO:teuthology.orchestra.run.smithi193.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 261 kB/s | 22 kB 00:00 2024-09-21T03:03:36.163 INFO:teuthology.orchestra.run.smithi193.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-21T03:03:36.189 INFO:teuthology.orchestra.run.smithi193.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-21T03:03:36.214 INFO:teuthology.orchestra.run.smithi193.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-09-21T03:03:36.240 INFO:teuthology.orchestra.run.smithi193.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 360 kB/s | 18 kB 00:00 2024-09-21T03:03:36.256 INFO:teuthology.orchestra.run.smithi193.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 264 kB/s | 11 kB 00:00 2024-09-21T03:03:36.324 INFO:teuthology.orchestra.run.smithi193.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T03:03:36.374 INFO:teuthology.orchestra.run.smithi193.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-21T03:03:36.389 INFO:teuthology.orchestra.run.smithi083.stdout:(27/157): ceph-test-19.3.0-5120.g18432218.el9.x 21 MB/s | 40 MB 00:01 2024-09-21T03:03:36.391 INFO:teuthology.orchestra.run.smithi193.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-09-21T03:03:36.414 INFO:teuthology.orchestra.run.smithi083.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 444 kB/s | 170 kB 00:00 2024-09-21T03:03:36.431 INFO:teuthology.orchestra.run.smithi083.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 824 kB/s | 34 kB 00:00 2024-09-21T03:03:36.456 INFO:teuthology.orchestra.run.smithi083.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.6 MB/s | 154 kB 00:00 2024-09-21T03:03:36.507 INFO:teuthology.orchestra.run.smithi083.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 6.2 MB/s | 3.1 MB 00:00 2024-09-21T03:03:36.566 INFO:teuthology.orchestra.run.smithi193.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-21T03:03:36.591 INFO:teuthology.orchestra.run.smithi193.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-21T03:03:36.598 INFO:teuthology.orchestra.run.smithi083.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T03:03:36.608 INFO:teuthology.orchestra.run.smithi193.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 582 kB/s | 24 kB 00:00 2024-09-21T03:03:36.633 INFO:teuthology.orchestra.run.smithi193.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T03:03:36.665 INFO:teuthology.orchestra.run.smithi083.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 742 kB/s | 49 kB 00:00 2024-09-21T03:03:36.717 INFO:teuthology.orchestra.run.smithi193.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T03:03:36.743 INFO:teuthology.orchestra.run.smithi193.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-21T03:03:36.749 INFO:teuthology.orchestra.run.smithi083.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 198 kB/s | 16 kB 00:00 2024-09-21T03:03:36.759 INFO:teuthology.orchestra.run.smithi193.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-21T03:03:36.949 INFO:teuthology.orchestra.run.smithi083.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 27 MB/s | 14 MB 00:00 2024-09-21T03:03:36.975 INFO:teuthology.orchestra.run.smithi083.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.2 MB/s | 285 kB 00:00 2024-09-21T03:03:37.008 INFO:teuthology.orchestra.run.smithi083.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T03:03:37.027 INFO:teuthology.orchestra.run.smithi193.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-09-21T03:03:37.034 INFO:teuthology.orchestra.run.smithi083.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 314 kB/s | 18 kB 00:00 2024-09-21T03:03:37.126 INFO:teuthology.orchestra.run.smithi083.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 11 MB/s | 7.4 MB 00:00 2024-09-21T03:03:37.251 INFO:teuthology.orchestra.run.smithi083.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T03:03:37.276 INFO:teuthology.orchestra.run.smithi083.stdout:(41/157): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-21T03:03:37.326 INFO:teuthology.orchestra.run.smithi083.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-21T03:03:37.393 INFO:teuthology.orchestra.run.smithi083.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-21T03:03:37.419 INFO:teuthology.orchestra.run.smithi083.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-21T03:03:37.452 INFO:teuthology.orchestra.run.smithi083.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-09-21T03:03:37.486 INFO:teuthology.orchestra.run.smithi083.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-21T03:03:37.502 INFO:teuthology.orchestra.run.smithi193.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-09-21T03:03:37.511 INFO:teuthology.orchestra.run.smithi083.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-21T03:03:37.538 INFO:teuthology.orchestra.run.smithi083.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T03:03:37.626 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-21T03:03:37.627 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-21T03:03:37.702 INFO:teuthology.orchestra.run.smithi193.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-21T03:03:37.794 INFO:teuthology.orchestra.run.smithi193.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.7 MB/s | 4.9 MB 00:01 2024-09-21T03:03:37.819 INFO:teuthology.orchestra.run.smithi193.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T03:03:37.887 INFO:teuthology.orchestra.run.smithi193.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 2.3 MB/s | 910 kB 00:00 2024-09-21T03:03:37.945 INFO:teuthology.orchestra.run.smithi193.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:03:37.995 INFO:teuthology.orchestra.run.smithi083.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-21T03:03:38.071 INFO:teuthology.orchestra.run.smithi193.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-21T03:03:38.112 INFO:teuthology.orchestra.run.smithi083.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-21T03:03:38.120 INFO:teuthology.orchestra.run.smithi193.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T03:03:38.204 INFO:teuthology.orchestra.run.smithi193.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-21T03:03:38.262 INFO:teuthology.orchestra.run.smithi083.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-09-21T03:03:38.296 INFO:teuthology.orchestra.run.smithi193.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-21T03:03:38.346 INFO:teuthology.orchestra.run.smithi193.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-21T03:03:38.371 INFO:teuthology.orchestra.run.smithi083.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-21T03:03:38.396 INFO:teuthology.orchestra.run.smithi083.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-21T03:03:38.397 INFO:teuthology.orchestra.run.smithi193.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-21T03:03:38.455 INFO:teuthology.orchestra.run.smithi193.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T03:03:38.496 INFO:teuthology.orchestra.run.smithi083.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 816 kB/s | 313 kB 00:00 2024-09-21T03:03:38.505 INFO:teuthology.orchestra.run.smithi193.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-21T03:03:38.521 INFO:teuthology.orchestra.run.smithi083.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-21T03:03:38.605 INFO:teuthology.orchestra.run.smithi083.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-21T03:03:38.639 INFO:teuthology.orchestra.run.smithi083.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-21T03:03:38.714 INFO:teuthology.orchestra.run.smithi083.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-09-21T03:03:38.739 INFO:teuthology.orchestra.run.smithi083.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T03:03:38.756 INFO:teuthology.orchestra.run.smithi083.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-09-21T03:03:38.815 INFO:teuthology.orchestra.run.smithi193.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 5.7 MB/s | 5.8 MB 00:01 2024-09-21T03:03:38.865 INFO:teuthology.orchestra.run.smithi083.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-21T03:03:38.898 INFO:teuthology.orchestra.run.smithi193.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-21T03:03:38.940 INFO:teuthology.orchestra.run.smithi083.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-21T03:03:38.957 INFO:teuthology.orchestra.run.smithi193.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T03:03:38.966 INFO:teuthology.orchestra.run.smithi083.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-09-21T03:03:39.015 INFO:teuthology.orchestra.run.smithi193.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T03:03:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T03:03:39.066 INFO:teuthology.orchestra.run.smithi193.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-21T03:03:39.083 INFO:teuthology.orchestra.run.smithi083.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T03:03:39.149 INFO:teuthology.orchestra.run.smithi193.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-21T03:03:39.158 INFO:teuthology.orchestra.run.smithi083.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-21T03:03:39.208 INFO:teuthology.orchestra.run.smithi193.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T03:03:39.292 INFO:teuthology.orchestra.run.smithi193.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-21T03:03:39.384 INFO:teuthology.orchestra.run.smithi193.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 5.1 MB 00:00 2024-09-21T03:03:39.400 INFO:teuthology.orchestra.run.smithi193.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-09-21T03:03:39.426 INFO:teuthology.orchestra.run.smithi193.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-21T03:03:39.468 INFO:teuthology.orchestra.run.smithi193.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T03:03:39.510 INFO:teuthology.orchestra.run.smithi193.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T03:03:39.602 INFO:teuthology.orchestra.run.smithi193.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-09-21T03:03:39.677 INFO:teuthology.orchestra.run.smithi193.stdout:(96/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-21T03:03:39.694 INFO:teuthology.orchestra.run.smithi193.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-09-21T03:03:39.719 INFO:teuthology.orchestra.run.smithi193.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T03:03:39.761 INFO:teuthology.orchestra.run.smithi193.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T03:03:39.767 INFO:teuthology.orchestra.run.smithi083.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-21T03:03:39.786 INFO:teuthology.orchestra.run.smithi193.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-21T03:03:39.800 INFO:teuthology.orchestra.run.smithi083.stdout:(68/157): librdkafka-1.6.1-102.el9.x86_64.rpm 610 kB/s | 662 kB 00:01 2024-09-21T03:03:39.820 INFO:teuthology.orchestra.run.smithi193.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T03:03:39.837 INFO:teuthology.orchestra.run.smithi193.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:03:39.863 INFO:teuthology.orchestra.run.smithi193.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-21T03:03:39.938 INFO:teuthology.orchestra.run.smithi193.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T03:03:39.984 INFO:teuthology.orchestra.run.smithi083.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-21T03:03:39.988 INFO:teuthology.orchestra.run.smithi193.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T03:03:40.030 INFO:teuthology.orchestra.run.smithi193.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-21T03:03:40.047 INFO:teuthology.orchestra.run.smithi193.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-21T03:03:40.109 INFO:teuthology.orchestra.run.smithi083.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T03:03:40.156 INFO:teuthology.orchestra.run.smithi193.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T03:03:40.190 INFO:teuthology.orchestra.run.smithi193.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-09-21T03:03:40.215 INFO:teuthology.orchestra.run.smithi193.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 441 kB/s | 26 kB 00:00 2024-09-21T03:03:40.249 INFO:teuthology.orchestra.run.smithi193.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-21T03:03:40.274 INFO:teuthology.orchestra.run.smithi193.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-21T03:03:40.299 INFO:teuthology.orchestra.run.smithi193.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-21T03:03:40.341 INFO:teuthology.orchestra.run.smithi193.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T03:03:40.367 INFO:teuthology.orchestra.run.smithi193.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:03:40.392 INFO:teuthology.orchestra.run.smithi193.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T03:03:40.417 INFO:teuthology.orchestra.run.smithi193.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T03:03:40.459 INFO:teuthology.orchestra.run.smithi193.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-21T03:03:40.484 INFO:teuthology.orchestra.run.smithi193.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-21T03:03:40.518 INFO:teuthology.orchestra.run.smithi193.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T03:03:40.543 INFO:teuthology.orchestra.run.smithi193.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T03:03:40.569 INFO:teuthology.orchestra.run.smithi193.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-21T03:03:40.602 INFO:teuthology.orchestra.run.smithi193.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:03:40.627 INFO:teuthology.orchestra.run.smithi193.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T03:03:40.653 INFO:teuthology.orchestra.run.smithi193.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-21T03:03:40.670 INFO:teuthology.orchestra.run.smithi193.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-21T03:03:40.687 INFO:teuthology.orchestra.run.smithi193.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T03:03:40.737 INFO:teuthology.orchestra.run.smithi193.stdout:(128/157): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-21T03:03:40.754 INFO:teuthology.orchestra.run.smithi193.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-09-21T03:03:40.779 INFO:teuthology.orchestra.run.smithi193.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:03:40.805 INFO:teuthology.orchestra.run.smithi193.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:03:40.809 INFO:teuthology.orchestra.run.smithi083.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.9 MB/s | 4.9 MB 00:01 2024-09-21T03:03:40.822 INFO:teuthology.orchestra.run.smithi193.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T03:03:40.855 INFO:teuthology.orchestra.run.smithi193.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-21T03:03:40.872 INFO:teuthology.orchestra.run.smithi193.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 989 kB/s | 16 kB 00:00 2024-09-21T03:03:40.885 INFO:teuthology.orchestra.run.smithi083.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 7.4 MB/s | 5.8 MB 00:00 2024-09-21T03:03:40.898 INFO:teuthology.orchestra.run.smithi193.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-21T03:03:40.923 INFO:teuthology.orchestra.run.smithi193.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:03:40.940 INFO:teuthology.orchestra.run.smithi193.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi193.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 913 kB/s | 31 kB 00:00 2024-09-21T03:03:40.974 INFO:teuthology.orchestra.run.smithi193.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-21T03:03:40.999 INFO:teuthology.orchestra.run.smithi193.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-21T03:03:41.010 INFO:teuthology.orchestra.run.smithi083.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 700 kB/s | 87 kB 00:00 2024-09-21T03:03:41.017 INFO:teuthology.orchestra.run.smithi193.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 850 kB/s | 36 kB 00:00 2024-09-21T03:03:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-21T03:03:41.067 INFO:teuthology.orchestra.run.smithi193.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:03:41.092 INFO:teuthology.orchestra.run.smithi193.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T03:03:41.118 INFO:teuthology.orchestra.run.smithi193.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 796 kB/s | 20 kB 00:00 2024-09-21T03:03:41.143 INFO:teuthology.orchestra.run.smithi193.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T03:03:41.169 INFO:teuthology.orchestra.run.smithi193.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:03:41.219 INFO:teuthology.orchestra.run.smithi193.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T03:03:41.244 INFO:teuthology.orchestra.run.smithi083.stdout:(74/157): postgresql-contrib-13.16-1.el9.x86_64 630 kB/s | 910 kB 00:01 2024-09-21T03:03:41.352 INFO:teuthology.orchestra.run.smithi083.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-21T03:03:41.419 INFO:teuthology.orchestra.run.smithi193.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T03:03:41.494 INFO:teuthology.orchestra.run.smithi193.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T03:03:41.727 INFO:teuthology.orchestra.run.smithi083.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-21T03:03:41.803 INFO:teuthology.orchestra.run.smithi083.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-09-21T03:03:41.911 INFO:teuthology.orchestra.run.smithi083.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-21T03:03:41.937 INFO:teuthology.orchestra.run.smithi193.stdout:(151/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-09-21T03:03:42.028 INFO:teuthology.orchestra.run.smithi083.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-21T03:03:42.145 INFO:teuthology.orchestra.run.smithi083.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T03:03:42.170 INFO:teuthology.orchestra.run.smithi083.stdout:(81/157): python3-jinja2-2.11.3-6.el9.noarch.rp 561 kB/s | 248 kB 00:00 2024-09-21T03:03:42.254 INFO:teuthology.orchestra.run.smithi083.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-21T03:03:42.371 INFO:teuthology.orchestra.run.smithi083.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T03:03:42.479 INFO:teuthology.orchestra.run.smithi083.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-21T03:03:42.503 INFO:teuthology.orchestra.run.smithi193.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 72 kB/s | 41 kB 00:00 2024-09-21T03:03:42.521 INFO:teuthology.orchestra.run.smithi083.stdout:(85/157): protobuf-3.14.0-13.el9.x86_64.rpm 604 kB/s | 1.0 MB 00:01 2024-09-21T03:03:42.579 INFO:teuthology.orchestra.run.smithi193.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-09-21T03:03:42.597 INFO:teuthology.orchestra.run.smithi083.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 660 kB/s | 77 kB 00:00 2024-09-21T03:03:42.620 INFO:teuthology.orchestra.run.smithi193.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.6 MB/s | 1.8 MB 00:01 2024-09-21T03:03:42.630 INFO:teuthology.orchestra.run.smithi083.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-21T03:03:42.706 INFO:teuthology.orchestra.run.smithi083.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-21T03:03:42.822 INFO:teuthology.orchestra.run.smithi083.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T03:03:42.931 INFO:teuthology.orchestra.run.smithi083.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-21T03:03:42.956 INFO:teuthology.orchestra.run.smithi083.stdout:(91/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 487 kB/s | 159 kB 00:00 2024-09-21T03:03:43.040 INFO:teuthology.orchestra.run.smithi083.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-09-21T03:03:43.148 INFO:teuthology.orchestra.run.smithi083.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-21T03:03:43.257 INFO:teuthology.orchestra.run.smithi083.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-21T03:03:43.262 INFO:teuthology.orchestra.run.smithi193.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 328 kB/s | 249 kB 00:00 2024-09-21T03:03:43.374 INFO:teuthology.orchestra.run.smithi083.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-21T03:03:43.491 INFO:teuthology.orchestra.run.smithi083.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-21T03:03:43.546 INFO:teuthology.orchestra.run.smithi193.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-21T03:03:43.608 INFO:teuthology.orchestra.run.smithi083.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-21T03:03:43.721 INFO:teuthology.orchestra.run.smithi193.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-09-21T03:03:43.725 INFO:teuthology.orchestra.run.smithi083.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.6 MB/s | 304 kB 00:00 2024-09-21T03:03:43.728 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:03:43.728 INFO:teuthology.orchestra.run.smithi193.stdout:Total 15 MB/s | 228 MB 00:14 2024-09-21T03:03:43.834 INFO:teuthology.orchestra.run.smithi083.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-09-21T03:03:43.858 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-21T03:03:43.942 INFO:teuthology.orchestra.run.smithi083.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 884 kB/s | 96 kB 00:00 2024-09-21T03:03:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:03:44.082 INFO:teuthology.orchestra.run.smithi083.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 456 kB/s | 64 kB 00:00 2024-09-21T03:03:44.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:03:44.193 INFO:teuthology.orchestra.run.smithi083.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 405 kB/s | 45 kB 00:00 2024-09-21T03:03:44.253 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:03:44.260 INFO:teuthology.orchestra.run.smithi083.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 336 kB/s | 22 kB 00:00 2024-09-21T03:03:44.287 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:03:44.313 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:03:44.364 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:03:44.377 INFO:teuthology.orchestra.run.smithi083.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-09-21T03:03:44.427 INFO:teuthology.orchestra.run.smithi083.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T03:03:44.469 INFO:teuthology.orchestra.run.smithi083.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-21T03:03:44.486 INFO:teuthology.orchestra.run.smithi083.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-21T03:03:44.595 INFO:teuthology.orchestra.run.smithi083.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T03:03:44.611 INFO:teuthology.orchestra.run.smithi083.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T03:03:44.628 INFO:teuthology.orchestra.run.smithi083.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-21T03:03:44.654 INFO:teuthology.orchestra.run.smithi083.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T03:03:44.679 INFO:teuthology.orchestra.run.smithi083.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T03:03:44.704 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:03:44.729 INFO:teuthology.orchestra.run.smithi083.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-21T03:03:44.763 INFO:teuthology.orchestra.run.smithi083.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T03:03:44.780 INFO:teuthology.orchestra.run.smithi083.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T03:03:44.797 INFO:teuthology.orchestra.run.smithi083.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T03:03:44.814 INFO:teuthology.orchestra.run.smithi083.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 841 kB/s | 14 kB 00:00 2024-09-21T03:03:44.822 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:03:44.839 INFO:teuthology.orchestra.run.smithi083.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-21T03:03:44.872 INFO:teuthology.orchestra.run.smithi083.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T03:03:44.906 INFO:teuthology.orchestra.run.smithi083.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-21T03:03:44.932 INFO:teuthology.orchestra.run.smithi083.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T03:03:44.948 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:03:44.965 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T03:03:44.973 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:03:44.998 INFO:teuthology.orchestra.run.smithi083.stdout:(122/157): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2024-09-21T03:03:45.024 INFO:teuthology.orchestra.run.smithi083.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T03:03:45.049 INFO:teuthology.orchestra.run.smithi083.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T03:03:45.063 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:03:45.074 INFO:teuthology.orchestra.run.smithi083.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-21T03:03:45.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:03:45.100 INFO:teuthology.orchestra.run.smithi083.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 919 kB/s | 23 kB 00:00 2024-09-21T03:03:45.114 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:03:45.118 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T03:03:45.119 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T03:03:45.125 INFO:teuthology.orchestra.run.smithi083.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-21T03:03:45.150 INFO:teuthology.orchestra.run.smithi083.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-21T03:03:45.200 INFO:teuthology.orchestra.run.smithi083.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-21T03:03:45.226 INFO:teuthology.orchestra.run.smithi083.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:03:45.251 INFO:teuthology.orchestra.run.smithi083.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:03:45.267 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:03:45.276 INFO:teuthology.orchestra.run.smithi083.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:03:45.310 INFO:teuthology.orchestra.run.smithi083.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T03:03:45.316 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:03:45.335 INFO:teuthology.orchestra.run.smithi083.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-21T03:03:45.344 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:03:45.360 INFO:teuthology.orchestra.run.smithi083.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T03:03:45.386 INFO:teuthology.orchestra.run.smithi083.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:03:45.411 INFO:teuthology.orchestra.run.smithi083.stdout:(137/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-21T03:03:45.437 INFO:teuthology.orchestra.run.smithi083.stdout:(138/157): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-21T03:03:45.450 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:03:45.461 INFO:teuthology.orchestra.run.smithi083.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:03:45.479 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:03:45.487 INFO:teuthology.orchestra.run.smithi083.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T03:03:45.512 INFO:teuthology.orchestra.run.smithi083.stdout:(141/157): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-21T03:03:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:(142/157): python3-webob-1.8.8-2.el9.noarch.rpm 9.1 MB/s | 230 kB 00:00 2024-09-21T03:03:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:03:45.554 INFO:teuthology.orchestra.run.smithi083.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-21T03:03:45.563 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:03:45.588 INFO:teuthology.orchestra.run.smithi083.stdout:(144/157): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-09-21T03:03:45.590 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:03:45.605 INFO:teuthology.orchestra.run.smithi083.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T03:03:45.630 INFO:teuthology.orchestra.run.smithi083.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T03:03:45.689 INFO:teuthology.orchestra.run.smithi083.stdout:(147/157): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-21T03:03:45.748 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:03:45.888 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:03:45.939 INFO:teuthology.orchestra.run.smithi083.stdout:(148/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T03:03:46.019 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:03:46.073 INFO:teuthology.orchestra.run.smithi083.stdout:(149/157): librbd1-19.3.0-5120.g18432218.el9.x8 24 MB/s | 3.2 MB 00:00 2024-09-21T03:03:46.118 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:46.188 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:46.198 INFO:teuthology.orchestra.run.smithi083.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 19 MB/s | 2.3 MB 00:00 2024-09-21T03:03:46.315 INFO:teuthology.orchestra.run.smithi083.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-21T03:03:46.382 INFO:teuthology.orchestra.run.smithi083.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 614 kB/s | 41 kB 00:00 2024-09-21T03:03:46.425 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:46.457 INFO:teuthology.orchestra.run.smithi083.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-09-21T03:03:46.534 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:03:46.549 INFO:teuthology.orchestra.run.smithi083.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-21T03:03:46.566 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:03:46.601 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:03:46.658 INFO:teuthology.orchestra.run.smithi083.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-21T03:03:46.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:03:46.710 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:03:46.742 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:03:46.786 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:03:46.818 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:03:46.847 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:03:46.905 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:03:46.936 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T03:03:46.936 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T03:03:46.959 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:03:46.991 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:03:47.026 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:03:47.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:03:47.145 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:03:47.188 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:03:47.217 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:03:47.409 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:03:47.542 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:03:47.596 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:03:48.115 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:03:48.208 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:03:48.245 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:03:48.288 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:03:48.326 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:03:48.368 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:03:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:03:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:03:48.679 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:03:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:03:48.817 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:03:48.848 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:03:48.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:03:48.930 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:03:48.966 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:03:49.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:03:49.052 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:03:49.204 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:03:49.262 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:03:49.303 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:03:49.403 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:03:49.437 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:03:49.477 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:03:49.502 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:03:49.527 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:03:49.600 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:03:49.801 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:03:49.834 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:03:49.862 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:03:49.896 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:03:49.936 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:03:49.967 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:03:49.997 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:03:50.031 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:03:50.065 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:03:50.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:03:50.141 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:03:50.226 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:03:50.258 INFO:teuthology.orchestra.run.smithi083.stdout:(156/157): python3-numpy-1.20.1-5.el9.x86_64.rp 645 kB/s | 5.1 MB 00:08 2024-09-21T03:03:50.279 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:03:50.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:03:50.412 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:03:50.460 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:03:51.173 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:03:51.242 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:03:51.275 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:03:51.374 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:03:51.422 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:03:51.467 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:03:51.546 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:03:51.578 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:03:51.762 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:03:51.792 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:03:51.869 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:03:52.507 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:03:52.631 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:03:53.415 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T03:03:53.525 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:03:53.571 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:03:53.624 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:03:53.659 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:03:53.684 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:03:53.727 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:03:53.829 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:03:53.912 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:03:53.940 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:03:53.973 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:03:54.059 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:03:54.177 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:03:54.296 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:03:54.303 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:03:54.336 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:03:54.337 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:03:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:03:54.428 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:03:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:03:54.458 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:03:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:03:54.487 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:03:54.649 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:03:54.706 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:03:54.734 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:03:54.841 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:03:54.868 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:03:54.931 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:03:54.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:03:54.974 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:03:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:03:55.006 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:03:55.076 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:03:55.099 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:03:55.180 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:03:55.306 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:03:55.426 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:03:55.516 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:55.590 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:55.809 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:03:55.934 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:03:55.961 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:03:55.997 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:03:56.070 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:03:56.123 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:03:56.171 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:03:56.223 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:03:56.255 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:03:56.283 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:03:56.353 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:03:56.406 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:03:56.436 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:03:56.473 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:03:56.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:03:56.600 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:03:56.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:03:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:03:56.752 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:03:56.795 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:03:56.869 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:03:57.026 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:03:57.081 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:03:57.330 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:03:57.362 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:03:57.597 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:03:57.678 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:03:57.705 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:03:57.739 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:03:57.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:03:57.824 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:03:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:03:57.965 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:03:57.976 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:03:58.027 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:03:58.058 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:03:58.058 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:03:58.058 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:03:58.058 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:03:58.091 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:03:58.118 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:03:58.183 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:03:58.265 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:03:58.312 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:03:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:03:58.396 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:03:58.432 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:03:58.475 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:03:58.517 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:03:58.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:03:58.694 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:03:58.735 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:03:58.845 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:03:58.885 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:03:58.925 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:03:59.177 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:03:59.260 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:03:59.344 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:03:59.472 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:03:59.517 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:03:59.545 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:03:59.579 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:03:59.618 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:03:59.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:03:59.682 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:03:59.716 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:03:59.750 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:03:59.791 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:03:59.826 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:03:59.914 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:03:59.956 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:03:59.996 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:04:00.101 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:04:00.155 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:04:00.885 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:04:00.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:04:00.978 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:04:01.079 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:04:01.134 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:04:01.178 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:04:01.267 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:04:01.299 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:04:01.482 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:04:01.513 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:04:01.591 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:04:02.238 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:04:02.354 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:04:03.587 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:04:03.676 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:04:03.704 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:04:03.738 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:04:04.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:04:04.117 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:04:04.216 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:04:04.256 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:04:04.292 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:04:04.791 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:04:04.820 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:04:04.891 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:04:04.914 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:04:07.040 INFO:teuthology.orchestra.run.smithi083.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 691 kB/s | 16 MB 00:24 2024-09-21T03:04:07.043 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:04:07.044 INFO:teuthology.orchestra.run.smithi083.stdout:Total 6.6 MB/s | 228 MB 00:34 2024-09-21T03:04:07.154 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:07.194 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:07.764 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:07.806 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:04:07.906 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:04:07.906 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:04:07.906 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:04:07.906 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:07.941 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:04:08.242 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-21T03:04:08.394 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-21T03:04:08.395 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-21T03:04:09.849 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:04:09.849 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-09-21T03:04:09.849 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-09-21T03:04:09.850 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:10.029 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:10.046 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:10.046 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:10.046 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:04:10.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:10.667 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-21T03:04:10.668 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-21T03:04:11.628 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:11.655 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:11.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:04:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:12.375 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:12.399 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:12.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:04:12.545 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:04:12.600 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:04:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:04:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:04:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:04:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:04:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:04:13.159 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:04:13.190 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:04:13.218 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:04:13.687 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:13.718 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:13.749 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:04:13.806 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:04:13.832 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:16.992 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:17.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:17.056 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-21T03:04:17.195 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:04:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:04:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:04:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:04:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:04:17.394 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:04:17.748 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:04:17.865 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:04:17.984 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:04:18.011 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:18.016 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:04:18.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:18.107 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:04:18.138 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:04:18.160 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:18.168 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:04:18.330 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:04:18.357 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:18.357 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:04:18.380 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:18.381 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:18.381 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:04:18.381 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:18.381 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:18.381 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:18.410 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:04:18.432 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:18.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:04:18.444 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:18.527 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:04:18.543 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:04:18.556 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:04:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:04:18.609 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:04:18.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:04:18.657 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:04:18.821 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:04:19.006 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:04:19.122 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:04:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:04:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-09-21T03:04:19.503 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-09-21T03:04:19.504 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-09-21T03:04:19.504 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-09-21T03:04:19.504 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:19.543 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:04:19.548 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:04:19.576 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:19.601 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:19.644 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:19.662 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:04:19.667 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:19.667 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:19.667 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:04:19.667 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:19.680 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:19.694 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:04:19.697 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:19.697 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:19.697 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:04:19.697 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:19.719 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:19.728 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:19.743 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:19.792 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:04:19.838 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:04:19.878 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:04:19.921 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:04:19.955 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:04:19.982 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:04:20.050 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:04:20.062 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:20.085 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:20.085 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:20.085 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:04:20.085 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:20.086 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:20.086 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:20.104 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:04:20.144 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:04:20.188 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:04:20.267 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:04:20.298 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:04:20.341 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:04:20.370 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:04:20.442 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:04:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:04:20.568 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:04:20.703 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:04:20.754 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:04:21.284 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:21.311 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:21.384 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:04:21.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:04:21.456 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:04:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:04:21.535 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:21.544 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:04:21.558 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:21.559 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:21.559 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:04:21.559 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:21.559 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:21.559 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:21.722 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:04:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:04:21.870 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:04:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:04:22.009 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:04:22.027 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:22.040 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:22.051 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:22.076 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:04:22.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:04:22.157 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:04:22.176 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:04:22.219 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:04:22.222 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:04:22.261 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:04:22.286 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:04:22.398 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:04:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:04:22.474 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:04:22.480 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:04:22.520 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:04:22.581 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:04:22.599 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:04:22.622 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:04:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:04:22.703 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:04:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:04:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:04:22.836 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:04:22.851 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:04:22.887 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:04:22.919 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:04:22.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:04:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:04:22.994 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:04:23.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:04:23.060 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:04:23.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:04:23.139 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:04:23.169 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:04:23.203 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:04:23.237 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:04:23.287 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:04:23.321 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:04:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:23.407 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:04:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:23.445 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:04:23.451 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:04:23.483 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:04:23.511 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:04:23.536 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:23.585 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:04:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:04:24.049 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:04:24.091 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:04:24.130 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:04:24.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:04:24.190 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:04:24.369 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:04:24.370 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:04:24.412 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:04:24.414 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:04:24.442 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:04:24.448 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:04:24.480 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:04:24.526 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:04:24.549 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:04:24.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:04:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:04:24.753 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:04:24.792 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:04:24.975 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:04:25.007 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:04:25.084 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:04:25.104 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:25.141 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:25.161 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:04:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:04:25.981 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:04:26.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:04:26.111 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:04:26.191 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:04:26.240 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:04:26.265 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:04:26.291 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:04:26.291 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:04:26.367 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:04:26.367 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:04:26.442 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:04:26.467 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:04:26.473 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:26.473 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:26.653 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:26.653 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:04:26.739 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:26.745 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:04:26.800 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:26.807 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:27.124 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:04:27.208 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:04:27.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:04:27.268 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:04:27.593 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:04:27.632 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:04:27.706 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:04:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:04:27.765 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:27.773 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:04:27.792 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:27.907 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:28.069 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:04:28.103 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:28.127 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:28.128 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:28.128 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:04:28.128 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:28.128 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:28.128 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:28.179 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:28.191 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:28.269 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:04:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:04:28.301 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:04:28.343 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:04:28.372 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:04:28.403 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:04:29.309 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:04:29.348 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:29.371 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:29.372 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:29.372 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:04:29.372 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:29.372 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:29.372 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:29.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:29.446 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:29.446 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:29.447 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:04:29.447 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:29.499 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:29.523 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:29.850 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:30.634 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:31.253 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:04:31.303 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:04:31.393 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:04:31.394 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:04:31.394 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:04:31.394 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:31.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:04:32.751 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:04:32.752 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:04:32.753 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:04:32.765 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:04:32.766 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:04:32.767 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:04:32.768 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:04:32.769 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:04:32.770 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:04:32.771 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:04:32.772 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:04:32.773 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:04:32.774 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:04:32.775 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:04:33.846 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:04:33.896 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:04:33.936 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:04:33.969 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:04:33.996 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:04:34.186 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:04:34.235 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:04:34.265 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:04:34.295 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:04:34.333 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:04:34.963 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:34.998 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:35.019 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:35.473 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-21T03:04:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.475 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.476 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.477 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:04:35.478 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:04:35.479 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:04:35.480 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:04:35.481 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:04:35.482 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.483 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:04:35.484 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:04:35.485 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:04:35.486 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:04:35.486 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:04:35.486 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-21T03:04:35.858 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:04:35.925 DEBUG:teuthology.parallel:result is None 2024-09-21T03:04:35.975 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:04:36.020 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:04:36.118 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:04:36.148 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:04:36.174 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:04:36.200 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:04:36.200 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:04:36.284 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:04:36.284 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:04:36.352 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:04:36.377 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:04:36.383 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:36.383 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:36.570 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:04:36.571 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:04:36.654 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:04:36.719 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:43.102 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:04:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:04:43.104 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:04:43.105 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:04:43.106 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:04:43.107 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:04:43.108 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:04:43.109 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:04:43.110 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:04:43.111 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:04:43.112 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:04:43.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:04:43.114 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:04:43.115 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:04:43.115 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:04:43.115 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:04:43.115 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:04:43.371 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:04:43.371 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-09-21T03:04:43.371 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-09-21T03:04:43.372 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:43.601 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:04:43.617 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:43.617 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:04:43.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:04:45.247 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:45.248 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:04:45.248 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:45.248 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:04:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:45.494 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:04:45.494 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:45.494 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:04:45.495 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:45.495 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:04:45.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:45.646 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:04:45.649 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:04:45.650 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:04:45.651 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:04:45.652 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:04:45.653 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:04:45.654 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:04:45.655 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:04:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:04:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:04:45.658 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T03:04:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:04:45.979 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:45.979 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:04:45.979 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:45.979 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:04:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:46.080 DEBUG:teuthology.parallel:result is None 2024-09-21T03:04:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:04:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:04:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:04:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:04:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:04:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:04:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:04:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:04:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:04:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:04:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:04:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:04:47.381 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:04:47.446 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:04:47.471 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:50.686 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:04:50.753 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:04:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:04:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:04:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:04:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:04:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:04:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:04:53.272 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:04:53.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:04:53.352 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:53.392 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:53.414 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:04:53.414 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:53.414 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:04:53.414 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:53.468 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:04:53.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:53.842 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:04:53.842 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:04:53.842 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:04:53.842 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:53.842 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:04:53.843 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:04:57.873 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:04:57.914 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:04:57.955 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:04:57.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:04:58.014 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:04:58.229 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:04:58.278 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:04:58.308 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:04:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:04:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:04:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:04:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:04:59.945 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:04:59.996 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:05:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:05:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:05:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:05:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:05:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:05:00.275 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:05:00.275 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:05:00.360 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:05:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:05:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:05:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:05:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:05:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:05:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:05:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:05:07.178 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:05:07.179 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:05:07.180 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:05:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:05:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:05:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:05:07.190 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:05:07.191 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:05:07.192 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:05:07.193 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:05:07.194 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:05:07.195 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:05:07.196 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:05:07.197 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:05:07.198 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:05:07.199 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:05:07.200 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:05:09.884 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:05:09.884 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:05:09.884 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-09-21T03:05:09.884 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.885 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.886 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:05:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:05:09.888 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:05:09.889 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.890 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:05:09.891 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:05:09.892 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.893 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:05:09.894 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:05:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:05:09.896 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:05:09.896 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:05:09.896 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-21T03:05:10.337 DEBUG:teuthology.parallel:result is None 2024-09-21T03:05:10.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:10.483 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:05:10.505 INFO:teuthology.orchestra.run.smithi083.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.505 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.505 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:05:10.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:10.646 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:05:10.672 INFO:teuthology.orchestra.run.smithi142.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.673 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.673 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:05:10.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:10.812 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:05:10.839 INFO:teuthology.orchestra.run.smithi193.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.839 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:05:10.839 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:05:10.841 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:05:10.841 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:05:10.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:05:10.868 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:05:10.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:05:10.900 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:05:10.900 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:05:10.932 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:05:10.932 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:05:10.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:05:10.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:05:11.023 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:05:11.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:05:11.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:05:11.122 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:05:11.122 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:05:11.150 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:05:11.218 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:05:11.218 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:05:11.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:05:11.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:05:11.308 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:05:11.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:05:11.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:05:11.400 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:05:11.400 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:05:11.425 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:05:11.490 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:05:11.490 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:05:11.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:05:11.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:05:11.579 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:05:11.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:05:11.605 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:05:11.671 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:05:11.672 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:05:11.696 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:05:11.761 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:05:11.849 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2}} 2024-09-21T03:05:11.849 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:11.850 INFO:tasks.cephadm:Cluster fsid is 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:05:11.850 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:05:11.850 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.193'} 2024-09-21T03:05:11.850 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-09-21T03:05:11.850 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:05:11.850 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:05:11.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-09-21T03:05:11.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-09-21T03:05:11.901 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-21T03:05:11.927 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:05:11.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:12.068 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:05:12.202 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:05:12.203 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:05:12.203 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:05:12.204 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:12.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:05 /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:12.728 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.248 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:05 /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.248 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.768 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:05 /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.768 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.788 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.807 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:05:13.839 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:05:13.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:05:13.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:05:13.850 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:05:15.329 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:05:15.378 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:05:15.572 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:05:48.523 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-21T03:05:48.524 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T03:05:48.569 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:05:48.569 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:05:48.569 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:05:48.569 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-09-21T03:05:48.569 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:05:48.570 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:05:48.570 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:05:51.294 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-21T03:05:51.295 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-21T03:05:51.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-09-21T03:05:51.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-09-21T03:05:51.381 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-21T03:05:51.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-09-21T03:05:51.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-09-21T03:05:51.467 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-21T03:05:51.495 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:05:51.496 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T03:05:51.496 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:05:51.496 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:05:51.496 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:05:51.496 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:05:51.497 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:05:51.498 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:05:51.498 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:05:51.498 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:05:51.498 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:05:51.498 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T03:05:51.498 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:05:51.499 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:05:51.516 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5108497e-77c6-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T03:05:51.517 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:05:51.559 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:05:51.602 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:05:51.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:05:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:05:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-09-21T03:05:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:05:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-09-21T03:05:51.829 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:05:51.829 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-09-21T03:05:51.830 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-09-21T03:05:51.837 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:05:51.837 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:05:51.844 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:05:51.844 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:05:51.851 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-09-21T03:05:51.857 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-09-21T03:05:51.857 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:05:51.857 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-09-21T03:05:51.879 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:05:51.879 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:05:51.879 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-09-21T03:05:51.879 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-09-21T03:05:51.886 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:05:51.886 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:05:51.892 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:05:51.892 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:05:51.898 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 139816470051184 on /run/cephadm/5108497e-77c6-11ef-baf5-efdc52797490.lock 2024-09-21T03:05:51.905 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 139816470051184 acquired on /run/cephadm/5108497e-77c6-11ef-baf5-efdc52797490.lock 2024-09-21T03:05:51.906 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-09-21T03:05:51.906 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-09-21T03:05:51.906 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-21T03:05:51.909 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-09-21T03:05:51.909 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-09-21T03:05:51.912 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:05:51.912 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T03:05:51.912 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link noprefixroute 2024-09-21T03:05:51.914 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:05:51.915 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-09-21T03:05:51.915 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-09-21T03:05:51.915 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:05:51.915 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:05:51.916 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:05:52.348 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:05:52.348 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:05:52.349 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:05:52.349 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:05:52.349 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:05:52.349 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:05:52.349 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:05:53.261 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:05:53.262 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:05:53.262 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:05:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-09-21T03:05:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-09-21T03:06:08.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQAZOO5mtcjlKxAAqCwi08D1fMC7NS+8DdF8qw== 2024-09-21T03:06:09.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQAgOO5m0zFsMBAAZyOx9QmKarbaSDDsyJyjVw== 2024-09-21T03:06:10.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQAhOO5maNDGKhAAMb+G5L9hxQCJtCiFCfoYkw== 2024-09-21T03:06:10.312 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:06:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-09-21T03:06:12.124 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-09-21T03:06:12.332 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:06:12.507 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:06:12.677 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target. 2024-09-21T03:06:12.677 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target. 2024-09-21T03:06:12.937 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:06:12.938 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:06:13.108 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target.wants/ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490@.service. 2024-09-21T03:06:13.607 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:06:13.607 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-21T03:06:13.614 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:06:13.614 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:06:13.614 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-21T03:06:13.614 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:06:13.615 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-09-21T03:06:13.615 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-09-21T03:06:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:14 smithi083 ceph-mon[30853]: from='client.? 172.21.15.83:0/1745381754' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:06:14.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.859212s) 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:06:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:06:14.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:06:14.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:14.972 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-09-21T03:06:14.972 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:06:15.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:15 smithi083 ceph-mon[30853]: from='client.? 172.21.15.83:0/3265248813' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:06:15.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:15 smithi083 ceph-mon[30853]: from='client.? 172.21.15.83:0/3265248813' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:06:16.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:06:16.053 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-09-21T03:06:16.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:16 smithi083 ceph-mon[30853]: from='client.? 172.21.15.83:0/2807684453' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-09-21T03:06:17.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 systemd[1]: Stopping Ceph mon.a for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:06:17.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a[30849]: 2024-09-21T03:06:17.397+0000 7f16e4d5f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:17.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a[30849]: 2024-09-21T03:06:17.397+0000 7f16e4d5f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:06:17.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 podman[31094]: 2024-09-21 03:06:17.450484552 +0000 UTC m=+0.173924575 container died db2ee07aef96802a2e2adf8a1f274ee86f1400b8473d26678d7ebf147ddfc97f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:06:18.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 podman[31094]: 2024-09-21 03:06:17.831940887 +0000 UTC m=+0.555380897 container cleanup db2ee07aef96802a2e2adf8a1f274ee86f1400b8473d26678d7ebf147ddfc97f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:06:18.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:17 smithi083 bash[31094]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:06:18.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 podman[31106]: 2024-09-21 03:06:18.066239699 +0000 UTC m=+0.612160695 container remove db2ee07aef96802a2e2adf8a1f274ee86f1400b8473d26678d7ebf147ddfc97f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:06:18.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 systemd[1]: ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:06:18.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 systemd[1]: Stopped Ceph mon.a for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:06:18.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 systemd[1]: Starting Ceph mon.a for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 podman[31200]: 2024-09-21 03:06:18.304042548 +0000 UTC m=+0.015759305 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 podman[31200]: 2024-09-21 03:06:18.404898924 +0000 UTC m=+0.116615706 container create 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 podman[31200]: 2024-09-21 03:06:18.602779437 +0000 UTC m=+0.314496186 container init 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 podman[31200]: 2024-09-21 03:06:18.605786533 +0000 UTC m=+0.317503275 container start 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:06:18.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: pidfile_write: ignore empty --pid-file 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: load: jerasure load: lrc 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Git sha 0 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: DB SUMMARY 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: DB Session ID: 4BXG42WXY99T00QYPP5Z 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:06:18.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94252 ; 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.env: 0x55e2439b8d60 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.info_log: 0x55e249ab5360 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.statistics: (nil) 2024-09-21T03:06:18.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.use_fsync: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.db_log_dir: 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.wal_dir: 2024-09-21T03:06:18.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.write_buffer_manager: 0x55e249ab9d60 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:06:18.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.unordered_write: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.row_cache: None 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.wal_filter: None 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:06:18.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.wal_compression: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:06:18.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_open_files: -1 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:06:18.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Compression algorithms supported: 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kZSTD supported: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:06:18.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.merge_operator: 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_filter: None 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e249ab4fa0) 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-09-21T03:06:18.697 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x55e245a79610 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-09-21T03:06:18.698 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-09-21T03:06:18.699 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression: NoCompression 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.num_levels: 7 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:06:18.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:06:18.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:06:18.702 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:06:18.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:06:18.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.ttl: 2592000 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:06:18.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 06ac8d59-0d4a-41ac-a37b-507d75ddb8ae 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887978642306, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887978643472, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 281, "table_properties": {"data_size": 88518, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11464, "raw_average_key_size": 47, "raw_value_size": 82075, "raw_average_value_size": 341, "num_data_blocks": 11, "num_entries": 240, "num_filter_entries": 240, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726887978, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "06ac8d59-0d4a-41ac-a37b-507d75ddb8ae", "db_session_id": "4BXG42WXY99T00QYPP5Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887978643566, "job": 1, "event": "recovery_finished"} 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:06:18.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e249ad8a80 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: DB pointer 0x55e24a87e000 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 90.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 90.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 115.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 115.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.708 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 25.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 25.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x55e245a79610#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:18.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???) e1 preinit fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).mds e1 new map 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: btime 2024-09-21T03:06:13:570144+0000 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:06:18.710 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:06:18.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 bash[31200]: 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 2024-09-21T03:06:18.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 systemd[1]: Started Ceph mon.a for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:06:18.711 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:06:18.939 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: monmap epoch 1 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: last_changed 2024-09-21T03:06:10.663660+0000 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: created 2024-09-21T03:06:10.663660+0000 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: min_mon_release 19 (squid) 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: election_strategy: 1 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: fsmap 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:06:18.940 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:18 smithi083 ceph-mon[31219]: mgrmap e1: no daemons active 2024-09-21T03:06:20.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3324236774' entity='client.admin' 2024-09-21T03:06:20.506 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:06:20.519 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:06:20.519 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-09-21T03:06:20.519 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:06:20.520 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:06:20.786 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:06:20.786 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:06:20.937 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490.target.wants/ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-5108497e-77c6-11ef-baf5-efdc52797490@.service. 2024-09-21T03:06:21.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:20 smithi083 systemd[1]: Starting Ceph mgr.x for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:06:21.449 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 podman[31431]: 2024-09-21 03:06:21.154638299 +0000 UTC m=+0.103142139 container create f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:06:21.449 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 podman[31431]: 2024-09-21 03:06:21.06703175 +0000 UTC m=+0.015535588 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:06:21.449 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 podman[31431]: 2024-09-21 03:06:21.340400244 +0000 UTC m=+0.288904081 container init f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:06:21.450 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 podman[31431]: 2024-09-21 03:06:21.343118282 +0000 UTC m=+0.291622113 container start f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:06:21.450 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 bash[31431]: f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f 2024-09-21T03:06:21.463 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:06:21.463 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-21T03:06:21.470 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:06:21.470 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:06:21.470 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-21T03:06:21.470 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:06:21.478 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:06:21.479 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-09-21T03:06:21.486 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-09-21T03:06:21.727 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:21 smithi083 systemd[1]: Started Ceph mgr.x for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:06:22.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:22 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1716054770' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:06:22.578 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:06:22.579 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:06:22.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:06:22.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:06:13:570144+0000", 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:22.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:06:13.570954+0000", 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:22.585 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:06:25.354 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2707929551' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:06:25.609 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:06:25.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:06:25.611 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:06:25.612 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:06:25.614 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:25.615 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:06:13:570144+0000", 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.616 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:06:13.570954+0000", 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:25.617 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: Activating manager daemon x 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: mgrmap e2: x(active, starting, since 0.00346005s) 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: Manager daemon x is now available 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:06:26.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' 2024-09-21T03:06:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:06:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' 2024-09-21T03:06:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:26 smithi083 ceph-mon[31219]: from='mgr.14100 172.21.15.83:0/100740762' entity='mgr.x' 2024-09-21T03:06:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:28 smithi083 ceph-mon[31219]: mgrmap e3: x(active, since 1.00813s) 2024-09-21T03:06:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:28 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3196542935' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:06:28.982 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:28.982 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:28.982 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:06:28.982 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:28.983 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:06:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:06:28.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:06:28.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:06:28.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:06:13:570144+0000", 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:06:28.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:06:13.570954+0000", 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:06:28.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:28.990 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-09-21T03:06:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:29 smithi083 ceph-mon[31219]: mgrmap e4: x(active, since 2s) 2024-09-21T03:06:30.138 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-21T03:06:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:06:30.140 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-09-21T03:06:30.516 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:30 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1164399633' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:06:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4087180465' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:06:31.622 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setuser ceph since I am not root 2024-09-21T03:06:31.622 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setgroup ceph since I am not root 2024-09-21T03:06:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:32 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4087180465' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:06:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:32 smithi083 ceph-mon[31219]: mgrmap e5: x(active, since 5s) 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:06:33.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:33.219 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-09-21T03:06:33.219 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-09-21T03:06:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:33 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2993443264' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:06:36.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: Active manager daemon x restarted 2024-09-21T03:06:36.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: Activating manager daemon x 2024-09-21T03:06:36.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: mgrmap e6: x(active, starting, since 0.00393086s) 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:06:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:36 smithi083 ceph-mon[31219]: Manager daemon x is now available 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-09-21T03:06:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: mgrmap e7: x(active, since 1.00648s) 2024-09-21T03:06:38.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-mon[31219]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:06:38.140+0000 7f3f792ff640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:06:38] ENGINE Error in HTTPServer.serve 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Traceback (most recent call last): 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._connections.run(self.expiration_interval) 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._run(expiration_interval) 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:06:38.278 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s = self.context.wrap_socket( 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: return self.sslsocket_class._create( 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self.do_handshake() 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._sslobj.do_handshake() 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:06:38.279 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:06:39.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: [21/Sep/2024:03:06:37] ENGINE Bus STARTING 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: [21/Sep/2024:03:06:38] ENGINE Serving on http://172.21.15.83:8765 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: [21/Sep/2024:03:06:38] ENGINE Serving on https://172.21.15.83:7150 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: [21/Sep/2024:03:06:38] ENGINE Bus STARTED 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: [21/Sep/2024:03:06:38] ENGINE Error in HTTPServer.serve 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:06:39.363 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:06:39.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:39 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/236038865' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:06:39.633 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:06:39.633 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:06:40.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:40 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/236038865' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:06:40.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:40 smithi083 ceph-mon[31219]: mgrmap e8: x(active, since 2s) 2024-09-21T03:06:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:41 smithi083 ceph-mon[31219]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:41 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:41 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:42.004 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:06:42.004 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-09-21T03:06:42.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-mon[31219]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:43.049 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Generating public/private rsa key pair. 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Your identification has been saved in /tmp/tmp3x5h97vx/key 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Your public key has been saved in /tmp/tmp3x5h97vx/key.pub 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: The key fingerprint is: 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: SHA256:X0tLIEHzVt6x6PSRzurGbGOWza1awwzwqe8Crwh4ieE ceph-5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: The key's randomart image is: 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: +---[RSA 3072]----+ 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | .+ . . | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | + o o + | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | . = + = | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | o * = . | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | . S O + | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | . + . .. = B | 2024-09-21T03:06:43.050 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | E + ooo++=. | 2024-09-21T03:06:43.051 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | . . . ooX.o..| 2024-09-21T03:06:43.051 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: | . .. B=o.. | 2024-09-21T03:06:43.051 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: +----[SHA256]-----+ 2024-09-21T03:06:43.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:43 smithi083 ceph-mon[31219]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:43.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:43 smithi083 ceph-mon[31219]: Generating ssh key... 2024-09-21T03:06:43.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:43 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:43.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:43 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:44.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:06:44.527 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:06:44.527 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:06:44.536 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-09-21T03:06:44.835 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:44 smithi083 ceph-mon[31219]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:45.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:45 smithi083 ceph-mon[31219]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:46.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:46 smithi083 ceph-mon[31219]: Deploying cephadm binary to smithi083 2024-09-21T03:06:48.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-09-21T03:06:48.911 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-09-21T03:06:49.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:49 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:49.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:49 smithi083 ceph-mon[31219]: Added host smithi083 2024-09-21T03:06:49.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:49 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:50.965 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:50 smithi083 ceph-mon[31219]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:50.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:50 smithi083 ceph-mon[31219]: Saving service mon spec with placement count:5 2024-09-21T03:06:50.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:50 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:50.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:50 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:50.968 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:06:50.968 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-09-21T03:06:52.124 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:06:52.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:52 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:52.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:52 smithi083 ceph-mon[31219]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:06:52.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:52 smithi083 ceph-mon[31219]: Saving service mgr spec with placement count:2 2024-09-21T03:06:52.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:52 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:54.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:53 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2350066697' entity='client.admin' 2024-09-21T03:06:54.926 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-09-21T03:06:55.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:55 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1159392477' entity='client.admin' 2024-09-21T03:06:55.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:55 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:56.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:56 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3307626206' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:06:56.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:56 smithi083 ceph-mon[31219]: from='mgr.14118 172.21.15.83:0/3361519783' entity='mgr.x' 2024-09-21T03:06:56.330 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setuser ceph since I am not root 2024-09-21T03:06:56.330 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:06:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setgroup ceph since I am not root 2024-09-21T03:06:57.210 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:57 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3307626206' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:06:57.210 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:57 smithi083 ceph-mon[31219]: mgrmap e9: x(active, since 19s) 2024-09-21T03:06:58.561 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:06:58 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1764600388' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:06:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-09-21T03:06:58.764 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-09-21T03:07:01.512 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: Active manager daemon x restarted 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: Activating manager daemon x 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: mgrmap e10: x(active, starting, since 0.00599745s) 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:07:01.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:01 smithi083 ceph-mon[31219]: Manager daemon x is now available 2024-09-21T03:07:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:07:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:07:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:07:02] ENGINE Bus STARTING 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-mon[31219]: mgrmap e11: x(active, since 1.00669s) 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:07:02.177+0000 7fbb2bbd6640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:07:02] ENGINE Error in HTTPServer.serve 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Traceback (most recent call last): 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._connections.run(self.expiration_interval) 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._run(expiration_interval) 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:07:02.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s = self.context.wrap_socket( 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: return self.sslsocket_class._create( 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self.do_handshake() 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._sslobj.do_handshake() 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:07:02.624 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-09-21T03:07:03.090 INFO:teuthology.orchestra.run.smithi083.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: [21/Sep/2024:03:07:02] ENGINE Serving on https://172.21.15.83:7150 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: [21/Sep/2024:03:07:02] ENGINE Error in HTTPServer.serve 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-21T03:07:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-21T03:07:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:07:04.124 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-21T03:07:04.124 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:07:04.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: [21/Sep/2024:03:07:02] ENGINE Serving on http://172.21.15.83:8765 2024-09-21T03:07:04.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: [21/Sep/2024:03:07:02] ENGINE Bus STARTED 2024-09-21T03:07:04.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:03 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:04 smithi083 ceph-mon[31219]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:04 smithi083 ceph-mon[31219]: mgrmap e12: x(active, since 2s) 2024-09-21T03:07:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:04 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:05.721 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUIexL04ITy76EawkJcsz4ivVLXYYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMDcwNFoXDTM0\nMDkyMjAzMDcwNFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjgzMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAoixGjPL3H+/7uwow6steyck5Ps/GfIU/uEki\nsJxHq4x9YPwHsXe6emVxNv1GupAbTChLDwb3xr8AgqzFQAiEUyfKOjAz6GwY+gsx\nS3N2vh9JUCeeu6WKiEws1wS/F1eFnN1jrEZT3yYNWHlwkie+7F4PeeXkbSHp4G5Y\nb1Jgm5yqPgnud9a4vexSoOvl01SpjlQwu7HefH0IAGBKlgQE0SdHz8EjyM8Ox7Hp\no7V0BasvCocbJFhFPrTTT4eeECq0B2P+W+jv9IyuMnEjCdFcpQxAKEUoe8MQoNjR\nC3+2KSFZrwQMHWZj9UNdiZMqK5Gu9C9ZtdcT/WchKUaRAhHTI/EUtgMNAduApvNJ\ndPYnTeGuFpAJWU6ngEwEZ3XApFZiFKAbNI4QSKwP88CRjKXYVX9MB1br/AmPQggM\ntmPll0sUUcmqBliYilrJNGhkaBjaTyPZYKlc2FaRBKNX5AU+xw9WiRSVQ9yX6Qe3\nZYzUIKHeNsgPwg24CZfIQtLCIhlWQcVxu3fnnECbuSyG61D1XdcO5dllmCrg/wNd\n0nrGgxw+BVLg8ffb4DK71m0E2z7SQ7asLroYuj55UHXyON3j3b1gvpvOFxWvWodd\nB8oBjwez7jFyr1lAM/8tlYmtQTf9K+YoCJxxORjwGYAnkK1aBvp6sJu9Tji2mHNM\nrSjWCtUCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDgzLmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPUzAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQBbOcLqeLllmV0/i0ZZ1PRJPJ/ZRH4rGqdom7Wz\n8/AI1fKokxQ8asZS9qXoZZ0HKvK4WwzB84FcSKKSC6NCbJXcaV6dduDv0768jff+\nwwNH4dupy3F8EUQxV2IS8q+hZdRJPUtrQBgu4yPHDvQaZHWyfY9xw7bIFFV+SohW\nf4TkA39K6zC/DKshJrOj+e0z0L+Dtcv3mESMzhCBhOvqgSoigx+aNKi+ISxR/HbC\nrMAz5QmoVZcqKJnJmb1eOo0Nfuxx3mDZ39U9a5hMUaj7ok73SvhOTQTusIk0FSEx\nefo8C48GE9xL0JDDJD+RguhyY9qA5ZyZprAF+HpZ0obGAfdHQBoBC1wXFQvh7P5a\niBo645A49/0hAkq9qAKrBLW3myeOJj7K8oUKw4sVhsIIM3FnqA+e1akj4/OTUATE\nqnH8D9yxcdfdYEUT8rIYbDd5TLV1O5By+ctPMvntV7q+BX8rlQ6A6v4h+NTkekWY\n23/HjWMnqcQEvPMt9s78QBuyV2dAHvaiY2feaFhlmDFaLe+yE6HUbMfXpwarCNr7\nBNiOoVWBCekrvrK4SOaCWFbBGRpCXEV6jzuztcLgoH5Xv2ei1NHl0RCINCbUgYNQ\nC78CS4dhzfwOoxG9QTDKZiInjJf5gP4ekcIdM8wNEGdILJRV1eOOmEB1bdzp6WOd\nflfa1w==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAoixGjPL3H+/7uwow6steyck5Ps/GfIU/uEkisJxHq4x9YPwH\nsXe6emVxNv1GupAbTChLDwb3xr8AgqzFQAiEUyfKOjAz6GwY+gsxS3N2vh9JUCee\nu6WKiEws1wS/F1eFnN1jrEZT3yYNWHlwkie+7F4PeeXkbSHp4G5Yb1Jgm5yqPgnu\nd9a4vexSoOvl01SpjlQwu7HefH0IAGBKlgQE0SdHz8EjyM8Ox7Hpo7V0BasvCocb\nJFhFPrTTT4eeECq0B2P+W+jv9IyuMnEjCdFcpQxAKEUoe8MQoNjRC3+2KSFZrwQM\nHWZj9UNdiZMqK5Gu9C9ZtdcT/WchKUaRAhHTI/EUtgMNAduApvNJdPYnTeGuFpAJ\nWU6ngEwEZ3XApFZiFKAbNI4QSKwP88CRjKXYVX9MB1br/AmPQggMtmPll0sUUcmq\nBliYilrJNGhkaBjaTyPZYKlc2FaRBKNX5AU+xw9WiRSVQ9yX6Qe3ZYzUIKHeNsgP\nwg24CZfIQtLCIhlWQcVxu3fnnECbuSyG61D1XdcO5dllmCrg/wNd0nrGgxw+BVLg\n8ffb4DK71m0E2z7SQ7asLroYuj55UHXyON3j3b1gvpvOFxWvWoddB8oBjwez7jFy\nr1lAM/8tlYmtQTf9K+YoCJxxORjwGYAnkK1aBvp6sJu9Tji2mHNMrSjWCtUCAwEA\nAQKCAgArF2SvsyejHtteubiBywQmzgcd/4uYLrfdueEz2HLazHvMN2u75jpwp1/L\n4sTWcdAvDk3TcmCIONxj3JCVES5IAjyj9Yb1bTO9zReOGuIkZp/OYpi096m1ybfo\ncmRsSV9BbTo83nsOwketiKc2zsTWfa/TDhkCPo6sieWJvvPYpWo/DdiFk1KHdLIH\n/GaXHtnm7cv9M6gQoBBuOSfYcLzu7mcQdxuREiO1QDr6uw5XK0NWosu9fo73uIil\niz2ewhoQFKvTtDN+YZsv4AwVYcIiQP0TNj0plY1nj/UL/yViDHCOJp8peLl6xCmo\nkqgT56P7UlGeFcZEARE4rH1/dUkKyX8zYezES0Frys6aZMtLGGGljhVSmPHwRowg\n/GLd/Ds4khMIxIYBL9Xl4maI5Uj2xUXg9ej7wKSg6XN1+PmYxSqbsaJPwQZkpqfI\nLcJ6DwE7XvOqcqxixAIIg5X0fHTBwKtFKsO26+NLExCh/TZJRAfbU8FhJ8g1Ev3I\nHmY68N7qW4NSUfRU+t9Rlk8JxeMGyBkQt1ZdFEu8QdynjwsZmZjyv37h7HaDJwPM\n6LlmOTxIt2EWdaCW8wZDtLIuMk4J2ufBULM/Jbxn2FZio1CZroGzBAR1lpiujCp/\n7MGv+gJG4X/hFzGb8l5Ht4vzWzvdVOm5RQSdgdo6v0bumx7xiQKCAQEA0opgGUkW\nhQYBO2226ov3C2MZCvNWsfK2mU4JmkQl8nFju5bo0I66vfz1tccOQxjwIJD5sZPH\nu+cStuih/3pYqOceA+LfzV3cBUs2WHmiD8nN+WZYfSBiBEFKO9oB1EaToKA8xQKA\n+E6satpLWHd47Jse3/n1CgrCNs/YLcdThSdhu30wqZ6CXCS/0QFXkFECO5j2CQaa\nqz277DU6UYq3Dg5xlJ2Bqs7KRqHydd4r3xz3izG68NLay24QvM3paS/5nOkRMfve\nNbQoi+ZXUaCM0blv8qNByCJjXnnawVbMbREmv2Zgs73TAmlC0N/+TJ3QEr7HM0aO\n2v2Z0yJLxGZLzwKCAQEAxTBiEaG401KWP+TW1vuDeTnSGP4gRkHtWunEfar45A4F\no3TvkpkFa6l5WSPij0sNizw2E4Nmd9A0/YRoamxWFbRoXlfJQyJe4BXH4betHEYv\nuHkviV5IRdVZ01R8KV2apACRqzSIn6FPVhknhpggqWWPwNpd43ZJAfYMUkSOUfF+\n8sqeMS7cdGkmBijfY6mNVpq0zIhwvkfYIhyqdA0RhW5JJQ6VRLla/l/8L0Y4FQVF\nS3hd/TXPoxGAL9aAI0k/wgF6xu2J3KpYz5fPhdOhhaL4tRJDgctTmXu32nRvYyFm\ngOQEiaoXUkR6VjKQ1OITUVvLjfPfhkz+Xkf9Vno0GwKCAQBR9Il57kh02aSLJKcq\nX/d3mEi8SYL62/cWcBku0s5OtLLU5+aSoKyynhBrhSPZHJEErEp2RIdgbo+LYb0F\nvPNcNDMr4hmqUJn/+YobIKXkLXMfXSae9ocRDvU9nRxPLHX09csSCWp4ve9B3N2t\nfzw0JiH7n8ox8YQuX/PPxiKN14mkuRPDQy+NfMSqMDKMu+v5lQ5Y7xgwBmTrisC7\n0xp0v0yZREKgYcS9f/boyOYdAKKfrWkOq5V90Md1J1Zvmrw4x0TpRxj1oL6TzsM4\nxB1s2hlowwItTK2PrjFoR0hmQGHr6YdFEGReMOlQc5EC8/91/IuAOzwKpONHG7eQ\nqR1jAoIBAGUpm0ZcZ4vStM52wllsk1if909nIHuq2vBnOa0XB7VIFFB1b2sQIh3H\nxj+bD2JgKMSj0UJViT4yjNz7Du56QffEpD6yXsiYW4Au7Gq6Zo842saTNXYN7poV\nCI5Iq6I1KV62BAWHYAg1I5SSNxrv9OGy/vr7jGtjhbUB2/Rgw3r6oBxDIsEsv2Dh\niOZkFab1l1M070h3Pw//n7xwZGpBQhnoKJJgVuf5B9u0tpYyOhdeqtw7Fk0PTETm\nd5MTQo3XyGPpiml/99hDdGicA5Uuq5D5uQp8IjH2dB8rv3jNf0+mFQZO9J0DrA9C\nnliWfIMasLpuk5Fx6dzmpyELYcb5zr0CggEBALRVAJJHafEX4DFzOOe++CLwT5cu\neLnsleKH2NUCXhldTN8ZmpbNQgjuhJwea9xYnchIUVY2a989N10jYxEmG43lmXX1\n0pZXQZMZQZ7Z2yNvzA/SJ2Q6OklYdXiHcb1vQE0pw6Ussqsw/RUa/sGGA7Fw8zOH\nIGythZH/P3gmjEtwzCJAGKwzFvEBRjV3AAPlZheF4ja4t1QKjBOMLqw5R2xi86dh\nF2D6F3+hRdosNWjoKULnkoLlG3wKVeED+DKnLrAIh55Ybj9Mp70tzigOHEop+mpS\n+UECIHxHVXwEXq5d+rkR+5whBIsrzH1CDB3kgm0WNKKAc53JxEpbZTrI+fo=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:07:06.885 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:07:07.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:07 smithi083 ceph-mon[31219]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:07.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:07 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:08.059 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:07:08.060 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-09-21T03:07:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:08 smithi083 ceph-mon[31219]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:08 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:09.441 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$glmdqsjzMqn9DYlT/qVWuuni/78yzsF6DHTMl1.Xaae8uzbs5GP0e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726888028, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:07:09.442 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-09-21T03:07:10.090 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:09 smithi083 ceph-mon[31219]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:10.090 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:09 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:10.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:07:10.666 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:07:10.666 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-21T03:07:10.673 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:07:10.674 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-21T03:07:10.674 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-21T03:07:10.674 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:07:10.675 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: Password: 5sgs8oqnhi 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:10.676 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config directory 2024-09-21T03:07:11.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:10 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/719483719' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:07:12.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:12.011 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-09-21T03:07:12.048 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:07:12.049 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:12.049 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:07:12.064 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:07:12.064 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:12.064 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:07:12.119 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:07:12.120 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:12.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:07:12.185 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:07:12.185 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:12.185 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:07:12.240 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:07:12.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:07:12.328 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:12.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:07:12.382 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:12.394 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:07:12.435 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzg7ltdDKgC9XT6A+EK51s59olsSfX1viQS6UBomuj7IiZsJcBrdeOMmxLClrMnWV2p3akPovKZ/WzDvHIsjv24ceIvsElxZIpJQ36Hkjs2l8WiM7QWt5VBvuK9n6uEw0N8KJ5g6rTMBEVK/NTbzLIBvUvrHi/CqsJ8ajefKg5XIf38JGERBWKFup/E0wd+3QWWm0t0Gh3RsaQvUPjd7lc2FAGbyOdL2ypvr8/xxXCNjguxTTZXRhDduAbdu8yXJKtXpjwZAcO3fXKuzNA2bx1cp94ZFuSEQNEeQX40TYmNBoOlyJTt41Inw6bYfFJad5iRLDXF66LDPsJXvI7oYMLwpChtWsAPW9I825jBX1auQb9o8TzWFoLNvlHxh54cu3UoIIuAUL9cuoXyfSSpRqL1kAzxxt30cgnIS667NPW3UDCgC07NSe2Qx+qZwcicWK6wZW/pK5VpiXSwo+fR2+OU2sc1gGhb9Q1GNVFI/TNNEUjCYxqpff9uHrsM2EamK0= ceph-5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:12.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:07:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:12 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3686263895' entity='client.admin' 2024-09-21T03:07:12.697 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:14.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1587159265' entity='client.admin' 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: mgrmap e13: x(active, since 12s) 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:14.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:14 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:14.386 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:07:14.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:07:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:16 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:16.764 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-09-21T03:07:16.764 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:07:16.764 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:07:16.783 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:07:16.783 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:16.841 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-09-21T03:07:16.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch host add smithi142 2024-09-21T03:07:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:18 smithi083 ceph-mon[31219]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:19 smithi083 ceph-mon[31219]: Deploying cephadm binary to smithi142 2024-09-21T03:07:20.688 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-09-21T03:07:21.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:07:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:21.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:21 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:21.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:21 smithi083 ceph-mon[31219]: Added host smithi142 2024-09-21T03:07:21.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:21 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:21.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:21 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:21.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:21 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:22.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:22.099 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-09-21T03:07:22.722 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-21T03:07:22.723 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:07:22.723 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:07:22.743 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:07:22.743 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:22.801 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-21T03:07:22.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch host add smithi193 2024-09-21T03:07:23.044 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:23.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:22 smithi083 ceph-mon[31219]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:23.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:22 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:23.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:22 smithi083 ceph-mon[31219]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:25 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: Deploying cephadm binary to smithi193 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:26 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:26.671 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:27 smithi083 ceph-mon[31219]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:27 smithi083 ceph-mon[31219]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:27 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:27 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:27 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:27.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:07:27.529 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:28.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:07:28.161 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-21T03:07:28.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:28 smithi083 ceph-mon[31219]: Added host smithi193 2024-09-21T03:07:28.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:28 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:28.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:28 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:28.779 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:07:28.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:07:29.024 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:29.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:29 smithi083 ceph-mon[31219]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:30.016 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-09-21T03:07:30.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:30 smithi083 ceph-mon[31219]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:07:30.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:30 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/746752288' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:07:30.618 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-09-21T03:07:30.619 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-09-21T03:07:30.619 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-09-21T03:07:30.619 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi142:172.21.15.142=b;smithi193:172.21.15.193=c' 2024-09-21T03:07:30.890 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:07:30.935 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/746752288' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:31.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:31.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:31 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:31.576 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-21T03:07:32.148 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:07:32.150 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:07:32.152 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:07:32.153 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:07:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:07:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:32.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:32 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:33.124 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:33 smithi083 ceph-mon[31219]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:33 smithi083 ceph-mon[31219]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi142:172.21.15.142=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:07:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:33 smithi083 ceph-mon[31219]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi142:172.21.15.142=b;smithi193:172.21.15.193=c;count:3 2024-09-21T03:07:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:33 smithi083 ceph-mon[31219]: Deploying daemon mon.c on smithi193 2024-09-21T03:07:35.020 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:07:35.020 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","modified":"2024-09-21T03:06:10.663660Z","created":"2024-09-21T03:06:10.663660Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:07:35.020 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-21T03:07:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:35 smithi083 ceph-mon[31219]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:35 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/1377158266' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:07:36.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:36 smithi193 systemd[1]: Started Ceph mon.c for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:07:36.730 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:07:36.730 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:07:36.975 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: mon.a calling monitor election 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: mon.c calling monitor election 2024-09-21T03:07:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: monmap epoch 2 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: last_changed 2024-09-21T03:07:36.127706+0000 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: created 2024-09-21T03:06:10.663660+0000 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: min_mon_release 19 (squid) 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: election_strategy: 1 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T03:07:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: fsmap 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: mgrmap e13: x(active, since 40s) 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: overall HEALTH_OK 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:41.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:41 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: mon.a calling monitor election 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: mon.c calling monitor election 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:07:41.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: monmap epoch 2 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: last_changed 2024-09-21T03:07:36.127706+0000 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: created 2024-09-21T03:06:10.663660+0000 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: min_mon_release 19 (squid) 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: election_strategy: 1 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: fsmap 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: mgrmap e13: x(active, since 40s) 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: overall HEALTH_OK 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:41.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:41.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:41 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:42.125 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:07:42.125 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","modified":"2024-09-21T03:07:36.127706Z","created":"2024-09-21T03:06:10.663660Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:07:42.125 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-21T03:07:42.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:42 smithi193 ceph-mon[35224]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:42.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:42 smithi193 ceph-mon[35224]: Deploying daemon mon.b on smithi142 2024-09-21T03:07:42.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:42 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/558496946' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:07:42.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:42 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:42 smithi083 ceph-mon[31219]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:42 smithi083 ceph-mon[31219]: Deploying daemon mon.b on smithi142 2024-09-21T03:07:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:42 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/558496946' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:07:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:42 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:43.678 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:07:43.679 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:07:43.933 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:07:44.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:44 smithi193 ceph-mon[35224]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:44 smithi083 ceph-mon[31219]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:49.708 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:07:49.709 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","modified":"2024-09-21T03:07:44.693343Z","created":"2024-09-21T03:06:10.663660Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T03:07:49.709 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-21T03:07:50.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:50.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: mon.a calling monitor election 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: mon.c calling monitor election 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/3933407210' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: mon.b calling monitor election 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:07:50.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: monmap epoch 3 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: last_changed 2024-09-21T03:07:44.693343+0000 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: created 2024-09-21T03:06:10.663660+0000 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: min_mon_release 19 (squid) 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: election_strategy: 1 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: fsmap 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: mgrmap e13: x(active, since 48s) 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: overall HEALTH_OK 2024-09-21T03:07:50.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:49 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:50.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: mon.a calling monitor election 2024-09-21T03:07:50.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: mon.c calling monitor election 2024-09-21T03:07:50.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/3933407210' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: mon.b calling monitor election 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: monmap epoch 3 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: last_changed 2024-09-21T03:07:44.693343+0000 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: created 2024-09-21T03:06:10.663660+0000 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: min_mon_release 19 (squid) 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: election_strategy: 1 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: fsmap 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: mgrmap e13: x(active, since 48s) 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: overall HEALTH_OK 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:49 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:49 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:50.324 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:07:50.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:07:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:51.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:50 smithi193 ceph-mon[35224]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:51.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:50 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:51.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:50 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:51.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:50 smithi083 ceph-mon[31219]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:51.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:50 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:51.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:50 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:51.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:50 smithi142 ceph-mon[34512]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:51.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:50 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:51.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:50 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:51.328 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:51.328 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-09-21T03:07:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:07:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-21T03:07:51.946 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:07:51.946 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:51.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:07:51.973 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:07:51.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:07:52.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3349606986' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:52.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:51 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.039 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:07:52.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:07:52.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3349606986' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:52.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:51 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.070 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:07:52.070 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:52.135 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:07:52.135 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3349606986' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:51 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:52.165 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:07:52.165 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:07:52.232 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T03:07:52.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T03:07:52.706 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:53.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:53.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: Reconfiguring daemon mon.a on smithi083 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:53.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:53 smithi083 ceph-mon[31219]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: Reconfiguring daemon mon.a on smithi083 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:53.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:53.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:53 smithi142 ceph-mon[34512]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:53.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: Reconfiguring daemon mon.a on smithi083 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:07:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:53 smithi193 ceph-mon[35224]: from='mgr.14152 172.21.15.83:0/3007113217' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:54 smithi083 ceph-mon[31219]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:07:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:54 smithi083 ceph-mon[31219]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:07:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:54 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:07:54.372 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setuser ceph since I am not root 2024-09-21T03:07:54.372 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:07:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ignoring --setgroup ceph since I am not root 2024-09-21T03:07:54.375 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:54 smithi142 ceph-mon[34512]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:07:54.375 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:54 smithi142 ceph-mon[34512]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:07:54.376 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:54 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:07:54.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:54 smithi193 ceph-mon[35224]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:07:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:54 smithi193 ceph-mon[35224]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:07:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:54 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:07:54.738 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-09-21T03:07:54.738 INFO:tasks.cephadm:Adding mgr.y on smithi142 2024-09-21T03:07:54.738 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-09-21T03:07:54.738 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi083=x;smithi142=y;smithi193=z' 2024-09-21T03:07:54.989 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:07:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:55 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:07:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:55 smithi083 ceph-mon[31219]: mgrmap e14: x(active, since 52s) 2024-09-21T03:07:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:55 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:07:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:55 smithi142 ceph-mon[34512]: mgrmap e14: x(active, since 52s) 2024-09-21T03:07:55.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:55 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4280152135' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:07:55.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:55 smithi193 ceph-mon[35224]: mgrmap e14: x(active, since 52s) 2024-09-21T03:07:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: Active manager daemon x restarted 2024-09-21T03:07:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: Activating manager daemon x 2024-09-21T03:07:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:07:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: mgrmap e15: x(active, starting, since 0.00753816s) 2024-09-21T03:07:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:07:59.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:07:59 smithi142 ceph-mon[34512]: Manager daemon x is now available 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: Active manager daemon x restarted 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: Activating manager daemon x 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: mgrmap e15: x(active, starting, since 0.00753816s) 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:07:59.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:07:59 smithi083 ceph-mon[31219]: Manager daemon x is now available 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: Active manager daemon x restarted 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: Activating manager daemon x 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: mgrmap e15: x(active, starting, since 0.00753816s) 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:07:59.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:07:59.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:07:59.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:07:59 smithi193 ceph-mon[35224]: Manager daemon x is now available 2024-09-21T03:08:00.097 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-21T03:08:00.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:00.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:08:00.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:08:00.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-mon[31219]: mgrmap e16: x(active, since 1.01008s) 2024-09-21T03:08:00.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:08:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:08:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:00 smithi142 ceph-mon[34512]: mgrmap e16: x(active, since 1.01008s) 2024-09-21T03:08:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:00.537 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:00.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:08:00.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:08:00.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:00 smithi193 ceph-mon[35224]: mgrmap e16: x(active, since 1.01008s) 2024-09-21T03:08:00.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:00.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:08:00.474+0000 7f46a0568640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:08:00] ENGINE Error in HTTPServer.serve 2024-09-21T03:08:00.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: Traceback (most recent call last): 2024-09-21T03:08:00.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._connections.run(self.expiration_interval) 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._run(expiration_interval) 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: s = self.context.wrap_socket( 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:08:00.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: return self.sslsocket_class._create( 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self.do_handshake() 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: self._sslobj.do_handshake() 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:08:00.874 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:08:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:08:01.138 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:08:01.141 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:08:01.143 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:08:01.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:08:01.145 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:08:01.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:08:01.189 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:08:01.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:08:01.232 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:08:01.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:08:01.276 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:08:01.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:08:01.319 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:08:01.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:08:01.362 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:08:01.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:08:01.395 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: Saving service mgr spec with placement smithi083=x;smithi142=y;smithi193=z;count:3 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:01.397 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:08:01.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: Saving service mgr spec with placement smithi083=x;smithi142=y;smithi193=z;count:3 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:01.440 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:08:01.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:08:01.479 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:01.485 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:08:01.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:08:01.528 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:08:01.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:08:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: Saving service mgr spec with placement smithi083=x;smithi142=y;smithi193=z;count:3 2024-09-21T03:08:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:01.571 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:08:01.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:08:01.615 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:08:01.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:08:01.659 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:08:01.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:08:01.705 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:08:01.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:08:01.754 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:08:01.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:08:01.811 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:08:01.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:08:01.862 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:08:01.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:08:01.915 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:08:01.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:08:01.965 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:08:01.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:08:02.019 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T03:08:02.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T03:08:02.073 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:08:02.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:08:02.127 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:08:02.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:08:02.170 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:08:02.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:08:02.217 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T03:08:02.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T03:08:02.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:08:00] ENGINE Bus STARTING 2024-09-21T03:08:02.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:08:00] ENGINE Serving on http://172.21.15.83:8765 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:08:00] ENGINE Serving on https://172.21.15.83:7150 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:08:00] ENGINE Bus STARTED 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: [21/Sep/2024:03:08:00] ENGINE Error in HTTPServer.serve 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:08:02.251 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.252 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:02 smithi083 ceph-mon[31219]: mgrmap e17: x(active, since 3s) 2024-09-21T03:08:02.254 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T03:08:02.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T03:08:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:02.301 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T03:08:02.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T03:08:02.357 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:08:02.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:08:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:02.396 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:08:02.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:08:02.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: [21/Sep/2024:03:08:00] ENGINE Bus STARTING 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: [21/Sep/2024:03:08:00] ENGINE Serving on http://172.21.15.83:8765 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: [21/Sep/2024:03:08:00] ENGINE Serving on https://172.21.15.83:7150 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: [21/Sep/2024:03:08:00] ENGINE Bus STARTED 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: [21/Sep/2024:03:08:00] ENGINE Error in HTTPServer.serve 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: Traceback (most recent call last): 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._run(expiration_interval) 2024-09-21T03:08:02.472 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: s = self.context.wrap_socket( 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: return self.sslsocket_class._create( 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: self.do_handshake() 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._sslobj.do_handshake() 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:08:02.473 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:02 smithi142 ceph-mon[34512]: mgrmap e17: x(active, since 3s) 2024-09-21T03:08:02.475 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:08:02.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:08:02.526 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T03:08:02.526 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi083') 2024-09-21T03:08:02.565 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: [21/Sep/2024:03:08:00] ENGINE Bus STARTING 2024-09-21T03:08:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: [21/Sep/2024:03:08:00] ENGINE Serving on http://172.21.15.83:8765 2024-09-21T03:08:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: [21/Sep/2024:03:08:00] ENGINE Serving on https://172.21.15.83:7150 2024-09-21T03:08:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: [21/Sep/2024:03:08:00] ENGINE Bus STARTED 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: [21/Sep/2024:03:08:00] ENGINE Error in HTTPServer.serve 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: Traceback (most recent call last): 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._run(expiration_interval) 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:08:02.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: s = self.context.wrap_socket( 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: return self.sslsocket_class._create( 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:08:02.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: self.do_handshake() 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._sslobj.do_handshake() 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"} : dispatch 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:02.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:02.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:02.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:02 smithi193 ceph-mon[35224]: mgrmap e17: x(active, since 3s) 2024-09-21T03:08:03.011 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:03 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3901730828' entity='client.admin' 2024-09-21T03:08:03.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:08:03.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:03 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3901730828' entity='client.admin' 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: Updating smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:03 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3901730828' entity='client.admin' 2024-09-21T03:08:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71f403fba0 msgr2=0x7f71f40420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71f403fba0 0x7f71f40420b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005d00 tx=0x7f71fc005c50 comp rx=0 tx=0).stop 2024-09-21T03:08:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f720c10de90 msgr2=0x7f720c19fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f720c10de90 0x7f720c19fa00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f720400cce0 tx=0x7f720400a620 comp rx=0 tx=0).stop 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 shutdown_connections 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.745+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71f403fba0 0x7f71f40420b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f720c10de90 0x7f720c19fa00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720c105380 0x7f720c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 --2- 172.21.15.83:0/1795706455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f720c104a30 0x7f720c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 >> 172.21.15.83:0/1795706455 conn(0x7f720c079770 msgr2=0x7f720c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 shutdown_connections 2024-09-21T03:08:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:03.746+0000 7f721284e640 1 -- 172.21.15.83:0/1795706455 wait complete. 2024-09-21T03:08:04.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:04.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:04.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:04.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:04 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1795706455' entity='client.admin' 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:04.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:04 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1795706455' entity='client.admin' 2024-09-21T03:08:04.435 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: Updating smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:04.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:04 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1795706455' entity='client.admin' 2024-09-21T03:08:04.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 -- 172.21.15.83:0/1670422473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad14103700 msgr2=0x7fad14103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 --2- 172.21.15.83:0/1670422473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad14103700 0x7fad14103b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad04009900 tx=0x7fad0402a480 comp rx=0 tx=0).stop 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 -- 172.21.15.83:0/1670422473 shutdown_connections 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 --2- 172.21.15.83:0/1670422473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad141040a0 0x7fad1410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 --2- 172.21.15.83:0/1670422473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad14103700 0x7fad14103b10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 --2- 172.21.15.83:0/1670422473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 -- 172.21.15.83:0/1670422473 >> 172.21.15.83:0/1670422473 conn(0x7fad140fdfe0 msgr2=0x7fad14100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 -- 172.21.15.83:0/1670422473 shutdown_connections 2024-09-21T03:08:04.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.571+0000 7fad1b43d640 1 -- 172.21.15.83:0/1670422473 wait complete. 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 Processor -- start 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 -- start start 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad14103700 0x7fad14199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410df70 con 0x7fad141040a0 2024-09-21T03:08:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410ddf0 con 0x7fad14103700 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad1b43d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410e0f0 con 0x7fad14102670 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad199da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad199da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48256/0 (socket says 172.21.15.83:48256) 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.572+0000 7fad199da640 1 -- 172.21.15.83:0/766514159 learned_addr learned my addr 172.21.15.83:0/766514159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad191d9640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad189d8640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad14103700 0x7fad14199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 -- 172.21.15.83:0/766514159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 msgr2=0x7fad14198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 -- 172.21.15.83:0/766514159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad14103700 msgr2=0x7fad14199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad14103700 0x7fad14199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 -- 172.21.15.83:0/766514159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419a650 con 0x7fad141040a0 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad191d9640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:04.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad199da640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00cce0 tx=0x7fad0c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00f450 con 0x7fad141040a0 2024-09-21T03:08:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7fad0c00fa50 con 0x7fad141040a0 2024-09-21T03:08:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c018960 con 0x7fad141040a0 2024-09-21T03:08:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1419a940 con 0x7fad141040a0 2024-09-21T03:08:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.573+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a0f00 con 0x7fad141040a0 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.574+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14102ae0 con 0x7fad141040a0 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.574+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fad0c0213f0 con 0x7fad141040a0 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.575+0000 7fad0a7fc640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 0x7fad00042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.575+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fad0c054a50 con 0x7fad141040a0 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.575+0000 7fad191d9640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 0x7fad00042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.575+0000 7fad191d9640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 0x7fad00042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad100058c0 tx=0x7fad10005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.580+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad0c014030 con 0x7fad141040a0 2024-09-21T03:08:04.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.772+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fad14063fc0 con 0x7fad141040a0 2024-09-21T03:08:04.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.775+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v12)=0 v12) ==== 114+0+0 (secure 0 0 0) 0x7fad0c0175c0 con 0x7fad141040a0 2024-09-21T03:08:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.776+0000 7fad0a7fc640 1 -- 172.21.15.83:0/766514159 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fad0c0166a0 con 0x7fad141040a0 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 msgr2=0x7fad00042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 0x7fad00042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad100058c0 tx=0x7fad10005850 comp rx=0 tx=0).stop 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 msgr2=0x7fad14199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00cce0 tx=0x7fad0c00a620 comp rx=0 tx=0).stop 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 shutdown_connections 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fad0003fb70 0x7fad00042080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad141040a0 0x7fad14199ec0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad14103700 0x7fad14199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 --2- 172.21.15.83:0/766514159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad14102670 0x7fad14198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.777+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 >> 172.21.15.83:0/766514159 conn(0x7fad140fdfe0 msgr2=0x7fad14106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.778+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 shutdown_connections 2024-09-21T03:08:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:04.778+0000 7fad1b43d640 1 -- 172.21.15.83:0/766514159 wait complete. 2024-09-21T03:08:05.065 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:04 smithi193 systemd[1]: Starting Ceph mgr.z for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:08:05.396 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi083') 2024-09-21T03:08:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:05 smithi142 ceph-mon[34512]: Deploying daemon mgr.z on smithi193 2024-09-21T03:08:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:05 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/766514159' entity='client.admin' 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:05 smithi193 ceph-mon[35224]: Deploying daemon mgr.z on smithi193 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:05 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/766514159' entity='client.admin' 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 podman[36975]: 2024-09-21 03:08:05.062518414 +0000 UTC m=+0.015430408 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 podman[36975]: 2024-09-21 03:08:05.160996843 +0000 UTC m=+0.113908823 container create 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 podman[36975]: 2024-09-21 03:08:05.363399116 +0000 UTC m=+0.316311099 container init 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 podman[36975]: 2024-09-21 03:08:05.366243557 +0000 UTC m=+0.319155546 container start 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:08:05.431 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 ceph-mgr[36995]: -- 172.21.15.193:0/3478272109 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56501e04b0e0 con 0x565022dee000 2024-09-21T03:08:05.432 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 bash[36975]: 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f 2024-09-21T03:08:05.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:05 smithi083 ceph-mon[31219]: Deploying daemon mgr.z on smithi193 2024-09-21T03:08:05.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:05 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/766514159' entity='client.admin' 2024-09-21T03:08:05.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 -- 172.21.15.83:0/4164791820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8108530 msgr2=0x7fd5e810ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 --2- 172.21.15.83:0/4164791820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8108530 0x7fd5e810ed60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009ca0 tx=0x7fd5e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 -- 172.21.15.83:0/4164791820 shutdown_connections 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 --2- 172.21.15.83:0/4164791820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8108530 0x7fd5e810ed60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 --2- 172.21.15.83:0/4164791820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 0x7fd5e8107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 --2- 172.21.15.83:0/4164791820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e81022a0 0x7fd5e81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.580+0000 7fd5ed427640 1 -- 172.21.15.83:0/4164791820 >> 172.21.15.83:0/4164791820 conn(0x7fd5e8079ec0 msgr2=0x7fd5e807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.581+0000 7fd5ed427640 1 -- 172.21.15.83:0/4164791820 shutdown_connections 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.581+0000 7fd5ed427640 1 -- 172.21.15.83:0/4164791820 wait complete. 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.581+0000 7fd5ed427640 1 Processor -- start 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 -- start start 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 0x7fd5e8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e8108530 0x7fd5e819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81127e0 con 0x7fd5e81022a0 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8112660 con 0x7fd5e8108530 2024-09-21T03:08:05.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5ed427640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8112960 con 0x7fd5e8107be0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48278/0 (socket says 172.21.15.83:48278) 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 -- 172.21.15.83:0/1486614888 learned_addr learned my addr 172.21.15.83:0/1486614888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e67fc640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 0x7fd5e8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e77fe640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e8108530 0x7fd5e819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 -- 172.21.15.83:0/1486614888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 msgr2=0x7fd5e8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 0x7fd5e8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 -- 172.21.15.83:0/1486614888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e8108530 msgr2=0x7fd5e819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e8108530 0x7fd5e819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.582+0000 7fd5e6ffd640 1 -- 172.21.15.83:0/1486614888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e819f260 con 0x7fd5e81022a0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5e6ffd640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800d950 tx=0x7fd5d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8017070 con 0x7fd5e81022a0 2024-09-21T03:08:05.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fd5d8009c30 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800f620 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e819f550 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.583+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a5cb0 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.584+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e81026b0 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.584+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fd5d800f7c0 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.584+0000 7fd5cffff640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 0x7fd5d0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.584+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd5d8054210 con 0x7fd5e81022a0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.584+0000 7fd5e67fc640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 0x7fd5d0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.585+0000 7fd5e67fc640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 0x7fd5d0042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005fd0 tx=0x7fd5dc00b510 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.589+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5d8010e20 con 0x7fd5e81022a0 2024-09-21T03:08:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:05.814 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:08:05 smithi193 systemd[1]: Started Ceph mgr.z for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:08:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.834+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fd5e810a320 con 0x7fd5e81022a0 2024-09-21T03:08:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.839+0000 7fd5cffff640 1 -- 172.21.15.83:0/1486614888 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v13)=0 v13) ==== 116+0+0 (secure 0 0 0) 0x7fd5d801e120 con 0x7fd5e81022a0 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 msgr2=0x7fd5d0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 0x7fd5d0042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005fd0 tx=0x7fd5dc00b510 comp rx=0 tx=0).stop 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 msgr2=0x7fd5e81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800d950 tx=0x7fd5d800de10 comp rx=0 tx=0).stop 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 shutdown_connections 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd5d003fb70 0x7fd5d0042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e8108530 0x7fd5e819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e8107be0 0x7fd5e8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 --2- 172.21.15.83:0/1486614888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e81022a0 0x7fd5e81992f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.842+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 >> 172.21.15.83:0/1486614888 conn(0x7fd5e8079ec0 msgr2=0x7fd5e8105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.843+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 shutdown_connections 2024-09-21T03:08:05.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.843+0000 7fd5ed427640 1 -- 172.21.15.83:0/1486614888 wait complete. 2024-09-21T03:08:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 -- 172.21.15.83:0/3912559768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 msgr2=0x7f1fe4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/3912559768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe4102e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009900 tx=0x7f1fd402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 -- 172.21.15.83:0/3912559768 shutdown_connections 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/3912559768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 0x7f1fe410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/3912559768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 0x7f1fe41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/3912559768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe4102e70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 -- 172.21.15.83:0/3912559768 >> 172.21.15.83:0/3912559768 conn(0x7f1fe4079730 msgr2=0x7f1fe4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 -- 172.21.15.83:0/3912559768 shutdown_connections 2024-09-21T03:08:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.913+0000 7f1fe93ed640 1 -- 172.21.15.83:0/3912559768 wait complete. 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 Processor -- start 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 -- start start 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 0x7f1fe419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 0x7f1fe419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4113840 con 0x7f1fe4102a60 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41136c0 con 0x7f1fe41083a0 2024-09-21T03:08:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe93ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41139c0 con 0x7f1fe4108cf0 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48298/0 (socket says 172.21.15.83:48298) 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.914+0000 7f1fe2ffd640 1 -- 172.21.15.83:0/1756055793 learned_addr learned my addr 172.21.15.83:0/1756055793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe37fe640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 0x7f1fe419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe27fc640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 0x7f1fe419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 -- 172.21.15.83:0/1756055793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 msgr2=0x7f1fe419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 0x7f1fe419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 -- 172.21.15.83:0/1756055793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 msgr2=0x7f1fe419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 0x7f1fe419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 -- 172.21.15.83:0/1756055793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41a0220 con 0x7f1fe4102a60 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe2ffd640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005940 tx=0x7f1fd4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd403f070 con 0x7f1fe4102a60 2024-09-21T03:08:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41a04b0 con 0x7f1fe4102a60 2024-09-21T03:08:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f1fd4031d20 con 0x7f1fe4102a60 2024-09-21T03:08:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a6c70 con 0x7f1fe4102a60 2024-09-21T03:08:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.915+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4035030 con 0x7f1fe4102a60 2024-09-21T03:08:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.917+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4102e70 con 0x7f1fe4102a60 2024-09-21T03:08:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.917+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f1fd4042070 con 0x7f1fe4102a60 2024-09-21T03:08:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.917+0000 7f1fcbfff640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 0x7f1fcc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.917+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1fd40754a0 con 0x7f1fe4102a60 2024-09-21T03:08:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.917+0000 7f1fe27fc640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 0x7f1fcc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.918+0000 7f1fe27fc640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 0x7f1fcc042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4079f10 tx=0x7f1fd800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:05.922+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fd404ce40 con 0x7f1fe4102a60 2024-09-21T03:08:06.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.114+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f1fe410aac0 con 0x7f1fe4102a60 2024-09-21T03:08:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.120+0000 7f1fcbfff640 1 -- 172.21.15.83:0/1756055793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) ==== 106+0+0 (secure 0 0 0) 0x7f1fd40303a0 con 0x7f1fe4102a60 2024-09-21T03:08:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 msgr2=0x7f1fcc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 0x7f1fcc042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4079f10 tx=0x7f1fd800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 msgr2=0x7f1fe419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005940 tx=0x7f1fd4003d50 comp rx=0 tx=0).stop 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 shutdown_connections 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1fcc03fb70 0x7f1fcc042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fe4108cf0 0x7f1fe419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fe41083a0 0x7f1fe419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 --2- 172.21.15.83:0/1756055793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe4102a60 0x7f1fe419a2b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 >> 172.21.15.83:0/1756055793 conn(0x7f1fe4079730 msgr2=0x7f1fe4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 shutdown_connections 2024-09-21T03:08:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:06.122+0000 7f1fe93ed640 1 -- 172.21.15.83:0/1756055793 wait complete. 2024-09-21T03:08:06.410 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi083') 2024-09-21T03:08:06.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: Deploying daemon mgr.y on smithi142 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1486614888' entity='client.admin' 2024-09-21T03:08:06.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:06 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1756055793' entity='client.admin' 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: Deploying daemon mgr.y on smithi142 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1486614888' entity='client.admin' 2024-09-21T03:08:06.726 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:06 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1756055793' entity='client.admin' 2024-09-21T03:08:06.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: Deploying daemon mgr.y on smithi142 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1486614888' entity='client.admin' 2024-09-21T03:08:06.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:06 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1756055793' entity='client.admin' 2024-09-21T03:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 -- 172.21.15.83:0/283180712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb81021f0 msgr2=0x7fccb8102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/283180712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb81021f0 0x7fccb8102600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009860 tx=0x7fcca802a340 comp rx=0 tx=0).stop 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 -- 172.21.15.83:0/283180712 shutdown_connections 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/283180712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb8108480 0x7fccb810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/283180712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 0x7fccb8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/283180712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb81021f0 0x7fccb8102600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 -- 172.21.15.83:0/283180712 >> 172.21.15.83:0/283180712 conn(0x7fccb8077c60 msgr2=0x7fccb80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 -- 172.21.15.83:0/283180712 shutdown_connections 2024-09-21T03:08:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.184+0000 7fccbdf1d640 1 -- 172.21.15.83:0/283180712 wait complete. 2024-09-21T03:08:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.185+0000 7fccbdf1d640 1 Processor -- start 2024-09-21T03:08:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.185+0000 7fccbdf1d640 1 -- start start 2024-09-21T03:08:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 0x7fccb819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8108480 0x7fccb819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8113860 con 0x7fccb81021f0 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81136e0 con 0x7fccb8108480 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccbdf1d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81139e0 con 0x7fccb8107b30 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48326/0 (socket says 172.21.15.83:48326) 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 -- 172.21.15.83:0/3332259838 learned_addr learned my addr 172.21.15.83:0/3332259838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb6ffd640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 0x7fccb819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb7fff640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8108480 0x7fccb819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 -- 172.21.15.83:0/3332259838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 msgr2=0x7fccb819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 0x7fccb819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 -- 172.21.15.83:0/3332259838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8108480 msgr2=0x7fccb819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8108480 0x7fccb819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.186+0000 7fccb77fe640 1 -- 172.21.15.83:0/3332259838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb81a02c0 con 0x7fccb81021f0 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccb77fe640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcca8035040 tx=0x7fcca8030770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8037040 con 0x7fccb81021f0 2024-09-21T03:08:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fcca8032c60 con 0x7fccb81021f0 2024-09-21T03:08:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb81a0550 con 0x7fccb81021f0 2024-09-21T03:08:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8003850 con 0x7fccb81021f0 2024-09-21T03:08:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.187+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb8110300 con 0x7fccb81021f0 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.188+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fcca80039f0 con 0x7fccb81021f0 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.188+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb8103470 con 0x7fccb81021f0 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.189+0000 7fccb4ff9640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 0x7fcca0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.189+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcca8073f50 con 0x7fccb81021f0 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.189+0000 7fccb6ffd640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 0x7fcca0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.189+0000 7fccb6ffd640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 0x7fcca0042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fccb8104b50 tx=0x7fccac005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.194+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcca80044a0 con 0x7fccb81021f0 2024-09-21T03:08:07.345 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 systemd[1]: Starting Ceph mgr.y for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:08:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.381+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fccb810a250 con 0x7fccb81021f0 2024-09-21T03:08:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.385+0000 7fccb4ff9640 1 -- 172.21.15.83:0/3332259838 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7fcca8044400 con 0x7fccb81021f0 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 msgr2=0x7fcca0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 0x7fcca0042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fccb8104b50 tx=0x7fccac005f90 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 msgr2=0x7fccb819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcca8035040 tx=0x7fcca8030770 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 shutdown_connections 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcca003fb70 0x7fcca0042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8108480 0x7fccb819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb8107b30 0x7fccb819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 --2- 172.21.15.83:0/3332259838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb81021f0 0x7fccb819a350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 >> 172.21.15.83:0/3332259838 conn(0x7fccb8077c60 msgr2=0x7fccb8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 shutdown_connections 2024-09-21T03:08:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.387+0000 7fccbdf1d640 1 -- 172.21.15.83:0/3332259838 wait complete. 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 -- 172.21.15.83:0/2276142844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c102a80 msgr2=0x7f3a0c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2276142844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c102a80 0x7f3a0c102e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009900 tx=0x7f39fc02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 -- 172.21.15.83:0/2276142844 shutdown_connections 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2276142844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c108d10 0x7f3a0c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2276142844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2276142844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c102a80 0x7f3a0c102e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 -- 172.21.15.83:0/2276142844 >> 172.21.15.83:0/2276142844 conn(0x7f3a0c079740 msgr2=0x7f3a0c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 -- 172.21.15.83:0/2276142844 shutdown_connections 2024-09-21T03:08:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.410+0000 7f3a125a2640 1 -- 172.21.15.83:0/2276142844 wait complete. 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.411+0000 7f3a125a2640 1 Processor -- start 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.411+0000 7f3a125a2640 1 -- start start 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.411+0000 7f3a125a2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c102a80 0x7f3a0c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a125a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a125a2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c108d10 0x7f3a0c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a125a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c112790 con 0x7f3a0c1083c0 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a125a2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c112610 con 0x7f3a0c102a80 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a125a2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c112910 con 0x7f3a0c108d10 2024-09-21T03:08:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48340/0 (socket says 172.21.15.83:48340) 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 -- 172.21.15.83:0/2628492914 learned_addr learned my addr 172.21.15.83:0/2628492914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0bfff640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c102a80 0x7f3a0c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a10b3f640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c108d10 0x7f3a0c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 -- 172.21.15.83:0/2628492914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c108d10 msgr2=0x7f3a0c19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c108d10 0x7f3a0c19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 -- 172.21.15.83:0/2628492914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c102a80 msgr2=0x7f3a0c19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c102a80 0x7f3a0c19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 -- 172.21.15.83:0/2628492914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c1a63f0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a0b7fe640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d740 tx=0x7f3a0000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.412+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00017070 con 0x7f3a0c1083c0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.413+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c1a66e0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.413+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f3a0000de50 con 0x7f3a0c1083c0 2024-09-21T03:08:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.413+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004be0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.413+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c1a6bf0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.414+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f3a00010460 con 0x7f3a0c1083c0 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.414+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0c103d00 con 0x7f3a0c1083c0 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.414+0000 7f3a097fa640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 0x7f39f40420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.414+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3a00053fa0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.414+0000 7f3a0bfff640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 0x7f39f40420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.415+0000 7f3a0bfff640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 0x7f39f40420b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005ab0 tx=0x7f39fc005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.420+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a00010d60 con 0x7f3a0c1083c0 2024-09-21T03:08:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.602+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f3a0c10aae0 con 0x7f3a0c1083c0 2024-09-21T03:08:07.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.607+0000 7f3a097fa640 1 -- 172.21.15.83:0/2628492914 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f3a00004d80 con 0x7f3a0c1083c0 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 msgr2=0x7f39f40420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 0x7f39f40420b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005ab0 tx=0x7f39fc005a20 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 msgr2=0x7f3a0c19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d740 tx=0x7f3a0000dc00 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 shutdown_connections 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39f403fba0 0x7f39f40420b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c108d10 0x7f3a0c19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a0c1083c0 0x7f3a0c19b6a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 --2- 172.21.15.83:0/2628492914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a0c102a80 0x7f3a0c19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 >> 172.21.15.83:0/2628492914 conn(0x7f3a0c079740 msgr2=0x7f3a0c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 shutdown_connections 2024-09-21T03:08:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:07.609+0000 7f3a125a2640 1 -- 172.21.15.83:0/2628492914 wait complete. 2024-09-21T03:08:07.630 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 podman[36028]: 2024-09-21 03:08:07.343793879 +0000 UTC m=+0.089316582 container create d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:08:07.630 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 podman[36028]: 2024-09-21 03:08:07.269925777 +0000 UTC m=+0.015448482 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:08:07.631 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 podman[36028]: 2024-09-21 03:08:07.554732986 +0000 UTC m=+0.300255681 container init d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:08:07.631 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 podman[36028]: 2024-09-21 03:08:07.557302988 +0000 UTC m=+0.302825690 container start d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:08:07.631 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 ceph-mgr[36047]: -- 172.21.15.142:0/352470681 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6670ef0e0 con 0x55b66be92000 2024-09-21T03:08:07.631 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 bash[36028]: d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 2024-09-21T03:08:07.927 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:08:07 smithi142 systemd[1]: Started Ceph mgr.y for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:08:08.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 -- 172.21.15.83:0/176202898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4102a60 msgr2=0x7f87e4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 --2- 172.21.15.83:0/176202898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4102a60 0x7f87e4102e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009ca0 tx=0x7f87d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 -- 172.21.15.83:0/176202898 shutdown_connections 2024-09-21T03:08:08.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 --2- 172.21.15.83:0/176202898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4108cf0 0x7f87e410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 --2- 172.21.15.83:0/176202898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 0x7f87e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 --2- 172.21.15.83:0/176202898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4102a60 0x7f87e4102e70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 -- 172.21.15.83:0/176202898 >> 172.21.15.83:0/176202898 conn(0x7f87e4079730 msgr2=0x7f87e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:08.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 -- 172.21.15.83:0/176202898 shutdown_connections 2024-09-21T03:08:08.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.366+0000 7f87eb13b640 1 -- 172.21.15.83:0/176202898 wait complete. 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 Processor -- start 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 -- start start 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4102a60 0x7f87e419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 0x7f87e419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4112770 con 0x7f87e4108cf0 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41125f0 con 0x7f87e41083a0 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87eb13b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41128f0 con 0x7f87e4102a60 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48358/0 (socket says 172.21.15.83:48358) 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e8ed7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4102a60 0x7f87e419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 -- 172.21.15.83:0/3681948067 learned_addr learned my addr 172.21.15.83:0/3681948067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e3fff640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 0x7f87e419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 -- 172.21.15.83:0/3681948067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4102a60 msgr2=0x7f87e419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4102a60 0x7f87e419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 -- 172.21.15.83:0/3681948067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 msgr2=0x7f87e419b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 0x7f87e419b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.367+0000 7f87e96d8640 1 -- 172.21.15.83:0/3681948067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41a6370 con 0x7f87e4108cf0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87e96d8640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d750 tx=0x7f87dc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87e4108cf0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f87dc005490 con 0x7f87e4108cf0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41a6660 con 0x7f87e4108cf0 2024-09-21T03:08:08.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004b40 con 0x7f87e4108cf0 2024-09-21T03:08:08.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.368+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a6ba0 con 0x7f87e4108cf0 2024-09-21T03:08:08.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.369+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f87dc004ce0 con 0x7f87e4108cf0 2024-09-21T03:08:08.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.369+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e4102e70 con 0x7f87e4108cf0 2024-09-21T03:08:08.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.369+0000 7f87e1ffb640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 0x7f87d0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.369+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f87dc053fd0 con 0x7f87e4108cf0 2024-09-21T03:08:08.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.369+0000 7f87e8ed7640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 0x7f87d0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.370+0000 7f87e8ed7640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 0x7f87d0042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f87e406ba20 tx=0x7f87d4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:08.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.374+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87dc01cd10 con 0x7f87e4108cf0 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3332259838' entity='client.admin' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2628492914' entity='client.admin' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.558+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f87e410aac0 con 0x7f87e4108cf0 2024-09-21T03:08:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 -- 172.21.15.83:0/2138226902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c1094d0 msgr2=0x7f949c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 --2- 172.21.15.83:0/2138226902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c1094d0 0x7f949c1098e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9490009900 tx=0x7f949002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 -- 172.21.15.83:0/2138226902 shutdown_connections 2024-09-21T03:08:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 --2- 172.21.15.83:0/2138226902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c109e20 0x7f949c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 --2- 172.21.15.83:0/2138226902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c1094d0 0x7f949c1098e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 --2- 172.21.15.83:0/2138226902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 -- 172.21.15.83:0/2138226902 >> 172.21.15.83:0/2138226902 conn(0x7f949c0ff600 msgr2=0x7f949c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:08.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 -- 172.21.15.83:0/2138226902 shutdown_connections 2024-09-21T03:08:08.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.559+0000 7f94a225b640 1 -- 172.21.15.83:0/2138226902 wait complete. 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 Processor -- start 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 -- start start 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 0x7f949c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1138a0 con 0x7f949c109e20 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f949c113720 con 0x7f949c1094d0 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f94a225b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f949c113a20 con 0x7f949c103b90 2024-09-21T03:08:08.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f949bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.560+0000 7f949affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 0x7f949c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48376/0 (socket says 172.21.15.83:48376) 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 -- 172.21.15.83:0/3651982979 learned_addr learned my addr 172.21.15.83:0/3651982979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 -- 172.21.15.83:0/3651982979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 msgr2=0x7f949c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 -- 172.21.15.83:0/3651982979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 msgr2=0x7f949c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 0x7f949c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 -- 172.21.15.83:0/3651982979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c1a02a0 con 0x7f949c109e20 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949affd640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 0x7f949c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949b7fe640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.561+0000 7f949bfff640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9494009fd0 tx=0x7f949400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3332259838' entity='client.admin' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2628492914' entity='client.admin' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:08.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.562+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) ==== 123+0+0 (secure 0 0 0) 0x7f87dc009bd0 con 0x7f87e4108cf0 2024-09-21T03:08:08.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.564+0000 7f87e1ffb640 1 -- 172.21.15.83:0/3681948067 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f87dc022880 con 0x7f87e4108cf0 2024-09-21T03:08:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.564+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494017070 con 0x7f949c109e20 2024-09-21T03:08:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.564+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9494003e00 con 0x7f949c109e20 2024-09-21T03:08:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.564+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c1a0590 con 0x7f949c109e20 2024-09-21T03:08:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.564+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c1a6cf0 con 0x7f949c109e20 2024-09-21T03:08:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494010460 con 0x7f949c109e20 2024-09-21T03:08:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f94940106c0 con 0x7f949c109e20 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 msgr2=0x7f87d0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 0x7f87d0042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f87e406ba20 tx=0x7f87d4005fd0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 msgr2=0x7f87e419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d750 tx=0x7f87dc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f9498ff9640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 0x7f9484042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9494055620 con 0x7f949c109e20 2024-09-21T03:08:08.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.566+0000 7f949b7fe640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 0x7f9484042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94880052c0 con 0x7f949c109e20 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 shutdown_connections 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87d003fb70 0x7f87d0042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e4108cf0 0x7f87e419bae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87e41083a0 0x7f87e419b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 --2- 172.21.15.83:0/3681948067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87e4102a60 0x7f87e419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 >> 172.21.15.83:0/3681948067 conn(0x7f87e4079730 msgr2=0x7f87e4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 shutdown_connections 2024-09-21T03:08:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f87eb13b640 1 -- 172.21.15.83:0/3681948067 wait complete. 2024-09-21T03:08:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.567+0000 7f949b7fe640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 0x7f9484042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f948c011490 tx=0x7f948c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:08.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.572+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f949400fc70 con 0x7f949c109e20 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3332259838' entity='client.admin' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2628492914' entity='client.admin' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:08.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.762+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f94880055b0 con 0x7f949c109e20 2024-09-21T03:08:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.766+0000 7f9498ff9640 1 -- 172.21.15.83:0/3651982979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f949401b400 con 0x7f949c109e20 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 msgr2=0x7f9484042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 0x7f9484042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f948c011490 tx=0x7f948c005f90 comp rx=0 tx=0).stop 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 msgr2=0x7f949c19fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9494009fd0 tx=0x7f949400b260 comp rx=0 tx=0).stop 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 shutdown_connections 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f948403fb70 0x7f9484042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f949c109e20 0x7f949c19fb70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f949c1094d0 0x7f949c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 --2- 172.21.15.83:0/3651982979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f949c103b90 0x7f949c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 >> 172.21.15.83:0/3651982979 conn(0x7f949c0ff600 msgr2=0x7f949c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 shutdown_connections 2024-09-21T03:08:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:08.768+0000 7f94a225b640 1 -- 172.21.15.83:0/3651982979 wait complete. 2024-09-21T03:08:09.265 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi083') 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 -- 172.21.15.83:0/2763308517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 msgr2=0x7f94dc116c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2763308517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc116c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009ca0 tx=0x7f94d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 -- 172.21.15.83:0/2763308517 shutdown_connections 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2763308517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc116c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2763308517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc10fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2763308517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 -- 172.21.15.83:0/2763308517 >> 172.21.15.83:0/2763308517 conn(0x7f94dc0780f0 msgr2=0x7f94dc1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 -- 172.21.15.83:0/2763308517 shutdown_connections 2024-09-21T03:08:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.415+0000 7f94e13d4640 1 -- 172.21.15.83:0/2763308517 wait complete. 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.416+0000 7f94e13d4640 1 Processor -- start 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.416+0000 7f94e13d4640 1 -- start start 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc1a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc1a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc119f90 con 0x7f94dc110420 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc119e10 con 0x7f94dc10a190 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94e13d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc11a110 con 0x7f94dc10fad0 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc1a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc1a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc1a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:50208/0 (socket says 172.21.15.83:50208) 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94da7fc640 1 -- 172.21.15.83:0/2196504681 learned_addr learned my addr 172.21.15.83:0/2196504681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc1a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33418/0 (socket says 172.21.15.83:33418) 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 -- 172.21.15.83:0/2196504681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 msgr2=0x7f94dc1a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc1a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 -- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 msgr2=0x7f94dc1a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc1a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 -- 172.21.15.83:0/2196504681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc1a6870 con 0x7f94dc10a190 2024-09-21T03:08:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.417+0000 7f94daffd640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc1a0900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9f0 tx=0x7f94cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.418+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00ccc0 con 0x7f94dc10a190 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.418+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f94cc0108c0 con 0x7f94dc10a190 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.418+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1a6b60 con 0x7f94dc10a190 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.418+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0187c0 con 0x7f94dc10a190 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.418+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1ad2c0 con 0x7f94dc10a190 2024-09-21T03:08:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f94cc018960 con 0x7f94dc10a190 2024-09-21T03:08:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc10a5a0 con 0x7f94dc10a190 2024-09-21T03:08:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94c3fff640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 0x7f94c4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f94cc0549b0 con 0x7f94dc10a190 2024-09-21T03:08:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94da7fc640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 0x7f94c4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.419+0000 7f94da7fc640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 0x7f94c4042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f94dc10c500 tx=0x7f94d000b440 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.424+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94cc01d210 con 0x7f94dc10a190 2024-09-21T03:08:09.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.611+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f94dc02e640 con 0x7f94dc10a190 2024-09-21T03:08:09.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.619+0000 7f94c3fff640 1 -- 172.21.15.83:0/2196504681 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v19)=0 v19) ==== 108+0+0 (secure 0 0 0) 0x7f94cc010d70 con 0x7f94dc10a190 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 msgr2=0x7f94c4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 0x7f94c4042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f94dc10c500 tx=0x7f94d000b440 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 msgr2=0x7f94dc1a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc1a0900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e9f0 tx=0x7f94cc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 shutdown_connections 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f94c403fb70 0x7f94c4042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94dc110420 0x7f94dc1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc10fad0 0x7f94dc1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 --2- 172.21.15.83:0/2196504681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94dc10a190 0x7f94dc1a0900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 >> 172.21.15.83:0/2196504681 conn(0x7f94dc0780f0 msgr2=0x7f94dc1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 shutdown_connections 2024-09-21T03:08:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:09.621+0000 7f94e13d4640 1 -- 172.21.15.83:0/2196504681 wait complete. 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3681948067' entity='client.admin' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3651982979' entity='client.admin' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: Reconfiguring daemon mgr.x on smithi083 2024-09-21T03:08:09.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3681948067' entity='client.admin' 2024-09-21T03:08:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3651982979' entity='client.admin' 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: Reconfiguring daemon mgr.x on smithi083 2024-09-21T03:08:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3681948067' entity='client.admin' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3651982979' entity='client.admin' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: Reconfiguring daemon mgr.x on smithi083 2024-09-21T03:08:09.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: from='client.? ' entity='client.admin' 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: Standby manager daemon z started 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:08:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:10 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: from='client.? ' entity='client.admin' 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: Standby manager daemon z started 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:08:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:10 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: from='client.? ' entity='client.admin' 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: Standby manager daemon z started 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:08:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:10 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.193:0/2821161043' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:11 smithi083 ceph-mon[31219]: mgrmap e18: x(active, since 11s), standbys: z 2024-09-21T03:08:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:11 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:08:11.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:11 smithi142 ceph-mon[34512]: mgrmap e18: x(active, since 11s), standbys: z 2024-09-21T03:08:11.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:11 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:08:12.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:11 smithi193 ceph-mon[35224]: mgrmap e18: x(active, since 11s), standbys: z 2024-09-21T03:08:12.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:11 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:08:12.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:12 smithi083 ceph-mon[31219]: Standby manager daemon y started 2024-09-21T03:08:12.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:12 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:08:12.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:12 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:12.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:12 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:08:12.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:12 smithi083 ceph-mon[31219]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:12 smithi142 ceph-mon[34512]: Standby manager daemon y started 2024-09-21T03:08:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:12 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:08:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:12 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:12 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:08:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:12 smithi142 ceph-mon[34512]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:12.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:12 smithi193 ceph-mon[35224]: Standby manager daemon y started 2024-09-21T03:08:12.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:12 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:08:12.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:12 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:08:12.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:12 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:08:12.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:12 smithi193 ceph-mon[35224]: from='mgr.? 172.21.15.142:0/4086303324' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:08:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 -- 172.21.15.83:0/3206916917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 msgr2=0x7f54501087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3206916917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f54501087e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5440009900 tx=0x7f544002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 -- 172.21.15.83:0/3206916917 shutdown_connections 2024-09-21T03:08:13.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3206916917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 0x7f545010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3206916917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f54501087e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3206916917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 0x7f5450102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.171+0000 7f5457dfd640 1 -- 172.21.15.83:0/3206916917 >> 172.21.15.83:0/3206916917 conn(0x7f5450079790 msgr2=0x7f5450079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 -- 172.21.15.83:0/3206916917 shutdown_connections 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 -- 172.21.15.83:0/3206916917 wait complete. 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 Processor -- start 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 -- start start 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 0x7f545019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.172+0000 7f5457dfd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 0x7f545019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54501139e0 con 0x7f54501083d0 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5450113860 con 0x7f5450108d20 2024-09-21T03:08:13.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5450113b60 con 0x7f5450102ae0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33440/0 (socket says 172.21.15.83:33440) 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 -- 172.21.15.83:0/3841538366 learned_addr learned my addr 172.21.15.83:0/3841538366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f545639a640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 0x7f545019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455b99640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 0x7f545019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 -- 172.21.15.83:0/3841538366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 msgr2=0x7f545019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 0x7f545019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 -- 172.21.15.83:0/3841538366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 msgr2=0x7f545019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 0x7f545019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 -- 172.21.15.83:0/3841538366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54501a0300 con 0x7f54501083d0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5455398640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5440005940 tx=0x7f54400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544003f030 con 0x7f54501083d0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54400044c0 con 0x7f54501083d0 2024-09-21T03:08:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501a0590 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440035070 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.173+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a0a70 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.174+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5450103d10 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.176+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f5440042030 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.176+0000 7f5446ffd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 0x7f543c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.177+0000 7f5455b99640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 0x7f543c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.177+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f54401060c0 con 0x7f54501083d0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.177+0000 7f5455b99640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 0x7f543c0b8a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f544c00abe0 tx=0x7f544c00a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:13.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.180+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54400ce550 con 0x7f54501083d0 2024-09-21T03:08:13.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.208 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.227 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:13.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.401+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f5450102ef0 con 0x7f54501083d0 2024-09-21T03:08:13.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.406+0000 7f5446ffd640 1 -- 172.21.15.83:0/3841538366 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f5450102ef0 con 0x7f54501083d0 2024-09-21T03:08:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 msgr2=0x7f543c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 0x7f543c0b8a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f544c00abe0 tx=0x7f544c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 msgr2=0x7f545019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5440005940 tx=0x7f54400040c0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 shutdown_connections 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f543c0b6500 0x7f543c0b8a10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450108d20 0x7f545019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54501083d0 0x7f545019a9f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 --2- 172.21.15.83:0/3841538366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5450102ae0 0x7f545019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 >> 172.21.15.83:0/3841538366 conn(0x7f5450079790 msgr2=0x7f5450106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 shutdown_connections 2024-09-21T03:08:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:13.408+0000 7f5457dfd640 1 -- 172.21.15.83:0/3841538366 wait complete. 2024-09-21T03:08:13.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:13 smithi142 ceph-mon[34512]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-09-21T03:08:13.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:08:13.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:13 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3841538366' entity='client.admin' 2024-09-21T03:08:13.977 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:13 smithi193 ceph-mon[35224]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-09-21T03:08:13.977 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:08:13.977 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:13 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3841538366' entity='client.admin' 2024-09-21T03:08:13.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:13 smithi083 ceph-mon[31219]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-09-21T03:08:13.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:08:13.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:13 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3841538366' entity='client.admin' 2024-09-21T03:08:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 -- 172.21.15.83:0/2826410452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 msgr2=0x7f298c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 --2- 172.21.15.83:0/2826410452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c103f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2988009900 tx=0x7f298802a410 comp rx=0 tx=0).stop 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 -- 172.21.15.83:0/2826410452 shutdown_connections 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 --2- 172.21.15.83:0/2826410452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c109df0 0x7f298c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 --2- 172.21.15.83:0/2826410452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c1094a0 0x7f298c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 --2- 172.21.15.83:0/2826410452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c103f70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 -- 172.21.15.83:0/2826410452 >> 172.21.15.83:0/2826410452 conn(0x7f298c07b850 msgr2=0x7f298c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:15.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.260+0000 7f2993a9d640 1 -- 172.21.15.83:0/2826410452 shutdown_connections 2024-09-21T03:08:15.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 -- 172.21.15.83:0/2826410452 wait complete. 2024-09-21T03:08:15.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 Processor -- start 2024-09-21T03:08:15.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 -- start start 2024-09-21T03:08:15.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 0x7f298c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 0x7f298c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c113990 con 0x7f298c103b60 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f298c113810 con 0x7f298c1094a0 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.261+0000 7f2993a9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f298c113b10 con 0x7f298c109df0 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33448/0 (socket says 172.21.15.83:33448) 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 -- 172.21.15.83:0/1562635495 learned_addr learned my addr 172.21.15.83:0/1562635495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f299203a640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 0x7f298c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991038640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 0x7f298c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 -- 172.21.15.83:0/1562635495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 msgr2=0x7f298c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 0x7f298c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 -- 172.21.15.83:0/1562635495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 msgr2=0x7f298c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 0x7f298c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 -- 172.21.15.83:0/1562635495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c1a0290 con 0x7f298c103b60 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991038640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 0x7f298c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f299203a640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 0x7f298c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2991839640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f29880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298803f030 con 0x7f298c103b60 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c1a0520 con 0x7f298c103b60 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2988003c90 con 0x7f298c103b60 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988035070 con 0x7f298c103b60 2024-09-21T03:08:15.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.262+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1a0a00 con 0x7f298c103b60 2024-09-21T03:08:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.263+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298c103f70 con 0x7f298c103b60 2024-09-21T03:08:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.265+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2988042030 con 0x7f298c103b60 2024-09-21T03:08:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.266+0000 7f2982ffd640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 0x7f29780b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.266+0000 7f2991038640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 0x7f29780b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.266+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2988105f60 con 0x7f298c103b60 2024-09-21T03:08:15.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.266+0000 7f2991038640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 0x7f29780b8a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f298c1064d0 tx=0x7f297c00a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:15.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.269+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29880ce3f0 con 0x7f298c103b60 2024-09-21T03:08:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.453+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f298c02e640 con 0x7f298c103b60 2024-09-21T03:08:15.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.458+0000 7f2982ffd640 1 -- 172.21.15.83:0/1562635495 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f29880d27d0 con 0x7f298c103b60 2024-09-21T03:08:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.460+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 msgr2=0x7f29780b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.460+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 0x7f29780b8a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f298c1064d0 tx=0x7f297c00a5c0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.460+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 msgr2=0x7f298c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.460+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f29880042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 shutdown_connections 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f29780b6500 0x7f29780b8a10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f298c109df0 0x7f298c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f298c1094a0 0x7f298c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 --2- 172.21.15.83:0/1562635495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c103b60 0x7f298c19a3e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 >> 172.21.15.83:0/1562635495 conn(0x7f298c07b850 msgr2=0x7f298c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 shutdown_connections 2024-09-21T03:08:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.461+0000 7f2993a9d640 1 -- 172.21.15.83:0/1562635495 wait complete. 2024-09-21T03:08:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 -- 172.21.15.83:0/2190314788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 msgr2=0x7f3f101098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 --2- 172.21.15.83:0/2190314788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f101098d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009900 tx=0x7f3f0002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 -- 172.21.15.83:0/2190314788 shutdown_connections 2024-09-21T03:08:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 --2- 172.21.15.83:0/2190314788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 0x7f3f10110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 --2- 172.21.15.83:0/2190314788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f101098d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 --2- 172.21.15.83:0/2190314788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 0x7f3f10103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 -- 172.21.15.83:0/2190314788 >> 172.21.15.83:0/2190314788 conn(0x7f3f100ff5b0 msgr2=0x7f3f10101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 -- 172.21.15.83:0/2190314788 shutdown_connections 2024-09-21T03:08:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.924+0000 7f3f17a32640 1 -- 172.21.15.83:0/2190314788 wait complete. 2024-09-21T03:08:15.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 Processor -- start 2024-09-21T03:08:15.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 -- start start 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 0x7f3f1019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 0x7f3f1019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101139b0 con 0x7f3f101094c0 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10113830 con 0x7f3f10103b80 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f17a32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10113b30 con 0x7f3f10109e10 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f14fcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.925+0000 7f3f14fcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33462/0 (socket says 172.21.15.83:33462) 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 -- 172.21.15.83:0/1634677579 learned_addr learned my addr 172.21.15.83:0/1634677579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f15fcf640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 0x7f3f1019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f157ce640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 0x7f3f1019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 -- 172.21.15.83:0/1634677579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 msgr2=0x7f3f1019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 0x7f3f1019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 -- 172.21.15.83:0/1634677579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 msgr2=0x7f3f1019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 0x7f3f1019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 -- 172.21.15.83:0/1634677579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101a02b0 con 0x7f3f101094c0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f14fcd640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005940 tx=0x7f3f000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0003f030 con 0x7f3f101094c0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f00003c90 con 0x7f3f101094c0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00035070 con 0x7f3f101094c0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f101a0540 con 0x7f3f101094c0 2024-09-21T03:08:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.926+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a0a70 con 0x7f3f101094c0 2024-09-21T03:08:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.927+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10104e00 con 0x7f3f101094c0 2024-09-21T03:08:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.929+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3f00042030 con 0x7f3f101094c0 2024-09-21T03:08:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.930+0000 7f3f067fc640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 0x7f3efc0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.930+0000 7f3f157ce640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 0x7f3efc0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.930+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3f00106250 con 0x7f3f101094c0 2024-09-21T03:08:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.931+0000 7f3f157ce640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 0x7f3efc0b8a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00abe0 tx=0x7f3f0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:15.933+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f101a0a70 con 0x7f3f101094c0 2024-09-21T03:08:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.122+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3f1019b530 con 0x7f3f101094c0 2024-09-21T03:08:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.126+0000 7f3f067fc640 1 -- 172.21.15.83:0/1634677579 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f3f1019b530 con 0x7f3f101094c0 2024-09-21T03:08:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 msgr2=0x7f3efc0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 0x7f3efc0b8a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00abe0 tx=0x7f3f0c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 msgr2=0x7f3f1019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005940 tx=0x7f3f000042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 shutdown_connections 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3efc0b6580 0x7f3efc0b8a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10109e10 0x7f3f1019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f101094c0 0x7f3f1019a9a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 --2- 172.21.15.83:0/1634677579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10103b80 0x7f3f1019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 >> 172.21.15.83:0/1634677579 conn(0x7f3f100ff5b0 msgr2=0x7f3f10108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 shutdown_connections 2024-09-21T03:08:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.128+0000 7f3f17a32640 1 -- 172.21.15.83:0/1634677579 wait complete. 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 -- 172.21.15.83:0/2285067635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 msgr2=0x7f58c410f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 --2- 172.21.15.83:0/2285067635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c410f580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009ca0 tx=0x7f58bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 -- 172.21.15.83:0/2285067635 shutdown_connections 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 --2- 172.21.15.83:0/2285067635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c410f580 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 --2- 172.21.15.83:0/2285067635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c41055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 --2- 172.21.15.83:0/2285067635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 0x7f58c4103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 -- 172.21.15.83:0/2285067635 >> 172.21.15.83:0/2285067635 conn(0x7f58c4079f90 msgr2=0x7f58c407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 -- 172.21.15.83:0/2285067635 shutdown_connections 2024-09-21T03:08:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.287+0000 7f58ca895640 1 -- 172.21.15.83:0/2285067635 wait complete. 2024-09-21T03:08:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.288+0000 7f58ca895640 1 Processor -- start 2024-09-21T03:08:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.288+0000 7f58ca895640 1 -- start start 2024-09-21T03:08:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 0x7f58c419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41139d0 con 0x7f58c4105b30 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4113850 con 0x7f58c4103210 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58ca895640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4113b50 con 0x7f58c41051e0 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33474/0 (socket says 172.21.15.83:33474) 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 -- 172.21.15.83:0/2727216283 learned_addr learned my addr 172.21.15.83:0/2727216283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c3fff640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 0x7f58c419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c37fe640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 -- 172.21.15.83:0/2727216283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 msgr2=0x7f58c419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 -- 172.21.15.83:0/2727216283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 msgr2=0x7f58c419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 0x7f58c419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c8e32640 1 -- 172.21.15.83:0/2727216283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c41a0330 con 0x7f58c4105b30 2024-09-21T03:08:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.289+0000 7f58c37fe640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58c8e32640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031040 tx=0x7f58bc032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc003b70 con 0x7f58c4105b30 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c41a05c0 con 0x7f58c4105b30 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f58bc003d10 con 0x7f58c4105b30 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc030b20 con 0x7f58c4105b30 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.290+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a6d80 con 0x7f58c4105b30 2024-09-21T03:08:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.291+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c4103620 con 0x7f58c4105b30 2024-09-21T03:08:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.293+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f58bc030cc0 con 0x7f58c4105b30 2024-09-21T03:08:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.293+0000 7f58c17fa640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 0x7f58ac0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.294+0000 7f58c3fff640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 0x7f58ac0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.294+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f58bc106970 con 0x7f58c4105b30 2024-09-21T03:08:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.294+0000 7f58c3fff640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 0x7f58ac0b89e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009bd0 tx=0x7f58b4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:16.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.297+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58bc0cee00 con 0x7f58c4105b30 2024-09-21T03:08:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.493+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f58c410aba0 con 0x7f58c4105b30 2024-09-21T03:08:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.497+0000 7f58c17fa640 1 -- 172.21.15.83:0/2727216283 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) ==== 138+0+0 (secure 0 0 0) 0x7f58bc0d31e0 con 0x7f58c4105b30 2024-09-21T03:08:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 msgr2=0x7f58ac0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 0x7f58ac0b89e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009bd0 tx=0x7f58b4005f90 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 msgr2=0x7f58c419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031040 tx=0x7f58bc032fb0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 shutdown_connections 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58ac0b64d0 0x7f58ac0b89e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c4105b30 0x7f58c419fc00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c41051e0 0x7f58c419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 --2- 172.21.15.83:0/2727216283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c4103210 0x7f58c419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 >> 172.21.15.83:0/2727216283 conn(0x7f58c4079f90 msgr2=0x7f58c4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 shutdown_connections 2024-09-21T03:08:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:16.500+0000 7f58ca895640 1 -- 172.21.15.83:0/2727216283 wait complete. 2024-09-21T03:08:16.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1562635495' entity='client.admin' 2024-09-21T03:08:16.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1634677579' entity='client.admin' 2024-09-21T03:08:16.850 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1562635495' entity='client.admin' 2024-09-21T03:08:16.850 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1634677579' entity='client.admin' 2024-09-21T03:08:16.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1562635495' entity='client.admin' 2024-09-21T03:08:16.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1634677579' entity='client.admin' 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 -- 172.21.15.83:0/1933368865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 msgr2=0x7f0c8010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 --2- 172.21.15.83:0/1933368865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8010d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009900 tx=0x7f0c6802a410 comp rx=0 tx=0).stop 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 -- 172.21.15.83:0/1933368865 shutdown_connections 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 --2- 172.21.15.83:0/1933368865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 0x7f0c80110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 --2- 172.21.15.83:0/1933368865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8010d870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 --2- 172.21.15.83:0/1933368865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 0x7f0c80104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 -- 172.21.15.83:0/1933368865 >> 172.21.15.83:0/1933368865 conn(0x7f0c80079740 msgr2=0x7f0c80079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 -- 172.21.15.83:0/1933368865 shutdown_connections 2024-09-21T03:08:17.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 -- 172.21.15.83:0/1933368865 wait complete. 2024-09-21T03:08:17.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.362+0000 7f0c86e84640 1 Processor -- start 2024-09-21T03:08:17.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 -- start start 2024-09-21T03:08:17.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 0x7f0c8019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 0x7f0c8019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113970 con 0x7f0c80105320 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801137f0 con 0x7f0c801049d0 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c86e84640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113af0 con 0x7f0c8010de30 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33502/0 (socket says 172.21.15.83:33502) 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 -- 172.21.15.83:0/4017798978 learned_addr learned my addr 172.21.15.83:0/4017798978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c84c20640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 0x7f0c8019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c85421640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 0x7f0c8019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 -- 172.21.15.83:0/4017798978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 msgr2=0x7f0c8019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 0x7f0c8019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 -- 172.21.15.83:0/4017798978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 msgr2=0x7f0c8019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 0x7f0c8019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.363+0000 7f0c7ffff640 1 -- 172.21.15.83:0/4017798978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c801a0410 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c7ffff640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005940 tx=0x7f0c680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6803f030 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0c68031800 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c801a06a0 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68035070 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.364+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801a6e60 con 0x7f0c80105320 2024-09-21T03:08:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.365+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8006bf20 con 0x7f0c80105320 2024-09-21T03:08:17.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.367+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f0c68042030 con 0x7f0c80105320 2024-09-21T03:08:17.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.368+0000 7f0c7dffb640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 0x7f0c6c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.368+0000 7f0c84c20640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 0x7f0c6c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.368+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0c68106220 con 0x7f0c80105320 2024-09-21T03:08:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.369+0000 7f0c84c20640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 0x7f0c6c0b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0c8006c620 tx=0x7f0c7000a590 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.371+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0c680ce6b0 con 0x7f0c80105320 2024-09-21T03:08:17.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.554+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f0c80108f90 con 0x7f0c80105320 2024-09-21T03:08:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.558+0000 7f0c7dffb640 1 -- 172.21.15.83:0/4017798978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v24)=0 v24) ==== 116+0+0 (secure 0 0 0) 0x7f0c680d2a90 con 0x7f0c80105320 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 msgr2=0x7f0c6c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 0x7f0c6c0b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0c8006c620 tx=0x7f0c7000a590 comp rx=0 tx=0).stop 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 msgr2=0x7f0c8019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005940 tx=0x7f0c680042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 shutdown_connections 2024-09-21T03:08:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0c6c0b6550 0x7f0c6c0b8a60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c8010de30 0x7f0c8019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c80105320 0x7f0c8019aa40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.560+0000 7f0c86e84640 1 --2- 172.21.15.83:0/4017798978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c801049d0 0x7f0c8019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.561+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 >> 172.21.15.83:0/4017798978 conn(0x7f0c80079740 msgr2=0x7f0c800ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.561+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 shutdown_connections 2024-09-21T03:08:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.561+0000 7f0c86e84640 1 -- 172.21.15.83:0/4017798978 wait complete. 2024-09-21T03:08:17.674 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:17 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2727216283' entity='client.admin' 2024-09-21T03:08:17.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 -- 172.21.15.83:0/3316421859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 msgr2=0x7f24fc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 --2- 172.21.15.83:0/3316421859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc1087b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009900 tx=0x7f24ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 -- 172.21.15.83:0/3316421859 shutdown_connections 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 --2- 172.21.15.83:0/3316421859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 0x7f24fc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 --2- 172.21.15.83:0/3316421859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc1087b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 --2- 172.21.15.83:0/3316421859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 0x7f24fc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 -- 172.21.15.83:0/3316421859 >> 172.21.15.83:0/3316421859 conn(0x7f24fc079730 msgr2=0x7f24fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 -- 172.21.15.83:0/3316421859 shutdown_connections 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.673+0000 7f25035f2640 1 -- 172.21.15.83:0/3316421859 wait complete. 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 Processor -- start 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 -- start start 2024-09-21T03:08:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 0x7f24fc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 0x7f24fc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc112840 con 0x7f24fc1083a0 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1126c0 con 0x7f24fc102a60 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f25035f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1129c0 con 0x7f24fc108cf0 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f2500b8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f2500b8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33516/0 (socket says 172.21.15.83:33516) 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.674+0000 7f2500b8d640 1 -- 172.21.15.83:0/2131780200 learned_addr learned my addr 172.21.15.83:0/2131780200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2501b8f640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 0x7f24fc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f250138e640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 0x7f24fc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 -- 172.21.15.83:0/2131780200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 msgr2=0x7f24fc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 0x7f24fc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 -- 172.21.15.83:0/2131780200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 msgr2=0x7f24fc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 0x7f24fc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 -- 172.21.15.83:0/2131780200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc1a02d0 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f2500b8d640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005940 tx=0x7f24ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec03f030 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24ec031800 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc1a0560 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec035070 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.675+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc1a0a90 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.676+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc102e70 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.678+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f24ec042030 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.678+0000 7f24f27fc640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 0x7f24e80b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.679+0000 7f250138e640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 0x7f24e80b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.679+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f24ec105fe0 con 0x7f24fc1083a0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.679+0000 7f250138e640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 0x7f24e80b8a90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005e10 tx=0x7f24f8005da0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.681+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24fc1a0a90 con 0x7f24fc1083a0 2024-09-21T03:08:17.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:17 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2727216283' entity='client.admin' 2024-09-21T03:08:17.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.862+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f24fc19b550 con 0x7f24fc1083a0 2024-09-21T03:08:17.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.866+0000 7f24f27fc640 1 -- 172.21.15.83:0/2131780200 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v25)=0 v25) ==== 127+0+0 (secure 0 0 0) 0x7f24fc19b550 con 0x7f24fc1083a0 2024-09-21T03:08:17.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 msgr2=0x7f24e80b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 0x7f24e80b8a90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005e10 tx=0x7f24f8005da0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 msgr2=0x7f24fc19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005940 tx=0x7f24ec0042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 shutdown_connections 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24e80b6580 0x7f24e80b8a90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24fc108cf0 0x7f24fc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24fc1083a0 0x7f24fc19a9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 --2- 172.21.15.83:0/2131780200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24fc102a60 0x7f24fc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 >> 172.21.15.83:0/2131780200 conn(0x7f24fc079730 msgr2=0x7f24fc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 shutdown_connections 2024-09-21T03:08:17.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:17.869+0000 7f25035f2640 1 -- 172.21.15.83:0/2131780200 wait complete. 2024-09-21T03:08:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:17 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2727216283' entity='client.admin' 2024-09-21T03:08:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:18 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4017798978' entity='client.admin' 2024-09-21T03:08:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:18 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2131780200' entity='client.admin' 2024-09-21T03:08:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 -- 172.21.15.83:0/1392319831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c1021f0 msgr2=0x7f0a4c102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 --2- 172.21.15.83:0/1392319831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c102600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009900 tx=0x7f0a3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 -- 172.21.15.83:0/1392319831 shutdown_connections 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 --2- 172.21.15.83:0/1392319831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 0x7f0a4c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 --2- 172.21.15.83:0/1392319831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c107b30 0x7f0a4c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 --2- 172.21.15.83:0/1392319831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c102600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 -- 172.21.15.83:0/1392319831 >> 172.21.15.83:0/1392319831 conn(0x7f0a4c077c60 msgr2=0x7f0a4c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 -- 172.21.15.83:0/1392319831 shutdown_connections 2024-09-21T03:08:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.649+0000 7f0a51e21640 1 -- 172.21.15.83:0/1392319831 wait complete. 2024-09-21T03:08:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 Processor -- start 2024-09-21T03:08:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 -- start start 2024-09-21T03:08:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 0x7f0a4c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c112010 con 0x7f0a4c107b30 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c111e90 con 0x7f0a4c108480 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a51e21640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c112190 con 0x7f0a4c1021f0 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a4affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a4affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33532/0 (socket says 172.21.15.83:33532) 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.650+0000 7f0a4affd640 1 -- 172.21.15.83:0/4225511845 learned_addr learned my addr 172.21.15.83:0/4225511845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4b7fe640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4bfff640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 0x7f0a4c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 -- 172.21.15.83:0/4225511845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c1021f0 msgr2=0x7f0a4c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 -- 172.21.15.83:0/4225511845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 msgr2=0x7f0a4c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 0x7f0a4c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 -- 172.21.15.83:0/4225511845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c19f240 con 0x7f0a4c107b30 2024-09-21T03:08:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a4affd640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009860 tx=0x7f0a34009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34005020 con 0x7f0a4c107b30 2024-09-21T03:08:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c19f530 con 0x7f0a4c107b30 2024-09-21T03:08:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a340047a0 con 0x7f0a4c107b30 2024-09-21T03:08:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c105940 con 0x7f0a4c107b30 2024-09-21T03:08:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.651+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34007a70 con 0x7f0a4c107b30 2024-09-21T03:08:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.653+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c102600 con 0x7f0a4c107b30 2024-09-21T03:08:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.654+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f0a34007c10 con 0x7f0a4c107b30 2024-09-21T03:08:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.655+0000 7f0a48ff9640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 0x7f0a380b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.655+0000 7f0a4b7fe640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 0x7f0a380b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.655+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0a340e6b60 con 0x7f0a4c107b30 2024-09-21T03:08:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.655+0000 7f0a4b7fe640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 0x7f0a380b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009230 tx=0x7f0a3c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.659+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a340aeff0 con 0x7f0a4c107b30 2024-09-21T03:08:18.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:18 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4017798978' entity='client.admin' 2024-09-21T03:08:18.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:18 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2131780200' entity='client.admin' 2024-09-21T03:08:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.846+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f0a4c105c60 con 0x7f0a4c107b30 2024-09-21T03:08:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.853+0000 7f0a48ff9640 1 -- 172.21.15.83:0/4225511845 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v26)=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7f0a340b33d0 con 0x7f0a4c107b30 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 msgr2=0x7f0a380b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 0x7f0a380b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009230 tx=0x7f0a3c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 msgr2=0x7f0a4c199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009860 tx=0x7f0a34009d20 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 shutdown_connections 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a380b64d0 0x7f0a380b89e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a4c108480 0x7f0a4c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4c107b30 0x7f0a4c199870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 --2- 172.21.15.83:0/4225511845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a4c1021f0 0x7f0a4c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 >> 172.21.15.83:0/4225511845 conn(0x7f0a4c077c60 msgr2=0x7f0a4c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:18.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 shutdown_connections 2024-09-21T03:08:18.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:18.855+0000 7f0a51e21640 1 -- 172.21.15.83:0/4225511845 wait complete. 2024-09-21T03:08:18.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:18 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4017798978' entity='client.admin' 2024-09-21T03:08:18.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:18 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2131780200' entity='client.admin' 2024-09-21T03:08:19.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.473+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/2281108283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc108cf0 msgr2=0x7fe5cc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:19.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.473+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/2281108283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc10f520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009ca0 tx=0x7fe5c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/2281108283 shutdown_connections 2024-09-21T03:08:19.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/2281108283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc10f520 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/2281108283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/2281108283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc102a60 0x7fe5cc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/2281108283 >> 172.21.15.83:0/2281108283 conn(0x7fe5cc079730 msgr2=0x7fe5cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/2281108283 shutdown_connections 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/2281108283 wait complete. 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 Processor -- start 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.474+0000 7fe5d3cf7640 1 -- start start 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc113960 con 0x7fe5cc102a60 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc1137e0 con 0x7fe5cc108cf0 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d3cf7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc113ae0 con 0x7fe5cc1083a0 2024-09-21T03:08:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48982/0 (socket says 172.21.15.83:48982) 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 -- 172.21.15.83:0/387445000 learned_addr learned my addr 172.21.15.83:0/387445000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1292640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d2294640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 -- 172.21.15.83:0/387445000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 msgr2=0x7fe5cc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 -- 172.21.15.83:0/387445000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 msgr2=0x7fe5cc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 -- 172.21.15.83:0/387445000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1a0420 con 0x7fe5cc102a60 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1292640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d2294640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5d1a93640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e9f0 tx=0x7fe5c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.475+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400cd80 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.476+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe5c40108c0 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.476+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40187d0 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.476+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a0710 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.476+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a6e70 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.477+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc102e70 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.479+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe5c4010430 con 0x7fe5cc102a60 2024-09-21T03:08:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.480+0000 7fe5c2ffd640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 0x7fe5b80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.480+0000 7fe5d1292640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 0x7fe5b80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.480+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe5c40e60a0 con 0x7fe5cc102a60 2024-09-21T03:08:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.481+0000 7fe5d1292640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 0x7fe5b80b8ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005fd0 tx=0x7fe5bc00b5d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.482+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5c40ae490 con 0x7fe5cc102a60 2024-09-21T03:08:19.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.671+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe5cc10ab40 con 0x7fe5cc102a60 2024-09-21T03:08:19.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.677+0000 7fe5c2ffd640 1 -- 172.21.15.83:0/387445000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fe5c40b2870 con 0x7fe5cc102a60 2024-09-21T03:08:19.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.679+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 msgr2=0x7fe5b80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:19.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.679+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 0x7fe5b80b8ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005fd0 tx=0x7fe5bc00b5d0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.679+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 msgr2=0x7fe5cc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:19.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.679+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e9f0 tx=0x7fe5c400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 shutdown_connections 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe5b80b65d0 0x7fe5b80b8ae0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc108cf0 0x7fe5cc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1083a0 0x7fe5cc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 --2- 172.21.15.83:0/387445000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5cc102a60 0x7fe5cc19a4b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 >> 172.21.15.83:0/387445000 conn(0x7fe5cc079730 msgr2=0x7fe5cc1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 shutdown_connections 2024-09-21T03:08:19.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.680+0000 7fe5d3cf7640 1 -- 172.21.15.83:0/387445000 wait complete. 2024-09-21T03:08:19.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:19 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4225511845' entity='client.admin' 2024-09-21T03:08:19.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:19 smithi083 ceph-mon[31219]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:19.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:19 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3606182410' entity='client.admin' 2024-09-21T03:08:19.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:19 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/387445000' entity='client.admin' 2024-09-21T03:08:20.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:19 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4225511845' entity='client.admin' 2024-09-21T03:08:20.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:19 smithi142 ceph-mon[34512]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:20.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:19 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3606182410' entity='client.admin' 2024-09-21T03:08:20.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:19 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/387445000' entity='client.admin' 2024-09-21T03:08:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:19 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4225511845' entity='client.admin' 2024-09-21T03:08:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:19 smithi193 ceph-mon[35224]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:19 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3606182410' entity='client.admin' 2024-09-21T03:08:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:19 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/387445000' entity='client.admin' 2024-09-21T03:08:20.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 -- 172.21.15.83:0/4125089136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01048c0 msgr2=0x7fb7a0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:20.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 --2- 172.21.15.83:0/4125089136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01048c0 0x7fb7a0104cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009900 tx=0x7fb79c02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:20.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 -- 172.21.15.83:0/4125089136 shutdown_connections 2024-09-21T03:08:20.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 --2- 172.21.15.83:0/4125089136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a010de70 0x7fb7a0110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 --2- 172.21.15.83:0/4125089136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a01052f0 0x7fb7a010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 --2- 172.21.15.83:0/4125089136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01048c0 0x7fb7a0104cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 -- 172.21.15.83:0/4125089136 >> 172.21.15.83:0/4125089136 conn(0x7fb7a0079770 msgr2=0x7fb7a0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:20.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.485+0000 7fb7a7633640 1 -- 172.21.15.83:0/4125089136 shutdown_connections 2024-09-21T03:08:20.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 -- 172.21.15.83:0/4125089136 wait complete. 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 Processor -- start 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 -- start start 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a01048c0 0x7fb7a019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.486+0000 7fb7a7633640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a010de70 0x7fb7a019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a7633640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0113a50 con 0x7fb7a01052f0 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a7633640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01138d0 con 0x7fb7a01048c0 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a7633640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0113bd0 con 0x7fb7a010de70 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49006/0 (socket says 172.21.15.83:49006) 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 -- 172.21.15.83:0/1386562894 learned_addr learned my addr 172.21.15.83:0/1386562894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a53cf640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a01048c0 0x7fb7a019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a5bd0640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a010de70 0x7fb7a019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 -- 172.21.15.83:0/1386562894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a010de70 msgr2=0x7fb7a019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a010de70 0x7fb7a019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 -- 172.21.15.83:0/1386562894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a01048c0 msgr2=0x7fb7a019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a01048c0 0x7fb7a019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 -- 172.21.15.83:0/1386562894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a01a03f0 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a4bce640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb79000d750 tx=0x7fb79000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790017070 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb7900054b0 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a01a06e0 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790004bd0 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.487+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a6e40 con 0x7fb7a01052f0 2024-09-21T03:08:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.488+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a006c200 con 0x7fb7a01052f0 2024-09-21T03:08:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.490+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fb790009c30 con 0x7fb7a01052f0 2024-09-21T03:08:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.490+0000 7fb7967fc640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 0x7fb78c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.491+0000 7fb7a53cf640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 0x7fb78c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:20.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.491+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb7900e5050 con 0x7fb7a01052f0 2024-09-21T03:08:20.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.491+0000 7fb7a53cf640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 0x7fb78c0b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009230 tx=0x7fb79c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.495+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7900b18c0 con 0x7fb7a01052f0 2024-09-21T03:08:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.687+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fb7a0109020 con 0x7fb7a01052f0 2024-09-21T03:08:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.691+0000 7fb7967fc640 1 -- 172.21.15.83:0/1386562894 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v29)=0 v29) ==== 119+0+0 (secure 0 0 0) 0x7fb7900b6800 con 0x7fb7a01052f0 2024-09-21T03:08:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 msgr2=0x7fb78c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 0x7fb78c0b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009230 tx=0x7fb79c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 msgr2=0x7fb7a019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb79000d750 tx=0x7fb79000dc10 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 shutdown_connections 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb78c0b6550 0x7fb78c0b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a010de70 0x7fb7a019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a01052f0 0x7fb7a019aa20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 --2- 172.21.15.83:0/1386562894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a01048c0 0x7fb7a019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 >> 172.21.15.83:0/1386562894 conn(0x7fb7a0079770 msgr2=0x7fb7a0100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 shutdown_connections 2024-09-21T03:08:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.694+0000 7fb7a7633640 1 -- 172.21.15.83:0/1386562894 wait complete. 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:21.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:08:21.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1386562894' entity='client.admin' 2024-09-21T03:08:21.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2920344021' entity='client.admin' 2024-09-21T03:08:21.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.222 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1386562894' entity='client.admin' 2024-09-21T03:08:21.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2920344021' entity='client.admin' 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: Reconfiguring daemon mon.b on smithi142 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1386562894' entity='client.admin' 2024-09-21T03:08:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2920344021' entity='client.admin' 2024-09-21T03:08:21.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 -- 172.21.15.83:0/2401712953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 msgr2=0x7f10cc1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 --2- 172.21.15.83:0/2401712953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc1035d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009900 tx=0x7f10bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 -- 172.21.15.83:0/2401712953 shutdown_connections 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 --2- 172.21.15.83:0/2401712953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 0x7f10cc10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 --2- 172.21.15.83:0/2401712953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 0x7f10cc1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 --2- 172.21.15.83:0/2401712953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc1035d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 -- 172.21.15.83:0/2401712953 >> 172.21.15.83:0/2401712953 conn(0x7f10cc077c60 msgr2=0x7f10cc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 -- 172.21.15.83:0/2401712953 shutdown_connections 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.637+0000 7f10d0968640 1 -- 172.21.15.83:0/2401712953 wait complete. 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.638+0000 7f10d0968640 1 Processor -- start 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.638+0000 7f10d0968640 1 -- start start 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 0x7f10cc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 0x7f10cc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112860 con 0x7f10cc1031c0 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1126e0 con 0x7f10cc105ae0 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10d0968640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1129e0 con 0x7f10cc105190 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49026/0 (socket says 172.21.15.83:49026) 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 -- 172.21.15.83:0/2920344021 learned_addr learned my addr 172.21.15.83:0/2920344021 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10cad9d640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 0x7f10cc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10c9d9b640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 0x7f10cc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 -- 172.21.15.83:0/2920344021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 msgr2=0x7f10cc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 0x7f10cc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 -- 172.21.15.83:0/2920344021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 msgr2=0x7f10cc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 0x7f10cc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 -- 172.21.15.83:0/2920344021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19f200 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.639+0000 7f10ca59c640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006010 tx=0x7f10bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.640+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc03f030 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.640+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19f490 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.640+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f10bc003c90 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.640+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc035070 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.640+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a5c50 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.641+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc1035d0 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.642+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f10bc042030 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.643+0000 7f10bb7fe640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 0x7f10b40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.643+0000 7f10c9d9b640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 0x7f10b40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.643+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f10bc106250 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.644+0000 7f10c9d9b640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 0x7f10b40b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.651+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10bc0ce6e0 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.842+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f10cc10aa60 con 0x7f10cc1031c0 2024-09-21T03:08:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.847+0000 7f10bb7fe640 1 -- 172.21.15.83:0/2920344021 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f10bc0d2ac0 con 0x7f10cc1031c0 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 msgr2=0x7f10b40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 0x7f10b40b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005850 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 msgr2=0x7f10cc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006010 tx=0x7f10bc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 shutdown_connections 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f10b40b64d0 0x7f10b40b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10cc105ae0 0x7f10cc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc105190 0x7f10cc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 --2- 172.21.15.83:0/2920344021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10cc1031c0 0x7f10cc199290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 >> 172.21.15.83:0/2920344021 conn(0x7f10cc077c60 msgr2=0x7f10cc107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 shutdown_connections 2024-09-21T03:08:21.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:20.850+0000 7f10d0968640 1 -- 172.21.15.83:0/2920344021 wait complete. 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 -- 172.21.15.83:0/1152468710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f010de10 msgr2=0x7fc8f01102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/1152468710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f010de10 0x7fc8f01102f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009ca0 tx=0x7fc8e802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 -- 172.21.15.83:0/1152468710 shutdown_connections 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/1152468710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f010de10 0x7fc8f01102f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/1152468710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f0105290 0x7fc8f010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/1152468710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 0x7fc8f0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 -- 172.21.15.83:0/1152468710 >> 172.21.15.83:0/1152468710 conn(0x7fc8f0079730 msgr2=0x7fc8f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 -- 172.21.15.83:0/1152468710 shutdown_connections 2024-09-21T03:08:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.940+0000 7fc8f5cce640 1 -- 172.21.15.83:0/1152468710 wait complete. 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.941+0000 7fc8f5cce640 1 Processor -- start 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 -- start start 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 0x7fc8f019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f010de10 0x7fc8f019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f01139a0 con 0x7fc8f0105290 2024-09-21T03:08:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f0113820 con 0x7fc8f010de10 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8f5cce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f0113b20 con 0x7fc8f0104860 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49034/0 (socket says 172.21.15.83:49034) 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 -- 172.21.15.83:0/2445546686 learned_addr learned my addr 172.21.15.83:0/2445546686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8ef7fe640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 0x7fc8f019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8effff640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f010de10 0x7fc8f019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 -- 172.21.15.83:0/2445546686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 msgr2=0x7fc8f019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 0x7fc8f019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 -- 172.21.15.83:0/2445546686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f010de10 msgr2=0x7fc8f019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f010de10 0x7fc8f019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.942+0000 7fc8eeffd640 1 -- 172.21.15.83:0/2445546686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f01a02a0 con 0x7fc8f0105290 2024-09-21T03:08:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.943+0000 7fc8eeffd640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c8c0 tx=0x7fc8e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.945+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007bb0 con 0x7fc8f0105290 2024-09-21T03:08:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.945+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f01a0590 con 0x7fc8f0105290 2024-09-21T03:08:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.945+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8e4007d50 con 0x7fc8f0105290 2024-09-21T03:08:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.945+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4018660 con 0x7fc8f0105290 2024-09-21T03:08:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.945+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01a6db0 con 0x7fc8f0105290 2024-09-21T03:08:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.946+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f0104c70 con 0x7fc8f0105290 2024-09-21T03:08:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.947+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc8e400f450 con 0x7fc8f0105290 2024-09-21T03:08:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.948+0000 7fc8ecff9640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 0x7fc8d80b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.948+0000 7fc8ef7fe640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 0x7fc8d80b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.948+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc8e40e57b0 con 0x7fc8f0105290 2024-09-21T03:08:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.949+0000 7fc8ef7fe640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 0x7fc8d80b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009990 tx=0x7fc8e0009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:21.955+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8e4014030 con 0x7fc8f0105290 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- 172.21.15.83:0/2966731515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3088102a60 msgr2=0x7f3088102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 --2- 172.21.15.83:0/2966731515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3088102a60 0x7f3088102e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3084009900 tx=0x7f308402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- 172.21.15.83:0/2966731515 shutdown_connections 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 --2- 172.21.15.83:0/2966731515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088108cf0 0x7f308810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 --2- 172.21.15.83:0/2966731515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30881083a0 0x7f30881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 --2- 172.21.15.83:0/2966731515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3088102a60 0x7f3088102e70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- 172.21.15.83:0/2966731515 >> 172.21.15.83:0/2966731515 conn(0x7f3088079730 msgr2=0x7f3088079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- 172.21.15.83:0/2966731515 shutdown_connections 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- 172.21.15.83:0/2966731515 wait complete. 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 Processor -- start 2024-09-21T03:08:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.195+0000 7f308f251640 1 -- start start 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088102a60 0x7f308819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3088108cf0 0x7f308819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3088112840 con 0x7f30881083a0 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30881126c0 con 0x7f3088108cf0 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308f251640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30881129c0 con 0x7f3088102a60 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48970/0 (socket says 172.21.15.83:48970) 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 -- 172.21.15.83:0/3606182410 learned_addr learned my addr 172.21.15.83:0/3606182410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308d7ee640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3088108cf0 0x7f308819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f308cfed640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088102a60 0x7f308819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 -- 172.21.15.83:0/3606182410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088102a60 msgr2=0x7f308819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088102a60 0x7f308819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 -- 172.21.15.83:0/3606182410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3088108cf0 msgr2=0x7f308819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3088108cf0 0x7f308819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 -- 172.21.15.83:0/3606182410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30881a6430 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.196+0000 7f307ffff640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f307800c970 tx=0x7f307800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.197+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078014070 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.197+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3078004440 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.197+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078003d50 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.197+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30881a6720 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.197+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30881a6cb0 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.198+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3088102e70 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.200+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3078017070 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.201+0000 7f307dffb640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 0x7f30700b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.201+0000 7f308cfed640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 0x7f30700b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.201+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f30780e59c0 con 0x7f30881083a0 2024-09-21T03:08:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.201+0000 7f308cfed640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 0x7f30700b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3084000c00 tx=0x7f3084005de0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.204+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f307801e020 con 0x7f30881083a0 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.393+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f308806b2d0 con 0x7f30881083a0 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.399+0000 7f307dffb640 1 -- 172.21.15.83:0/3606182410 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v27)=0 v27) ==== 125+0+0 (secure 0 0 0) 0x7f308806b2d0 con 0x7f30881083a0 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 msgr2=0x7f30700b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 0x7f30700b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3084000c00 tx=0x7f3084005de0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 msgr2=0x7f308819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f307800c970 tx=0x7f307800ce30 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 shutdown_connections 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30700b6550 0x7f30700b8a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3088108cf0 0x7f308819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30881083a0 0x7f308819b6f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 --2- 172.21.15.83:0/3606182410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3088102a60 0x7f308819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 >> 172.21.15.83:0/3606182410 conn(0x7f3088079730 msgr2=0x7f3088101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 shutdown_connections 2024-09-21T03:08:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:19.402+0000 7f308f251640 1 -- 172.21.15.83:0/3606182410 wait complete. 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:22.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:22.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.135+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fc8f01089d0 con 0x7fc8f0105290 2024-09-21T03:08:22.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.139+0000 7fc8ecff9640 1 -- 172.21.15.83:0/2445546686 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v31)=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7fc8e40adc40 con 0x7fc8f0105290 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 msgr2=0x7fc8d80b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 0x7fc8d80b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009990 tx=0x7fc8e0009110 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 msgr2=0x7fc8f019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c8c0 tx=0x7fc8e400ccc0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 shutdown_connections 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8d80b6500 0x7fc8d80b8a10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f010de10 0x7fc8f019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f0105290 0x7fc8f019a990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 --2- 172.21.15.83:0/2445546686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0104860 0x7fc8f019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 >> 172.21.15.83:0/2445546686 conn(0x7fc8f0079730 msgr2=0x7fc8f00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 shutdown_connections 2024-09-21T03:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.141+0000 7fc8f5cce640 1 -- 172.21.15.83:0/2445546686 wait complete. 2024-09-21T03:08:22.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:08:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:08:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 -- 172.21.15.83:0/3876124514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc944109e10 msgr2=0x7fc944110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 --2- 172.21.15.83:0/3876124514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc944109e10 0x7fc944110640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009ca0 tx=0x7fc93c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 -- 172.21.15.83:0/3876124514 shutdown_connections 2024-09-21T03:08:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 --2- 172.21.15.83:0/3876124514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc944109e10 0x7fc944110640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 --2- 172.21.15.83:0/3876124514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9441094c0 0x7fc9441098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 --2- 172.21.15.83:0/3876124514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944103b80 0x7fc944103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 -- 172.21.15.83:0/3876124514 >> 172.21.15.83:0/3876124514 conn(0x7fc9440ff610 msgr2=0x7fc944101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 -- 172.21.15.83:0/3876124514 shutdown_connections 2024-09-21T03:08:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.906+0000 7fc94aa81640 1 -- 172.21.15.83:0/3876124514 wait complete. 2024-09-21T03:08:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 Processor -- start 2024-09-21T03:08:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 -- start start 2024-09-21T03:08:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc944103b80 0x7fc94419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 0x7fc94419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441139b0 con 0x7fc9441094c0 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113830 con 0x7fc944109e10 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94aa81640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113b30 con 0x7fc944103b80 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc943fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc943fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49062/0 (socket says 172.21.15.83:49062) 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc943fff640 1 -- 172.21.15.83:0/987834697 learned_addr learned my addr 172.21.15.83:0/987834697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.907+0000 7fc94881d640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc944103b80 0x7fc94419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc94901e640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 0x7fc94419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 -- 172.21.15.83:0/987834697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc944103b80 msgr2=0x7fc94419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc944103b80 0x7fc94419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 -- 172.21.15.83:0/987834697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 msgr2=0x7fc94419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 0x7fc94419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 -- 172.21.15.83:0/987834697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9441a0250 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc94901e640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 0x7fc94419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc943fff640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc93000c8c0 tx=0x7fc93000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930007bb0 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9441a0540 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc930007d50 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930018660 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.908+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9441a6d60 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.909+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc944103f90 con 0x7fc9441094c0 2024-09-21T03:08:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.911+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc93000f450 con 0x7fc9441094c0 2024-09-21T03:08:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.912+0000 7fc941ffb640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 0x7fc9340b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.912+0000 7fc94881d640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 0x7fc9340b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.912+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc9300e5a50 con 0x7fc9441094c0 2024-09-21T03:08:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.913+0000 7fc94881d640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 0x7fc9340b8a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc93800abe0 tx=0x7fc93800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:22.915+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc930014030 con 0x7fc9441094c0 2024-09-21T03:08:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.098+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fc94419b4d0 con 0x7fc9441094c0 2024-09-21T03:08:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.099+0000 7fc941ffb640 1 -- 172.21.15.83:0/987834697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v31)=0 v31) ==== 106+0+0 (secure 0 0 0) 0x7fc94419b4d0 con 0x7fc9441094c0 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 msgr2=0x7fc9340b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 0x7fc9340b8a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc93800abe0 tx=0x7fc93800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 msgr2=0x7fc94419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc93000c8c0 tx=0x7fc93000ccc0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 shutdown_connections 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc9340b6500 0x7fc9340b8a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc944109e10 0x7fc94419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9441094c0 0x7fc94419a940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 --2- 172.21.15.83:0/987834697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc944103b80 0x7fc94419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 >> 172.21.15.83:0/987834697 conn(0x7fc9440ff610 msgr2=0x7fc944108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 shutdown_connections 2024-09-21T03:08:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.101+0000 7fc94aa81640 1 -- 172.21.15.83:0/987834697 wait complete. 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 -- 172.21.15.83:0/343801963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 msgr2=0x7fadb8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 --2- 172.21.15.83:0/343801963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb8102e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009900 tx=0x7fadb402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 -- 172.21.15.83:0/343801963 shutdown_connections 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 --2- 172.21.15.83:0/343801963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 0x7fadb810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 --2- 172.21.15.83:0/343801963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 0x7fadb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 --2- 172.21.15.83:0/343801963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb8102e70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.116+0000 7fadbf44b640 1 -- 172.21.15.83:0/343801963 >> 172.21.15.83:0/343801963 conn(0x7fadb8079730 msgr2=0x7fadb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.117+0000 7fadbf44b640 1 -- 172.21.15.83:0/343801963 shutdown_connections 2024-09-21T03:08:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.117+0000 7fadbf44b640 1 -- 172.21.15.83:0/343801963 wait complete. 2024-09-21T03:08:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.117+0000 7fadbf44b640 1 Processor -- start 2024-09-21T03:08:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.117+0000 7fadbf44b640 1 -- start start 2024-09-21T03:08:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 0x7fadb819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 0x7fadb81a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8112840 con 0x7fadb8102a60 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81126c0 con 0x7fadb81083a0 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbf44b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81129c0 con 0x7fadb8108cf0 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49072/0 (socket says 172.21.15.83:49072) 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 -- 172.21.15.83:0/3200326084 learned_addr learned my addr 172.21.15.83:0/3200326084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd9e8640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 0x7fadb81a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbc9e6640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 0x7fadb819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 -- 172.21.15.83:0/3200326084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 msgr2=0x7fadb81a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 0x7fadb81a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 -- 172.21.15.83:0/3200326084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 msgr2=0x7fadb819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 0x7fadb819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.118+0000 7fadbd1e7640 1 -- 172.21.15.83:0/3200326084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb81a2550 con 0x7fadb8102a60 2024-09-21T03:08:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadbd1e7640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006010 tx=0x7fadb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb403f030 con 0x7fadb8102a60 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb81a27e0 con 0x7fadb8102a60 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fadb4003c90 con 0x7fadb8102a60 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4035070 con 0x7fadb8102a60 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.119+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81a8fa0 con 0x7fadb8102a60 2024-09-21T03:08:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.120+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8102e70 con 0x7fadb8102a60 2024-09-21T03:08:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.123+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fadb4042030 con 0x7fadb8102a60 2024-09-21T03:08:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.123+0000 7fadae7fc640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 0x7fada40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.124+0000 7fadbc9e6640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 0x7fada40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.124+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fadb4106100 con 0x7fadb8102a60 2024-09-21T03:08:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.124+0000 7fadbc9e6640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 0x7fada40b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fadb8100f60 tx=0x7fada800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.126+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fadb40d2970 con 0x7fadb8102a60 2024-09-21T03:08:23.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.311+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fadb819d710 con 0x7fadb8102a60 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: Reconfiguring daemon mon.c on smithi193 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2445546686' entity='client.admin' 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:23.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:23.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:23.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.312+0000 7fadae7fc640 1 -- 172.21.15.83:0/3200326084 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v31)=0 v31) ==== 108+0+0 (secure 0 0 0) 0x7fadb40d78b0 con 0x7fadb8102a60 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 msgr2=0x7fada40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 0x7fada40b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fadb8100f60 tx=0x7fada800b390 comp rx=0 tx=0).stop 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 msgr2=0x7fadb819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006010 tx=0x7fadb40042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 shutdown_connections 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fada40b64d0 0x7fada40b89e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8108cf0 0x7fadb81a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadb81083a0 0x7fadb819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 --2- 172.21.15.83:0/3200326084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8102a60 0x7fadb819c5e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:23.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 >> 172.21.15.83:0/3200326084 conn(0x7fadb8079730 msgr2=0x7fadb806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:23.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.315+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 shutdown_connections 2024-09-21T03:08:23.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:23.316+0000 7fadbf44b640 1 -- 172.21.15.83:0/3200326084 wait complete. 2024-09-21T03:08:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:08:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: Reconfiguring daemon mon.c on smithi193 2024-09-21T03:08:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2445546686' entity='client.admin' 2024-09-21T03:08:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: Reconfiguring daemon mon.c on smithi193 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2445546686' entity='client.admin' 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:24.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:24 smithi193 ceph-mon[35224]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:24.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:24 smithi083 ceph-mon[31219]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:24 smithi142 ceph-mon[34512]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 -- 172.21.15.83:0/4059349791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790103b60 msgr2=0x7f2790103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 --2- 172.21.15.83:0/4059349791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790103b60 0x7f2790103f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 -- 172.21.15.83:0/4059349791 shutdown_connections 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 --2- 172.21.15.83:0/4059349791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2790109df0 0x7f2790110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 --2- 172.21.15.83:0/4059349791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27901094a0 0x7f27901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 --2- 172.21.15.83:0/4059349791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790103b60 0x7f2790103f70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 -- 172.21.15.83:0/4059349791 >> 172.21.15.83:0/4059349791 conn(0x7f279007b850 msgr2=0x7f279007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.178+0000 7f27977c7640 1 -- 172.21.15.83:0/4059349791 shutdown_connections 2024-09-21T03:08:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.179+0000 7f27977c7640 1 -- 172.21.15.83:0/4059349791 wait complete. 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.179+0000 7f27977c7640 1 Processor -- start 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.179+0000 7f27977c7640 1 -- start start 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2790103b60 0x7f279019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27901094a0 0x7f279019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2790113990 con 0x7f2790109df0 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2790113810 con 0x7f2790103b60 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f27977c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2790113b10 con 0x7f27901094a0 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49094/0 (socket says 172.21.15.83:49094) 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 -- 172.21.15.83:0/948454491 learned_addr learned my addr 172.21.15.83:0/948454491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795563640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2790103b60 0x7f279019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2794d62640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27901094a0 0x7f279019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 -- 172.21.15.83:0/948454491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27901094a0 msgr2=0x7f279019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27901094a0 0x7f279019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 -- 172.21.15.83:0/948454491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2790103b60 msgr2=0x7f279019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2790103b60 0x7f279019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 -- 172.21.15.83:0/948454491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27901a0270 con 0x7f2790109df0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.180+0000 7f2795d64640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f278800cad0 tx=0x7f278800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.181+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278800ae20 con 0x7f2790109df0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.181+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2788005020 con 0x7f2790109df0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.181+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901a0560 con 0x7f2790109df0 2024-09-21T03:08:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.181+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278800f6b0 con 0x7f2790109df0 2024-09-21T03:08:26.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.181+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a6d80 con 0x7f2790109df0 2024-09-21T03:08:26.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.182+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2790103f70 con 0x7f2790109df0 2024-09-21T03:08:26.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.183+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2788004a40 con 0x7f2790109df0 2024-09-21T03:08:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.184+0000 7f27827fc640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 0x7f277c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.184+0000 7f2795563640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 0x7f277c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.184+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f27880e5a30 con 0x7f2790109df0 2024-09-21T03:08:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.184+0000 7f2795563640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 0x7f277c0b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.187+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2788014030 con 0x7f2790109df0 2024-09-21T03:08:26.355 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:26 smithi083 ceph-mon[31219]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/2931552051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 msgr2=0x7fc0cc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/2931552051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc10f520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009ca0 tx=0x7fc0c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/2931552051 shutdown_connections 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/2931552051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc10f520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/2931552051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 0x7fc0cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/2931552051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 0x7fc0cc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/2931552051 >> 172.21.15.83:0/2931552051 conn(0x7fc0cc079730 msgr2=0x7fc0cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/2931552051 shutdown_connections 2024-09-21T03:08:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.362+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/2931552051 wait complete. 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 Processor -- start 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 -- start start 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 0x7fc0cc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 0x7fc0cc19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc112840 con 0x7fc0cc108cf0 2024-09-21T03:08:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc1126c0 con 0x7fc0cc102a60 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d2ce7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc1129c0 con 0x7fc0cc1083a0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d1284640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d1284640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49116/0 (socket says 172.21.15.83:49116) 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d1284640 1 -- 172.21.15.83:0/1108434081 learned_addr learned my addr 172.21.15.83:0/1108434081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0cbfff640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 0x7fc0cc19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.363+0000 7fc0d0a83640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 0x7fc0cc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 -- 172.21.15.83:0/1108434081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 msgr2=0x7fc0cc19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 0x7fc0cc19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 -- 172.21.15.83:0/1108434081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 msgr2=0x7fc0cc19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 0x7fc0cc19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 -- 172.21.15.83:0/1108434081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc1a02d0 con 0x7fc0cc108cf0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d1284640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006010 tx=0x7fc0c40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4003b70 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc0c4003d10 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1a0560 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004600 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.364+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc1a6d20 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.365+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc102e70 con 0x7fc0cc108cf0 2024-09-21T03:08:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.367+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc0c4004b40 con 0x7fc0cc108cf0 2024-09-21T03:08:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.367+0000 7fc0c9ffb640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 0x7fc0bc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.368+0000 7fc0d0a83640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 0x7fc0bc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.368+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc0c4106670 con 0x7fc0cc108cf0 2024-09-21T03:08:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.368+0000 7fc0d0a83640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 0x7fc0bc0b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000bbe0 tx=0x7fc0c000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.371+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0c40ceb00 con 0x7fc0cc108cf0 2024-09-21T03:08:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.375+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f279019b4f0 con 0x7f2790109df0 2024-09-21T03:08:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.379+0000 7f27827fc640 1 -- 172.21.15.83:0/948454491 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v32)=0 v32) ==== 120+0+0 (secure 0 0 0) 0x7f279019b4f0 con 0x7f2790109df0 2024-09-21T03:08:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 msgr2=0x7f277c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 0x7f277c0b8a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 msgr2=0x7f279019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f278800cad0 tx=0x7f278800cf90 comp rx=0 tx=0).stop 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 shutdown_connections 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f277c0b6580 0x7f277c0b8a90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2790109df0 0x7f279019fc00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27901094a0 0x7f279019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 --2- 172.21.15.83:0/948454491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2790103b60 0x7f279019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 >> 172.21.15.83:0/948454491 conn(0x7f279007b850 msgr2=0x7f2790107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 shutdown_connections 2024-09-21T03:08:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.381+0000 7f27977c7640 1 -- 172.21.15.83:0/948454491 wait complete. 2024-09-21T03:08:26.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:26 smithi142 ceph-mon[34512]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.555+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fc0cc19b490 con 0x7fc0cc108cf0 2024-09-21T03:08:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.561+0000 7fc0c9ffb640 1 -- 172.21.15.83:0/1108434081 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v33)=0 v33) ==== 115+0+0 (secure 0 0 0) 0x7fc0cc19b490 con 0x7fc0cc108cf0 2024-09-21T03:08:26.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:26 smithi193 ceph-mon[35224]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 msgr2=0x7fc0bc0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 0x7fc0bc0b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000bbe0 tx=0x7fc0c000b390 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 msgr2=0x7fc0cc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006010 tx=0x7fc0c40093d0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 shutdown_connections 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0bc0b6550 0x7fc0bc0b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0cc108cf0 0x7fc0cc19fba0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc1083a0 0x7fc0cc19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 --2- 172.21.15.83:0/1108434081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0cc102a60 0x7fc0cc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 >> 172.21.15.83:0/1108434081 conn(0x7fc0cc079730 msgr2=0x7fc0cc06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 shutdown_connections 2024-09-21T03:08:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:26.564+0000 7fc0d2ce7640 1 -- 172.21.15.83:0/1108434081 wait complete. 2024-09-21T03:08:27.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/948454491' entity='client.admin' 2024-09-21T03:08:27.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1108434081' entity='client.admin' 2024-09-21T03:08:27.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:27.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:27.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:27.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:27.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:27 smithi083 ceph-mon[31219]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/948454491' entity='client.admin' 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1108434081' entity='client.admin' 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:27.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:27 smithi142 ceph-mon[34512]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 -- 172.21.15.83:0/4158958825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 msgr2=0x7f7d2010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/4158958825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2010d8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009900 tx=0x7f7d1002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 -- 172.21.15.83:0/4158958825 shutdown_connections 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/4158958825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2010de90 0x7f7d20110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/4158958825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2010d8d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/4158958825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d20104a30 0x7f7d20104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 -- 172.21.15.83:0/4158958825 >> 172.21.15.83:0/4158958825 conn(0x7f7d20079770 msgr2=0x7f7d20079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 -- 172.21.15.83:0/4158958825 shutdown_connections 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.704+0000 7f7d27f5f640 1 -- 172.21.15.83:0/4158958825 wait complete. 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 Processor -- start 2024-09-21T03:08:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 -- start start 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 0x7f7d2019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 0x7f7d2019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201139d0 con 0x7f7d20105380 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20113850 con 0x7f7d20104a30 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d27f5f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20113b50 con 0x7f7d2010de90 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d254fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d254fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49126/0 (socket says 172.21.15.83:49126) 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.705+0000 7f7d254fa640 1 -- 172.21.15.83:0/3101124106 learned_addr learned my addr 172.21.15.83:0/3101124106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d25cfb640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 0x7f7d2019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d264fc640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 0x7f7d2019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 -- 172.21.15.83:0/3101124106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 msgr2=0x7f7d2019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 0x7f7d2019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 -- 172.21.15.83:0/3101124106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 msgr2=0x7f7d2019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 0x7f7d2019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 -- 172.21.15.83:0/3101124106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d201a0330 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d25cfb640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 0x7f7d2019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d264fc640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 0x7f7d2019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d254fa640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005940 tx=0x7f7d100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1003f030 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d10031800 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d201a05c0 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10035070 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.706+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201a0aa0 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.707+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2006c260 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.709+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7d10042030 con 0x7f7d20105380 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.709+0000 7f7d16ffd640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 0x7f7d0c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.709+0000 7f7d25cfb640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 0x7f7d0c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.710+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7d10106330 con 0x7f7d20105380 2024-09-21T03:08:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.710+0000 7f7d25cfb640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 0x7f7d0c0b8a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fd0 tx=0x7f7d1c0093e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.712+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7d100ce7c0 con 0x7f7d20105380 2024-09-21T03:08:27.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/948454491' entity='client.admin' 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1108434081' entity='client.admin' 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:27 smithi193 ceph-mon[35224]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:27.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.897+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f7d20108fc0 con 0x7f7d20105380 2024-09-21T03:08:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.903+0000 7f7d16ffd640 1 -- 172.21.15.83:0/3101124106 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v34)=0 v34) ==== 111+0+0 (secure 0 0 0) 0x7f7d100d2ba0 con 0x7f7d20105380 2024-09-21T03:08:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 msgr2=0x7f7d0c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 0x7f7d0c0b8a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fd0 tx=0x7f7d1c0093e0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 msgr2=0x7f7d2019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005940 tx=0x7f7d100042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 shutdown_connections 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7d0c0b6500 0x7f7d0c0b8a10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d2010de90 0x7f7d2019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d20105380 0x7f7d2019aa20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 --2- 172.21.15.83:0/3101124106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d20104a30 0x7f7d2019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 >> 172.21.15.83:0/3101124106 conn(0x7f7d20079770 msgr2=0x7f7d200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 shutdown_connections 2024-09-21T03:08:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:27.906+0000 7f7d27f5f640 1 -- 172.21.15.83:0/3101124106 wait complete. 2024-09-21T03:08:29.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:28 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3101124106' entity='client.admin' 2024-09-21T03:08:29.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:28 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:29.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:28 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:29.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:28 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:29.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:28 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:29.063 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:29.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:28 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3101124106' entity='client.admin' 2024-09-21T03:08:29.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:28 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:29.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:28 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:29.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:28 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:29.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:28 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:28 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3101124106' entity='client.admin' 2024-09-21T03:08:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:28 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:28 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:28 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:28 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 -- 172.21.15.83:0/2660456932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80103b60 msgr2=0x7f8a80103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 --2- 172.21.15.83:0/2660456932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80103b60 0x7f8a80103f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009900 tx=0x7f8a7002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 -- 172.21.15.83:0/2660456932 shutdown_connections 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 --2- 172.21.15.83:0/2660456932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80109df0 0x7f8a80110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 --2- 172.21.15.83:0/2660456932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 --2- 172.21.15.83:0/2660456932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80103b60 0x7f8a80103f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 -- 172.21.15.83:0/2660456932 >> 172.21.15.83:0/2660456932 conn(0x7f8a8007b850 msgr2=0x7f8a8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 -- 172.21.15.83:0/2660456932 shutdown_connections 2024-09-21T03:08:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.072+0000 7f8a87732640 1 -- 172.21.15.83:0/2660456932 wait complete. 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 Processor -- start 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 -- start start 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80103b60 0x7f8a8019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a8019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80113990 con 0x7f8a80109df0 2024-09-21T03:08:30.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80113810 con 0x7f8a80103b60 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a87732640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80113b10 con 0x7f8a801094a0 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a85ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a85ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35810/0 (socket says 172.21.15.83:35810) 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.073+0000 7f8a85ccf640 1 -- 172.21.15.83:0/3819641047 learned_addr learned my addr 172.21.15.83:0/3819641047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a854ce640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80103b60 0x7f8a8019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a84ccd640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a8019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 -- 172.21.15.83:0/3819641047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 msgr2=0x7f8a8019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a8019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 -- 172.21.15.83:0/3819641047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80103b60 msgr2=0x7f8a8019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80103b60 0x7f8a8019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 -- 172.21.15.83:0/3819641047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801a0330 con 0x7f8a80109df0 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a84ccd640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a8019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a85ccf640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cad0 tx=0x7f8a7800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800ae20 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801a0620 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a78005020 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800f6b0 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.074+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a6d80 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.075+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a80103f70 con 0x7f8a80109df0 2024-09-21T03:08:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.077+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f8a78004a40 con 0x7f8a80109df0 2024-09-21T03:08:30.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.078+0000 7f8a767fc640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 0x7f8a6c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.078+0000 7f8a854ce640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 0x7f8a6c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:30.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.078+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8a780e5830 con 0x7f8a80109df0 2024-09-21T03:08:30.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.078+0000 7f8a854ce640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 0x7f8a6c0b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000c00 tx=0x7f8a70009680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:30.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.082+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a78014030 con 0x7f8a80109df0 2024-09-21T03:08:30.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:29 smithi083 ceph-mon[31219]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:30.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:29 smithi142 ceph-mon[34512]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:30.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.266+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f8a8002e640 con 0x7f8a80109df0 2024-09-21T03:08:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.272+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v35)=0 v35) ==== 123+0+0 (secure 0 0 0) 0x7f8a780adcc0 con 0x7f8a80109df0 2024-09-21T03:08:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.273+0000 7f8a767fc640 1 -- 172.21.15.83:0/3819641047 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a780eb080 con 0x7f8a80109df0 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.274+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 msgr2=0x7f8a6c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.274+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 0x7f8a6c0b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000c00 tx=0x7f8a70009680 comp rx=0 tx=0).stop 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.274+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 msgr2=0x7f8a8019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.274+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cad0 tx=0x7f8a7800cf90 comp rx=0 tx=0).stop 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 shutdown_connections 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a6c0b64d0 0x7f8a6c0b89e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80109df0 0x7f8a8019fc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a801094a0 0x7f8a8019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 --2- 172.21.15.83:0/3819641047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a80103b60 0x7f8a8019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 >> 172.21.15.83:0/3819641047 conn(0x7f8a8007b850 msgr2=0x7f8a80107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 shutdown_connections 2024-09-21T03:08:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.275+0000 7f8a87732640 1 -- 172.21.15.83:0/3819641047 wait complete. 2024-09-21T03:08:30.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:29 smithi193 ceph-mon[35224]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.994+0000 7fc3a453e640 1 -- 172.21.15.83:0/964643595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 msgr2=0x7fc39c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.994+0000 7fc3a453e640 1 --2- 172.21.15.83:0/964643595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c1087d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc398009900 tx=0x7fc39802a410 comp rx=0 tx=0).stop 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 -- 172.21.15.83:0/964643595 shutdown_connections 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 --2- 172.21.15.83:0/964643595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c108d10 0x7fc39c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 --2- 172.21.15.83:0/964643595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c1087d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 --2- 172.21.15.83:0/964643595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c102a80 0x7fc39c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 -- 172.21.15.83:0/964643595 >> 172.21.15.83:0/964643595 conn(0x7fc39c079740 msgr2=0x7fc39c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 -- 172.21.15.83:0/964643595 shutdown_connections 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 -- 172.21.15.83:0/964643595 wait complete. 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 Processor -- start 2024-09-21T03:08:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.995+0000 7fc3a453e640 1 -- start start 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 0x7fc39c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 0x7fc39c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c112980 con 0x7fc39c1083c0 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c112800 con 0x7fc39c108d10 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c112b00 con 0x7fc39c102a80 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:30.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35826/0 (socket says 172.21.15.83:35826) 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 -- 172.21.15.83:0/1771406934 learned_addr learned my addr 172.21.15.83:0/1771406934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a22da640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 0x7fc39c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a2adb640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 0x7fc39c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 -- 172.21.15.83:0/1771406934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 msgr2=0x7fc39c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 0x7fc39c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 -- 172.21.15.83:0/1771406934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 msgr2=0x7fc39c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 0x7fc39c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 -- 172.21.15.83:0/1771406934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c1a6520 con 0x7fc39c1083c0 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a22da640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 0x7fc39c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a2adb640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 0x7fc39c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a1ad9640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc398006010 tx=0x7fc398004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39803f030 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398003aa0 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1a67b0 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.996+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398035070 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.997+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1a6ce0 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:30.997+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39c103d00 con 0x7fc39c1083c0 2024-09-21T03:08:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.000+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc398042030 con 0x7fc39c1083c0 2024-09-21T03:08:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.000+0000 7fc3937fe640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 0x7fc3880b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.001+0000 7fc3a22da640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 0x7fc3880b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.001+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc398105fb0 con 0x7fc39c1083c0 2024-09-21T03:08:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.001+0000 7fc3a22da640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 0x7fc3880b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc39c06c6c0 tx=0x7fc38c005db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:31.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.004+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3980ce440 con 0x7fc39c1083c0 2024-09-21T03:08:31.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.188+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fc39c102e90 con 0x7fc39c1083c0 2024-09-21T03:08:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.193+0000 7fc3937fe640 1 -- 172.21.15.83:0/1771406934 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v36)=0 v36) ==== 120+0+0 (secure 0 0 0) 0x7fc39c102e90 con 0x7fc39c1083c0 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 msgr2=0x7fc3880b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 0x7fc3880b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc39c06c6c0 tx=0x7fc38c005db0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 msgr2=0x7fc39c19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc398006010 tx=0x7fc398004060 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 shutdown_connections 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3880b6550 0x7fc3880b8a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc39c108d10 0x7fc39c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc39c1083c0 0x7fc39c19b830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 --2- 172.21.15.83:0/1771406934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39c102a80 0x7fc39c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 >> 172.21.15.83:0/1771406934 conn(0x7fc39c079740 msgr2=0x7fc39c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 shutdown_connections 2024-09-21T03:08:31.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:31.195+0000 7fc3a453e640 1 -- 172.21.15.83:0/1771406934 wait complete. 2024-09-21T03:08:31.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3819641047' entity='client.admin' 2024-09-21T03:08:31.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:31 smithi083 ceph-mon[31219]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:31.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1771406934' entity='client.admin' 2024-09-21T03:08:31.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:31 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3819641047' entity='client.admin' 2024-09-21T03:08:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:31 smithi193 ceph-mon[35224]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:31 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1771406934' entity='client.admin' 2024-09-21T03:08:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:31 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3819641047' entity='client.admin' 2024-09-21T03:08:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:31 smithi142 ceph-mon[34512]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:31 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1771406934' entity='client.admin' 2024-09-21T03:08:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 -- 172.21.15.83:0/2507169666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79901048e0 msgr2=0x7f7990104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 --2- 172.21.15.83:0/2507169666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79901048e0 0x7f7990104cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7980009900 tx=0x7f798002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 -- 172.21.15.83:0/2507169666 shutdown_connections 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 --2- 172.21.15.83:0/2507169666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 0x7f7990110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 --2- 172.21.15.83:0/2507169666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7990105230 0x7f799010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 --2- 172.21.15.83:0/2507169666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79901048e0 0x7f7990104cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 -- 172.21.15.83:0/2507169666 >> 172.21.15.83:0/2507169666 conn(0x7f7990079720 msgr2=0x7f7990079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 -- 172.21.15.83:0/2507169666 shutdown_connections 2024-09-21T03:08:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.254+0000 7f7996a53640 1 -- 172.21.15.83:0/2507169666 wait complete. 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 Processor -- start 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 -- start start 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79901048e0 0x7f799019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 0x7f799019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7990113a80 con 0x7f7990105230 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7990113900 con 0x7f799010de90 2024-09-21T03:08:32.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7996a53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7990113c00 con 0x7f79901048e0 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f798f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f798f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35840/0 (socket says 172.21.15.83:35840) 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f798f7fe640 1 -- 172.21.15.83:0/2038048627 learned_addr learned my addr 172.21.15.83:0/2038048627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f798ffff640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79901048e0 0x7f799019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.255+0000 7f7994ff0640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 0x7f799019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 -- 172.21.15.83:0/2038048627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79901048e0 msgr2=0x7f799019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79901048e0 0x7f799019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 -- 172.21.15.83:0/2038048627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 msgr2=0x7f799019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 0x7f799019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 -- 172.21.15.83:0/2038048627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79901a0260 con 0x7f7990105230 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798f7fe640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f798400c970 tx=0x7f798400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984014070 con 0x7f7990105230 2024-09-21T03:08:32.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7984004440 con 0x7f7990105230 2024-09-21T03:08:32.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901a0550 con 0x7f7990105230 2024-09-21T03:08:32.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984003e50 con 0x7f7990105230 2024-09-21T03:08:32.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.256+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a6d70 con 0x7f7990105230 2024-09-21T03:08:32.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.257+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7990104cf0 con 0x7f7990105230 2024-09-21T03:08:32.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.259+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7984017070 con 0x7f7990105230 2024-09-21T03:08:32.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.259+0000 7f798d7fa640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 0x7f79780b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:32.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.260+0000 7f798ffff640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 0x7f79780b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:32.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.260+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f79840e5a10 con 0x7f7990105230 2024-09-21T03:08:32.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.260+0000 7f798ffff640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 0x7f79780b8bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7980031440 tx=0x7f7980005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:32.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.263+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f798401e020 con 0x7f7990105230 2024-09-21T03:08:32.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.445+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f799019b4e0 con 0x7f7990105230 2024-09-21T03:08:32.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.450+0000 7f798d7fa640 1 -- 172.21.15.83:0/2038048627 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v37)=0 v37) ==== 129+0+0 (secure 0 0 0) 0x7f799019b4e0 con 0x7f7990105230 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.452+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 msgr2=0x7f79780b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.452+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 0x7f79780b8bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7980031440 tx=0x7f7980005d50 comp rx=0 tx=0).stop 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.452+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 msgr2=0x7f799019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.452+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f798400c970 tx=0x7f798400ce30 comp rx=0 tx=0).stop 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 shutdown_connections 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79780b66a0 0x7f79780b8bb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799010de90 0x7f799019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7990105230 0x7f799019a950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 --2- 172.21.15.83:0/2038048627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79901048e0 0x7f799019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 >> 172.21.15.83:0/2038048627 conn(0x7f7990079720 msgr2=0x7f7990105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 shutdown_connections 2024-09-21T03:08:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:32.453+0000 7f7996a53640 1 -- 172.21.15.83:0/2038048627 wait complete. 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.224 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.225 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.226 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.597 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2038048627' entity='client.admin' 2024-09-21T03:08:33.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:33.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:33.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:33.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:33.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:33 smithi083 ceph-mon[31219]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:33.625 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.625 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.625 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5108497e-77c6-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi083') 2024-09-21T03:08:33.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2038048627' entity='client.admin' 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:33 smithi193 ceph-mon[35224]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:33.866 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2038048627' entity='client.admin' 2024-09-21T03:08:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:33 smithi142 ceph-mon[34512]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:34.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- 172.21.15.83:0/2081149863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 msgr2=0x7fba30103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:34.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 --2- 172.21.15.83:0/2081149863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba30103f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009900 tx=0x7fba2c02a410 comp rx=0 tx=0).stop 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- 172.21.15.83:0/2081149863 shutdown_connections 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 --2- 172.21.15.83:0/2081149863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 0x7fba30110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 --2- 172.21.15.83:0/2081149863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 0x7fba301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 --2- 172.21.15.83:0/2081149863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba30103f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- 172.21.15.83:0/2081149863 >> 172.21.15.83:0/2081149863 conn(0x7fba3007b850 msgr2=0x7fba3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- 172.21.15.83:0/2081149863 shutdown_connections 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- 172.21.15.83:0/2081149863 wait complete. 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 Processor -- start 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.288+0000 7fba37fda640 1 -- start start 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 0x7fba3019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 0x7fba3019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba30113b70 con 0x7fba30103b60 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba301139f0 con 0x7fba30109df0 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba37fda640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba30113cf0 con 0x7fba301094a0 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35868/0 (socket says 172.21.15.83:35868) 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 -- 172.21.15.83:0/552018441 learned_addr learned my addr 172.21.15.83:0/552018441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35575640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 0x7fba3019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba36577640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 0x7fba3019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 -- 172.21.15.83:0/552018441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 msgr2=0x7fba3019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 0x7fba3019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 -- 172.21.15.83:0/552018441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 msgr2=0x7fba3019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 0x7fba3019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 -- 172.21.15.83:0/552018441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba301a0530 con 0x7fba30103b60 2024-09-21T03:08:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.289+0000 7fba35d76640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006010 tx=0x7fba2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.290+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c03f030 con 0x7fba30103b60 2024-09-21T03:08:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.290+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c003800 con 0x7fba30103b60 2024-09-21T03:08:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.290+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba301a07c0 con 0x7fba30103b60 2024-09-21T03:08:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.290+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c035070 con 0x7fba30103b60 2024-09-21T03:08:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.290+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301a6f80 con 0x7fba30103b60 2024-09-21T03:08:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.291+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba30103ff0 con 0x7fba30103b60 2024-09-21T03:08:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.292+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fba2c0039b0 con 0x7fba30103b60 2024-09-21T03:08:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.293+0000 7fba26ffd640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 0x7fba1c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.293+0000 7fba35575640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 0x7fba1c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.293+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fba2c1063d0 con 0x7fba30103b60 2024-09-21T03:08:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.294+0000 7fba35575640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 0x7fba1c0b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba30106450 tx=0x7fba20008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:34.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.297+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba2c0ce860 con 0x7fba30103b60 2024-09-21T03:08:34.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.475+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fba3019b6f0 con 0x7fba30103b60 2024-09-21T03:08:34.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.476+0000 7fba26ffd640 1 -- 172.21.15.83:0/552018441 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+21924 (secure 0 0 0) 0x7fba2c0d2c40 con 0x7fba30103b60 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:global dev auth_debug true 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:global basic log_to_file true 2024-09-21T03:08:34.478 INFO:teuthology.orchestra.run.smithi083.stdout:global basic log_to_journald false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global basic log_to_stderr false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global dev ms_die_on_bug true 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global dev ms_die_on_old_message true 2024-09-21T03:08:34.479 INFO:teuthology.orchestra.run.smithi083.stdout:global dev ms_die_on_skipped_message false 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced debug_mon 20/20 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced debug_ms 1/1 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:08:34.480 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced debug_client 20/20 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:08:34.481 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev bdev_debug_aio true 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T03:08:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced debug_bluefs 20/20 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced debug_bluestore 20/20 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced debug_ms 1/1 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced debug_osd 20/20 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_op_order true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:08:34.483 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:08:34.484 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced debug_mds 20/20 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced debug_ms 2/2 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds dev mds_debug_frag true 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds dev mds_verify_scatter true 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:08:34.485 INFO:teuthology.orchestra.run.smithi083.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client advanced client_mount_timeout 600 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client advanced debug_client 20/20 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client advanced debug_ms 1/1 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:08:34.486 INFO:teuthology.orchestra.run.smithi083.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 msgr2=0x7fba1c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 0x7fba1c0b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba30106450 tx=0x7fba20008040 comp rx=0 tx=0).stop 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 msgr2=0x7fba3019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006010 tx=0x7fba2c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 shutdown_connections 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba1c0b64d0 0x7fba1c0b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba30109df0 0x7fba3019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba301094a0 0x7fba3019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 --2- 172.21.15.83:0/552018441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba30103b60 0x7fba3019a5c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:34.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 >> 172.21.15.83:0/552018441 conn(0x7fba3007b850 msgr2=0x7fba301075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:34.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 shutdown_connections 2024-09-21T03:08:34.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:34.478+0000 7fba37fda640 1 -- 172.21.15.83:0/552018441 wait complete. 2024-09-21T03:08:34.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:34 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/552018441' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:08:34.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:34 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/552018441' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:08:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:34 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/552018441' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:08:35.133 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:08:35.134 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:08:35.134 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:08:35.150 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:08:35.150 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-09-21T03:08:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:08:35.205 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:35.205 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-21 03:07:15.158601371 +0000 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-21 03:02:22.084567730 +0000 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-21 03:02:22.084567730 +0000 2024-09-21T03:08:35.206 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-21 03:02:22.084567730 +0000 2024-09-21T03:08:35.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:08:35.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-21T03:08:35.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-21T03:08:35.269 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00020652 s, 2.5 MB/s 2024-09-21T03:08:35.271 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:08:35.327 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-09-21T03:08:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:08:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:35.382 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T03:08:35.382 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:35.383 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:35.383 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-21 03:07:15.188600656 +0000 2024-09-21T03:08:35.383 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-21 03:02:22.417559698 +0000 2024-09-21T03:08:35.383 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-21 03:02:22.417559698 +0000 2024-09-21T03:08:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-21 03:02:22.417559698 +0000 2024-09-21T03:08:35.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:08:35.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-21T03:08:35.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-21T03:08:35.445 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000197085 s, 2.6 MB/s 2024-09-21T03:08:35.446 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:08:35.502 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:35.558 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-21 03:07:15.219599917 +0000 2024-09-21T03:08:35.559 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-21 03:02:22.744551810 +0000 2024-09-21T03:08:35.559 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-21 03:02:22.744551810 +0000 2024-09-21T03:08:35.559 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-21 03:02:22.744551810 +0000 2024-09-21T03:08:35.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:08:35.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-21T03:08:35.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-21T03:08:35.621 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000175833 s, 2.9 MB/s 2024-09-21T03:08:35.622 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:08:35.678 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-09-21T03:08:35.733 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:08:35.733 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:35.733 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:08:35.733 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:35.733 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:35.734 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-21 03:07:15.249599202 +0000 2024-09-21T03:08:35.734 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-21 03:02:23.089543489 +0000 2024-09-21T03:08:35.734 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-21 03:02:23.089543489 +0000 2024-09-21T03:08:35.734 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-21 03:02:23.089543489 +0000 2024-09-21T03:08:35.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:08:35.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:35 smithi083 ceph-mon[31219]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-21T03:08:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-21T03:08:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000192415 s, 2.7 MB/s 2024-09-21T03:08:35.797 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:08:35.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:35 smithi193 ceph-mon[35224]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:35.855 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-21T03:08:35.855 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:08:35.874 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:08:35.875 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-09-21T03:08:35.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:35 smithi142 ceph-mon[34512]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-21 03:08:07.100761576 +0000 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-21 03:02:34.669235035 +0000 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-21 03:02:34.669235035 +0000 2024-09-21T03:08:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-21 03:02:34.669235035 +0000 2024-09-21T03:08:35.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:08:36.004 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-21T03:08:36.004 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-21T03:08:36.004 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00028923 s, 1.8 MB/s 2024-09-21T03:08:36.005 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:08:36.064 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-09-21T03:08:36.121 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:08:36.121 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:36.121 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:08:36.121 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:36.122 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:36.122 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-21 03:08:07.101761558 +0000 2024-09-21T03:08:36.122 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-21 03:02:35.031228365 +0000 2024-09-21T03:08:36.122 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-21 03:02:35.031228365 +0000 2024-09-21T03:08:36.122 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-21 03:02:35.031228365 +0000 2024-09-21T03:08:36.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:08:36.187 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-21T03:08:36.187 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-21T03:08:36.187 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000207197 s, 2.5 MB/s 2024-09-21T03:08:36.188 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:08:36.245 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-09-21T03:08:36.301 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:08:36.301 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-21 03:08:07.101761558 +0000 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-21 03:02:35.368222156 +0000 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-21 03:02:35.368222156 +0000 2024-09-21T03:08:36.302 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-21 03:02:35.368222156 +0000 2024-09-21T03:08:36.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:08:36.366 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-21T03:08:36.366 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-21T03:08:36.366 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000200932 s, 2.5 MB/s 2024-09-21T03:08:36.367 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:08:36.424 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-09-21T03:08:36.480 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:08:36.480 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:36.480 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-21T03:08:36.480 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:36.480 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:36.481 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-21 03:08:07.101761558 +0000 2024-09-21T03:08:36.481 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-21 03:02:35.688216260 +0000 2024-09-21T03:08:36.481 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-21 03:02:35.688216260 +0000 2024-09-21T03:08:36.481 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-21 03:02:35.688216260 +0000 2024-09-21T03:08:36.481 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:08:36.544 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-21T03:08:36.544 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-21T03:08:36.544 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000201675 s, 2.5 MB/s 2024-09-21T03:08:36.545 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:08:36.602 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T03:08:36.602 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:08:36.622 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:08:36.622 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 03:08:04.892039207 +0000 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 03:02:43.715526336 +0000 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 03:02:43.715526336 +0000 2024-09-21T03:08:36.681 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 03:02:43.715526336 +0000 2024-09-21T03:08:36.682 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:08:36.752 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T03:08:36.752 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T03:08:36.752 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000318311 s, 1.6 MB/s 2024-09-21T03:08:36.753 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:08:36.811 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 03:08:04.892039207 +0000 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 03:02:44.041521178 +0000 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 03:02:44.041521178 +0000 2024-09-21T03:08:36.869 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 03:02:44.041521178 +0000 2024-09-21T03:08:36.870 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:08:36.934 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T03:08:36.934 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T03:08:36.934 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000205008 s, 2.5 MB/s 2024-09-21T03:08:36.936 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:08:36.993 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-21T03:08:37.050 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 03:08:04.892039207 +0000 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 03:02:44.358516162 +0000 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 03:02:44.358516162 +0000 2024-09-21T03:08:37.051 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 03:02:44.358516162 +0000 2024-09-21T03:08:37.051 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:08:37.116 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T03:08:37.116 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T03:08:37.116 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000204988 s, 2.5 MB/s 2024-09-21T03:08:37.117 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:08:37.174 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-21T03:08:37.230 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 03:08:04.893039192 +0000 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 03:02:44.693510861 +0000 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 03:02:44.693510861 +0000 2024-09-21T03:08:37.231 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 03:02:44.693510861 +0000 2024-09-21T03:08:37.232 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:08:37.295 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T03:08:37.295 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T03:08:37.295 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000194367 s, 2.6 MB/s 2024-09-21T03:08:37.296 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:08:37.352 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-09-21T03:08:37.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:08:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:38.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:38 smithi083 ceph-mon[31219]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:38 smithi142 ceph-mon[34512]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:38 smithi193 ceph-mon[35224]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:39.847 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:08:39.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-09-21T03:08:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:40.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:40 smithi142 ceph-mon[34512]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:40.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:40 smithi083 ceph-mon[31219]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:40 smithi193 ceph-mon[35224]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 -- 172.21.15.83:0/1294877321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f01094c0 msgr2=0x7f14f01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 --2- 172.21.15.83:0/1294877321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f01094c0 0x7f14f01098d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 -- 172.21.15.83:0/1294877321 shutdown_connections 2024-09-21T03:08:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 --2- 172.21.15.83:0/1294877321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0109e10 0x7f14f0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 --2- 172.21.15.83:0/1294877321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f01094c0 0x7f14f01098d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 --2- 172.21.15.83:0/1294877321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f0103b80 0x7f14f0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 -- 172.21.15.83:0/1294877321 >> 172.21.15.83:0/1294877321 conn(0x7f14f00ff610 msgr2=0x7f14f0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.605+0000 7f14f6061640 1 -- 172.21.15.83:0/1294877321 shutdown_connections 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.606+0000 7f14f6061640 1 -- 172.21.15.83:0/1294877321 wait complete. 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.606+0000 7f14f6061640 1 Processor -- start 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.606+0000 7f14f6061640 1 -- start start 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0103b80 0x7f14f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f01094c0 0x7f14f0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0113b40 con 0x7f14f0109e10 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01139c0 con 0x7f14f0103b80 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14f6061640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0113cc0 con 0x7f14f01094c0 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55992/0 (socket says 172.21.15.83:55992) 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 -- 172.21.15.83:0/1725749079 learned_addr learned my addr 172.21.15.83:0/1725749079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14eeffd640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f01094c0 0x7f14f0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14ef7fe640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0103b80 0x7f14f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 -- 172.21.15.83:0/1725749079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f01094c0 msgr2=0x7f14f0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f01094c0 0x7f14f0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 -- 172.21.15.83:0/1725749079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0103b80 msgr2=0x7f14f0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0103b80 0x7f14f0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 -- 172.21.15.83:0/1725749079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f01a8810 con 0x7f14f0109e10 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14effff640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14e800cad0 tx=0x7f14e800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.607+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800ad60 con 0x7f14f0109e10 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.608+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e8005020 con 0x7f14f0109e10 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.608+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f01a8b00 con 0x7f14f0109e10 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.608+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800f6b0 con 0x7f14f0109e10 2024-09-21T03:08:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.608+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01a9010 con 0x7f14f0109e10 2024-09-21T03:08:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.609+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f0103f90 con 0x7f14f0109e10 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.610+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f14e80051d0 con 0x7f14f0109e10 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.611+0000 7f14ecff9640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 0x7f14d80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.611+0000 7f14ef7fe640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 0x7f14d80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.611+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f14e80e5a00 con 0x7f14f0109e10 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.611+0000 7f14ef7fe640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 0x7f14d80b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14f0077160 tx=0x7f14e0009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.614+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14e8014030 con 0x7f14f0109e10 2024-09-21T03:08:40.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:08:40.802+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f14f0063800 con 0x7f14d80b6550 2024-09-21T03:08:41.419 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:41 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:08:41.419 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:41 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:08:41.419 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:41 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:41 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:08:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:41 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:08:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:41 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:41 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:08:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:41 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:08:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:41 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:42.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:42 smithi083 ceph-mon[31219]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:08:42.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:42 smithi083 ceph-mon[31219]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:42 smithi142 ceph-mon[34512]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:08:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:42 smithi142 ceph-mon[34512]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:42.466 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:42 smithi193 ceph-mon[35224]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:08:42.466 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:42 smithi193 ceph-mon[35224]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:43.423 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"} : dispatch 2024-09-21T03:08:43.423 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"}]': finished 2024-09-21T03:08:43.424 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:43 smithi083 ceph-mon[31219]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:08:43.424 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:43 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:08:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"} : dispatch 2024-09-21T03:08:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"}]': finished 2024-09-21T03:08:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:43 smithi142 ceph-mon[34512]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:08:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:43 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:08:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"} : dispatch 2024-09-21T03:08:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3378074393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f96fcc94-0c90-4aaa-83e9-93e0c3cdc835"}]': finished 2024-09-21T03:08:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:43 smithi193 ceph-mon[35224]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:08:43.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:43 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:08:44.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:44 smithi142 ceph-mon[34512]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:44.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:44 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3976609315' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:08:44.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:44 smithi193 ceph-mon[35224]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:44 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3976609315' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:08:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:44 smithi083 ceph-mon[31219]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:44 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3976609315' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:08:46.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:46 smithi142 ceph-mon[34512]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:46.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:46 smithi193 ceph-mon[35224]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:46 smithi083 ceph-mon[31219]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:48.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:48 smithi083 ceph-mon[31219]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:48 smithi142 ceph-mon[34512]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:48.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:48 smithi193 ceph-mon[35224]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:50 smithi142 ceph-mon[34512]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:50.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:50 smithi083 ceph-mon[31219]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:50.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:50 smithi193 ceph-mon[35224]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:08:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:51.428 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:08:51.428 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:51.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:08:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:52.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:52 smithi083 ceph-mon[31219]: Deploying daemon osd.0 on smithi083 2024-09-21T03:08:52.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:52 smithi083 ceph-mon[31219]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:52 smithi193 ceph-mon[35224]: Deploying daemon osd.0 on smithi083 2024-09-21T03:08:52.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:52 smithi193 ceph-mon[35224]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:52 smithi142 ceph-mon[34512]: Deploying daemon osd.0 on smithi083 2024-09-21T03:08:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:52 smithi142 ceph-mon[34512]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:54 smithi083 ceph-mon[31219]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:54.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:54 smithi193 ceph-mon[35224]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:54 smithi142 ceph-mon[34512]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:56.341 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:56 smithi083 ceph-mon[31219]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:56.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:56 smithi193 ceph-mon[35224]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:56 smithi142 ceph-mon[34512]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:58.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:58 smithi083 ceph-mon[31219]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:58.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:58.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:58.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:08:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:58 smithi193 ceph-mon[35224]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:08:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:58 smithi142 ceph-mon[34512]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:08:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:08:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:08:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:00 smithi083 ceph-mon[31219]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:00.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:00 smithi193 ceph-mon[35224]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:00 smithi142 ceph-mon[34512]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:01.454 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:01.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.749+0000 7f14ecff9640 1 -- 172.21.15.83:0/1725749079 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14f0063800 con 0x7f14d80b6550 2024-09-21T03:09:01.751 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-09-21T03:09:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.751+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 msgr2=0x7f14d80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.751+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 0x7f14d80b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14f0077160 tx=0x7f14e0009110 comp rx=0 tx=0).stop 2024-09-21T03:09:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.751+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 msgr2=0x7f14f0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.751+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14e800cad0 tx=0x7f14e800ced0 comp rx=0 tx=0).stop 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 shutdown_connections 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f14d80b6550 0x7f14d80b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f0109e10 0x7f14f0076190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14f01094c0 0x7f14f0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 --2- 172.21.15.83:0/1725749079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14f0103b80 0x7f14f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 >> 172.21.15.83:0/1725749079 conn(0x7f14f00ff610 msgr2=0x7f14f0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 shutdown_connections 2024-09-21T03:09:01.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:01.752+0000 7f14f6061640 1 -- 172.21.15.83:0/1725749079 wait complete. 2024-09-21T03:09:02.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:02 smithi083 ceph-mon[31219]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:02.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:02.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.439 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:09:02.441 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2024-09-21T03:09:02.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:09:02.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:02 smithi142 ceph-mon[34512]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:02.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:02.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:02 smithi193 ceph-mon[35224]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:02.679 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:09:03.832 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:03 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:09:03.595+0000 7f29bbb96640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:09:04.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:04 smithi142 ceph-mon[34512]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:04.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:04 smithi142 ceph-mon[34512]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:09:04.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:04 smithi083 ceph-mon[31219]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:04.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:04 smithi083 ceph-mon[31219]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:09:04.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:04 smithi193 ceph-mon[35224]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:04.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:04 smithi193 ceph-mon[35224]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:09:05.034 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:09:05.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-09-21T03:09:05.351 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:09:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:05 smithi142 ceph-mon[34512]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:09:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:05 smithi142 ceph-mon[34512]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:09:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:05 smithi142 ceph-mon[34512]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:09:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:05 smithi142 ceph-mon[34512]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:09:05.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:05 smithi193 ceph-mon[35224]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:09:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:05 smithi193 ceph-mon[35224]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:09:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:05 smithi193 ceph-mon[35224]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:09:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:05 smithi193 ceph-mon[35224]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:09:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-mon[31219]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:09:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-mon[31219]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:09:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-mon[31219]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:09:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-mon[31219]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 -- 172.21.15.142:0/3421396938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408103bc0 msgr2=0x7f4408103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 --2- 172.21.15.142:0/3421396938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408103bc0 0x7f4408103fd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009900 tx=0x7f43f802a410 comp rx=0 tx=0).stop 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 -- 172.21.15.142:0/3421396938 shutdown_connections 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 --2- 172.21.15.142:0/3421396938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4408109e00 0x7f4408110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 --2- 172.21.15.142:0/3421396938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44081094b0 0x7f44081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 --2- 172.21.15.142:0/3421396938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408103bc0 0x7f4408103fd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:05.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 -- 172.21.15.142:0/3421396938 >> 172.21.15.142:0/3421396938 conn(0x7f440807b910 msgr2=0x7f440807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 -- 172.21.15.142:0/3421396938 shutdown_connections 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.785+0000 7f440edc0640 1 -- 172.21.15.142:0/3421396938 wait complete. 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 Processor -- start 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 -- start start 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4408103bc0 0x7f440819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44081094b0 0x7f440819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4408113b20 con 0x7f4408103bc0 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44081139a0 con 0x7f4408109e00 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440edc0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4408113ca0 con 0x7f44081094b0 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440d35d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440d35d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40108/0 (socket says 172.21.15.142:40108) 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440d35d640 1 -- 172.21.15.142:0/118214107 learned_addr learned my addr 172.21.15.142:0/118214107 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-21T03:09:05.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f440cb5c640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4408103bc0 0x7f440819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.786+0000 7f4407fff640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44081094b0 0x7f440819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 -- 172.21.15.142:0/118214107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44081094b0 msgr2=0x7f440819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44081094b0 0x7f440819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 -- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4408103bc0 msgr2=0x7f440819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4408103bc0 0x7f440819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 -- 172.21.15.142:0/118214107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44081a0460 con 0x7f4408109e00 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440d35d640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f440000b970 tx=0x7f440000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400019070 con 0x7f4408109e00 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4400014440 con 0x7f4408109e00 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44081a0750 con 0x7f4408109e00 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000075d0 con 0x7f4408109e00 2024-09-21T03:09:05.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.787+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44081a6eb0 con 0x7f4408109e00 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.788+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4408103fd0 con 0x7f4408109e00 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.789+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f440001c070 con 0x7f4408109e00 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.790+0000 7f4405ffb640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 0x7f43f40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.790+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7f44000e4cf0 con 0x7f4408109e00 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.790+0000 7f440cb5c640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 0x7f43f40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.791+0000 7f440cb5c640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 0x7f43f40b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43f8031440 tx=0x7f43f8009560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:05.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.794+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44000b1410 con 0x7f4408109e00 2024-09-21T03:09:05.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:05.983+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4408063800 con 0x7f43f40b6550 2024-09-21T03:09:06.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:09:05.656+0000 7f29b7b18640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:09:06.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:05 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:09:05.659+0000 7f29af90e640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi083 2024-09-21T03:09:06.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.353 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi083 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi083 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 17088M 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745]' entity='osd.0' 2024-09-21T03:09:06.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:06.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:06.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:06.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:06.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:07.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:09:07.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:09:07.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] boot 2024-09-21T03:09:07.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:09:07.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:07.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:07 smithi142 ceph-mon[34512]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] boot 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:07 smithi193 ceph-mon[35224]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: osd.0 [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] boot 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:09:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:07 smithi083 ceph-mon[31219]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/869717459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"}]': finished 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:09:08.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/869717459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"}]': finished 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:09:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:09.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:09:09.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/869717459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:09.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"} : dispatch 2024-09-21T03:09:09.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949b227b-4c32-4ca5-a1db-df21fe923196"}]': finished 2024-09-21T03:09:09.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:09:09.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:09 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/1266476896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:09.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:09 smithi142 ceph-mon[34512]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:10.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:09 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/1266476896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:10.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:09 smithi193 ceph-mon[35224]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:10.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:09 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/1266476896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:10.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:09 smithi083 ceph-mon[31219]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:12 smithi142 ceph-mon[34512]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:12.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:12 smithi193 ceph-mon[35224]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:12 smithi083 ceph-mon[31219]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:14.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:14 smithi142 ceph-mon[34512]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:14.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:14 smithi193 ceph-mon[35224]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:14 smithi083 ceph-mon[31219]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:16.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:16 smithi142 ceph-mon[34512]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:16 smithi193 ceph-mon[35224]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:16 smithi083 ceph-mon[31219]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:09:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:17.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:09:17.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:09:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:18 smithi142 ceph-mon[34512]: Deploying daemon osd.1 on smithi142 2024-09-21T03:09:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:18 smithi142 ceph-mon[34512]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:18.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:18 smithi083 ceph-mon[31219]: Deploying daemon osd.1 on smithi142 2024-09-21T03:09:18.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:18 smithi083 ceph-mon[31219]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:18.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:18 smithi193 ceph-mon[35224]: Deploying daemon osd.1 on smithi142 2024-09-21T03:09:18.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:18 smithi193 ceph-mon[35224]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:20 smithi142 ceph-mon[34512]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:20.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:20 smithi193 ceph-mon[35224]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:20.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:20 smithi083 ceph-mon[31219]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:22.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:22 smithi142 ceph-mon[34512]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:22.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:22 smithi193 ceph-mon[35224]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:22 smithi083 ceph-mon[31219]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:23.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:23.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:23.352 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:23.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:24.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:24 smithi142 ceph-mon[34512]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:24 smithi193 ceph-mon[35224]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:24 smithi083 ceph-mon[31219]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:26.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:26.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:26.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:26.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:26.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.770+0000 7f4405ffb640 1 -- 172.21.15.142:0/118214107 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4408063800 con 0x7f43f40b6550 2024-09-21T03:09:26.772 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-09-21T03:09:26.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.772+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 msgr2=0x7f43f40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:26.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.772+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 0x7f43f40b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43f8031440 tx=0x7f43f8009560 comp rx=0 tx=0).stop 2024-09-21T03:09:26.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.772+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 msgr2=0x7f440819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.772+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f440000b970 tx=0x7f440000be30 comp rx=0 tx=0).stop 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 shutdown_connections 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f43f40b6550 0x7f43f40b8a60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4408109e00 0x7f440819fd30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44081094b0 0x7f440819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 --2- 172.21.15.142:0/118214107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4408103bc0 0x7f440819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 >> 172.21.15.142:0/118214107 conn(0x7f440807b910 msgr2=0x7f44081075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 shutdown_connections 2024-09-21T03:09:26.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:09:26.773+0000 7f440edc0640 1 -- 172.21.15.142:0/118214107 wait complete. 2024-09-21T03:09:27.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:27.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:27.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:27 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:27.351 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:09:27.354 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-21T03:09:27.354 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:09:27.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:27 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:27.604 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:09:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:27 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:28.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:28 smithi193 ceph-mon[35224]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:28.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:28 smithi142 ceph-mon[34512]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:28 smithi083 ceph-mon[31219]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:28.860 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:09:28.770+0000 7f5ce9d82640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:09:29.255 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:29 smithi193 ceph-mon[35224]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.255 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:29 smithi193 ceph-mon[35224]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:29 smithi142 ceph-mon[34512]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:29 smithi142 ceph-mon[34512]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:29 smithi083 ceph-mon[31219]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:29 smithi083 ceph-mon[31219]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:09:29.941 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:09:29.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-21T03:09:30.222 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:30.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:30.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:30.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:09:30.160+0000 7f5ce5d04640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:09:30.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:09:30.164+0000 7f5ce1b02640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:09:30.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:30.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:09:30.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:09:30.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:09:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:09:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:30.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 -- 172.21.15.193:0/2487609260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81027a0 msgr2=0x7fb1b8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2487609260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81027a0 0x7fb1b8102bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009900 tx=0x7fb1a802a410 comp rx=0 tx=0).stop 2024-09-21T03:09:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 -- 172.21.15.193:0/2487609260 shutdown_connections 2024-09-21T03:09:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2487609260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b8107be0 0x7fb1b81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2487609260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b81030f0 0x7fb1b8107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2487609260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81027a0 0x7fb1b8102bb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 -- 172.21.15.193:0/2487609260 >> 172.21.15.193:0/2487609260 conn(0x7fb1b8079730 msgr2=0x7fb1b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 -- 172.21.15.193:0/2487609260 shutdown_connections 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 -- 172.21.15.193:0/2487609260 wait complete. 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.668+0000 7fb1bd763640 1 Processor -- start 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 -- start start 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b81027a0 0x7fb1b81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 0x7fb1b81a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b811a0b0 con 0x7fb1b8107be0 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8119f30 con 0x7fb1b81027a0 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1bd763640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b811a230 con 0x7fb1b81030f0 2024-09-21T03:09:30.672 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1b6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b81027a0 0x7fb1b81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35444/0 (socket says 172.21.15.193:35444) 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 -- 172.21.15.193:0/2283995648 learned_addr learned my addr 172.21.15.193:0/2283995648 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1b77fe640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 0x7fb1b81a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 -- 172.21.15.193:0/2283995648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b81027a0 msgr2=0x7fb1b81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b81027a0 0x7fb1b81a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 -- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 msgr2=0x7fb1b81a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 0x7fb1b81a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 -- 172.21.15.193:0/2283995648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b81a6a10 con 0x7fb1b81030f0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1b77fe640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 0x7fb1b81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.669+0000 7fb1affff640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d950 tx=0x7fb1a000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.670+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0017070 con 0x7fb1b81030f0 2024-09-21T03:09:30.673 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.670+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a0009c30 con 0x7fb1b81030f0 2024-09-21T03:09:30.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.670+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b81a6d00 con 0x7fb1b81030f0 2024-09-21T03:09:30.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.670+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00047a0 con 0x7fb1b81030f0 2024-09-21T03:09:30.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.670+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81ad460 con 0x7fb1b81030f0 2024-09-21T03:09:30.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.671+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8107620 con 0x7fb1b81030f0 2024-09-21T03:09:30.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.672+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fb1a0004020 con 0x7fb1b81030f0 2024-09-21T03:09:30.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.673+0000 7fb1b4ff9640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 0x7fb1a40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:30.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.673+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fb1a00e5b90 con 0x7fb1b81030f0 2024-09-21T03:09:30.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.674+0000 7fb1b6ffd640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 0x7fb1a40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:30.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.674+0000 7fb1b6ffd640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 0x7fb1a40b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8004770 tx=0x7fb1a80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:30.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.677+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1a0020020 con 0x7fb1b81030f0 2024-09-21T03:09:30.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:30.864+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb1b8106d10 con 0x7fb1a40b64d0 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi142 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:31.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi142 2024-09-21T03:09:31.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-21T03:09:31.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:09:31.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:09:31.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:31.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:31.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi142 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:31.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:31.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:09:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:09:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:32.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:32.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] boot 2024-09-21T03:09:32.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:09:32.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:32 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:32.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:09:32.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:09:32.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:32.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:32.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] boot 2024-09-21T03:09:32.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:09:32.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:32 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:09:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:09:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:09:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: osd.1 [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] boot 2024-09-21T03:09:32.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:09:32.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:32 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:09:33.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:33 smithi193 ceph-mon[35224]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:09:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:33 smithi083 ceph-mon[31219]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:09:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:33 smithi142 ceph-mon[34512]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:09:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/1532386370' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"}]': finished 2024-09-21T03:09:34.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:09:34.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:34.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:34 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/1125568762' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/1532386370' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"}]': finished 2024-09-21T03:09:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:09:34.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:34.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:34 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/1125568762' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/1532386370' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"} : dispatch 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49bca58f-0193-459d-915a-a295270c7fb4"}]': finished 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:34 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/1125568762' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:36.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:36 smithi193 ceph-mon[35224]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:36 smithi083 ceph-mon[31219]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:36 smithi142 ceph-mon[34512]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:38.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:38 smithi193 ceph-mon[35224]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:38 smithi083 ceph-mon[31219]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:38 smithi142 ceph-mon[34512]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:40.337 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:40 smithi193 ceph-mon[35224]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:40 smithi083 ceph-mon[31219]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:40 smithi142 ceph-mon[34512]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:42.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:42 smithi142 ceph-mon[34512]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:42.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:09:42.480 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:42.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:42 smithi193 ceph-mon[35224]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:09:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:42 smithi083 ceph-mon[31219]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:09:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:43.471 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:43 smithi193 ceph-mon[35224]: Deploying daemon osd.2 on smithi193 2024-09-21T03:09:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:43 smithi083 ceph-mon[31219]: Deploying daemon osd.2 on smithi193 2024-09-21T03:09:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:43 smithi142 ceph-mon[34512]: Deploying daemon osd.2 on smithi193 2024-09-21T03:09:44.482 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:44 smithi193 ceph-mon[35224]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:44 smithi083 ceph-mon[31219]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:44 smithi142 ceph-mon[34512]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:46.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:46 smithi193 ceph-mon[35224]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:46 smithi083 ceph-mon[31219]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:46 smithi142 ceph-mon[34512]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:48.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:48 smithi193 ceph-mon[35224]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:48.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:48.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:48.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:48 smithi083 ceph-mon[31219]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:48 smithi142 ceph-mon[34512]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:50.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:50.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:51.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.920+0000 7fb1b4ff9640 1 -- 172.21.15.193:0/2283995648 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1b8106d10 con 0x7fb1a40b64d0 2024-09-21T03:09:51.922 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.923+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 msgr2=0x7fb1a40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.923+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 0x7fb1a40b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8004770 tx=0x7fb1a80058d0 comp rx=0 tx=0).stop 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.923+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 msgr2=0x7fb1b81a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.923+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d950 tx=0x7fb1a000de10 comp rx=0 tx=0).stop 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 shutdown_connections 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1a40b64d0 0x7fb1a40b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:51.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b8107be0 0x7fb1b81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:51.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b81030f0 0x7fb1b81a1040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:51.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 --2- 172.21.15.193:0/2283995648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1b81027a0 0x7fb1b81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:51.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 >> 172.21.15.193:0/2283995648 conn(0x7fb1b8079730 msgr2=0x7fb1b81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:51.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 shutdown_connections 2024-09-21T03:09:51.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:09:51.924+0000 7fb1bd763640 1 -- 172.21.15.193:0/2283995648 wait complete. 2024-09-21T03:09:52.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:52 smithi193 ceph-mon[35224]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:52.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:52 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:52.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:52 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:52 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.527 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:09:52.529 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-09-21T03:09:52.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:09:52.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:52 smithi083 ceph-mon[31219]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:52.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:52 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:52.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:52 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:52 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:52 smithi142 ceph-mon[34512]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:52 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:52 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:52 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:53.314 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:09:53.270+0000 7fb5f16a8640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:09:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:54 smithi083 ceph-mon[31219]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:54 smithi083 ceph-mon[31219]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:54 smithi083 ceph-mon[31219]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:54.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:54 smithi193 ceph-mon[35224]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:54.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:54 smithi193 ceph-mon[35224]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:54.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:54 smithi193 ceph-mon[35224]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:54 smithi142 ceph-mon[34512]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:54 smithi142 ceph-mon[34512]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:54 smithi142 ceph-mon[34512]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:09:55.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:09:55.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-09-21T03:09:55.464 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:09:55.230+0000 7fb5ed62a640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:09:55.464 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:09:55.235+0000 7fb5e9428640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:09:55.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-mon[35224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:09:55.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-mon[35224]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:09:55.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:55.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-mon[35224]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:55 smithi193 ceph-mon[35224]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.487 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:55.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:55 smithi083 ceph-mon[31219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:09:55.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:55 smithi083 ceph-mon[31219]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:09:55.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:55.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:55 smithi083 ceph-mon[31219]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:55 smithi083 ceph-mon[31219]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:55 smithi142 ceph-mon[34512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:09:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:55 smithi142 ceph-mon[34512]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:09:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:55 smithi142 ceph-mon[34512]: from='osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:55 smithi142 ceph-mon[34512]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:09:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 -- 172.21.15.83:0/1717395751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750102ae0 msgr2=0x7fd750102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 --2- 172.21.15.83:0/1717395751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750102ae0 0x7fd750102ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd744009900 tx=0x7fd74402a410 comp rx=0 tx=0).stop 2024-09-21T03:09:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 -- 172.21.15.83:0/1717395751 shutdown_connections 2024-09-21T03:09:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 --2- 172.21.15.83:0/1717395751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750108d70 0x7fd75010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 --2- 172.21.15.83:0/1717395751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108420 0x7fd750108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 --2- 172.21.15.83:0/1717395751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750102ae0 0x7fd750102ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.984+0000 7fd757f6d640 1 -- 172.21.15.83:0/1717395751 >> 172.21.15.83:0/1717395751 conn(0x7fd750079770 msgr2=0x7fd750079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.985+0000 7fd757f6d640 1 -- 172.21.15.83:0/1717395751 shutdown_connections 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.985+0000 7fd757f6d640 1 -- 172.21.15.83:0/1717395751 wait complete. 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.985+0000 7fd757f6d640 1 Processor -- start 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.985+0000 7fd757f6d640 1 -- start start 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750102ae0 0x7fd750199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 0x7fd75019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd750112a50 con 0x7fd750108420 2024-09-21T03:09:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501128d0 con 0x7fd750108d70 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd757f6d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd750112bd0 con 0x7fd750102ae0 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51752/0 (socket says 172.21.15.83:51752) 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 -- 172.21.15.83:0/4173050453 learned_addr learned my addr 172.21.15.83:0/4173050453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755d09640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750102ae0 0x7fd750199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd75650a640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 0x7fd75019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 -- 172.21.15.83:0/4173050453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750102ae0 msgr2=0x7fd750199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750102ae0 0x7fd750199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 -- 172.21.15.83:0/4173050453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 msgr2=0x7fd75019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 0x7fd75019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 -- 172.21.15.83:0/4173050453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75019f3b0 con 0x7fd750108420 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd75650a640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 0x7fd75019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:09:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd755508640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c970 tx=0x7fd74c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.986+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c004220 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.987+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75019f6a0 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.987+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c0043c0 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.987+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00f450 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.987+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7501a5e00 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.988+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd750103d60 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.989+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd74c0079b0 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.990+0000 7fd742ffd640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 0x7fd73c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.990+0000 7fd755d09640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 0x7fd73c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.990+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7fd74c0e5f50 con 0x7fd750108420 2024-09-21T03:09:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.991+0000 7fd755d09640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 0x7fd73c0b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd744031440 tx=0x7fd744005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:55.994+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd74c014030 con 0x7fd750108420 2024-09-21T03:09:56.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:09:56.182+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd75010ab70 con 0x7fd73c0b64d0 2024-09-21T03:09:56.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:56.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:56.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:56.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:56.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:56.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:09:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:09:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:09:56.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:09:56.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:09:56.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi193 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] boot 2024-09-21T03:09:57.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:09:57.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:57.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:57 smithi083 ceph-mon[31219]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi193 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] boot 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:57 smithi193 ceph-mon[35224]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi193 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: osd.2 [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] boot 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:09:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:09:57.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:57 smithi142 ceph-mon[34512]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:09:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:58 smithi193 ceph-mon[35224]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:09:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:58 smithi083 ceph-mon[31219]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:09:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:58 smithi142 ceph-mon[34512]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:09:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:59.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"} : dispatch 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"}]': finished 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:09:59.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2915246360' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52658]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52658]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52666]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.447 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52666]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.500 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44142]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44142]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44142]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44145]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44145]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.501 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44145]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:09:59.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"} : dispatch 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"}]': finished 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2915246360' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44148]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44148]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.525 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43080]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43080]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43083]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.526 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43083]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.526 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.526 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:09:59.526 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:09:59.526 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"} : dispatch 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3722412507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "280e4e6e-e486-492c-b238-36b38206052c"}]': finished 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2915246360' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:09:59.527 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44151]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44151]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44151]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:09:59 smithi193 sudo[44151]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52683]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52683]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52686]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52686]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:09:59 smithi083 sudo[52686]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43086]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43086]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43089]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:09:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:09:59 smithi142 sudo[43089]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:10:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:10:00.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:00.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:00 smithi083 ceph-mon[31219]: overall HEALTH_OK 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:00.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:00 smithi193 ceph-mon[35224]: overall HEALTH_OK 2024-09-21T03:10:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:00 smithi142 ceph-mon[34512]: overall HEALTH_OK 2024-09-21T03:10:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:01.307+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd74c0a9920 con 0x7fd750108420 2024-09-21T03:10:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:01 smithi193 ceph-mon[35224]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:01 smithi083 ceph-mon[31219]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:01 smithi142 ceph-mon[34512]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:02.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:02 smithi193 ceph-mon[35224]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:10:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:02 smithi083 ceph-mon[31219]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:10:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:02 smithi142 ceph-mon[34512]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:10:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:03 smithi193 ceph-mon[35224]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:03 smithi083 ceph-mon[31219]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:03 smithi142 ceph-mon[34512]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:06.236 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:06 smithi083 ceph-mon[31219]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:06.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:06 smithi142 ceph-mon[34512]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:06.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:06 smithi193 ceph-mon[35224]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:08.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:08 smithi083 ceph-mon[31219]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:08.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:10:08.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:08 smithi142 ceph-mon[34512]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:10:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:08 smithi193 ceph-mon[35224]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:10:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:09.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:09 smithi083 ceph-mon[31219]: Deploying daemon osd.3 on smithi083 2024-09-21T03:10:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:09 smithi142 ceph-mon[34512]: Deploying daemon osd.3 on smithi083 2024-09-21T03:10:09.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:09 smithi193 ceph-mon[35224]: Deploying daemon osd.3 on smithi083 2024-09-21T03:10:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:10 smithi142 ceph-mon[34512]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:10.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:10 smithi083 ceph-mon[31219]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:10.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:10 smithi193 ceph-mon[35224]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:12.239 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:12 smithi083 ceph-mon[31219]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:12.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:12 smithi193 ceph-mon[35224]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:12 smithi142 ceph-mon[34512]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:14 smithi083 ceph-mon[31219]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:14 smithi193 ceph-mon[35224]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:14 smithi142 ceph-mon[34512]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:16 smithi193 ceph-mon[35224]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:16 smithi083 ceph-mon[31219]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:16 smithi142 ceph-mon[34512]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:17.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.030+0000 7fd742ffd640 1 -- 172.21.15.83:0/4173050453 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd75010ab70 con 0x7fd73c0b64d0 2024-09-21T03:10:17.032 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-09-21T03:10:17.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 msgr2=0x7fd73c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:17.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 0x7fd73c0b89e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd744031440 tx=0x7fd744005d50 comp rx=0 tx=0).stop 2024-09-21T03:10:17.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 msgr2=0x7fd7501999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c970 tx=0x7fd74c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 shutdown_connections 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd73c0b64d0 0x7fd73c0b89e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd750108d70 0x7fd75019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd750108420 0x7fd7501999e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 --2- 172.21.15.83:0/4173050453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750102ae0 0x7fd750199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 >> 172.21.15.83:0/4173050453 conn(0x7fd750079770 msgr2=0x7fd7501064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 shutdown_connections 2024-09-21T03:10:17.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:10:17.033+0000 7fd757f6d640 1 -- 172.21.15.83:0/4173050453 wait complete. 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:17 smithi193 ceph-mon[35224]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:17 smithi083 ceph-mon[31219]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:17.667 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:10:17.670 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-09-21T03:10:17.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:17 smithi142 ceph-mon[34512]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:17.923 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:10:19.384 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:10:19 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:10:19.232+0000 7fbc76f27640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:10:20.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:20 smithi083 ceph-mon[31219]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:20.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:20 smithi083 ceph-mon[31219]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:10:20.389 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:20 smithi142 ceph-mon[34512]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:20.389 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:20 smithi142 ceph-mon[34512]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:10:20.392 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:10:20.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-09-21T03:10:20.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:20 smithi193 ceph-mon[35224]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:20 smithi193 ceph-mon[35224]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:10:20.709 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:10:21.178 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3410447539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a41083d0 msgr2=0x7fe1a41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3410447539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a41083d0 0x7fe1a41087e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02a410 comp rx=0 tx=0).stop 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3410447539 shutdown_connections 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3410447539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4108d20 0x7fe1a410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3410447539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a41083d0 0x7fe1a41087e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3410447539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a4102ae0 0x7fe1a4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3410447539 >> 172.21.15.142:0/3410447539 conn(0x7fe1a4079790 msgr2=0x7fe1a4079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3410447539 shutdown_connections 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.200+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3410447539 wait complete. 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 Processor -- start 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 -- start start 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4113b00 con 0x7fe1a4102ae0 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4113980 con 0x7fe1a4108d20 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.201+0000 7fe1a8daf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4113c80 con 0x7fe1a41083d0 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a1d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:21.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a1d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.142:43828/0 (socket says 172.21.15.142:43828) 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33194/0 (socket says 172.21.15.142:33194) 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a1d9b640 1 -- 172.21.15.142:0/3248996437 learned_addr learned my addr 172.21.15.142:0/3248996437 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.142:34434/0 (socket says 172.21.15.142:34434) 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 -- 172.21.15.142:0/3248996437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 msgr2=0x7fe1a419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 -- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 msgr2=0x7fe1a419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 -- 172.21.15.142:0/3248996437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a41a0440 con 0x7fe1a4108d20 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a1d9b640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a259c640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:21.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe1a2d9d640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009fd0 tx=0x7fe19c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c00ce20 con 0x7fe1a4108d20 2024-09-21T03:10:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe19c005020 con 0x7fe1a4108d20 2024-09-21T03:10:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.202+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c010690 con 0x7fe1a4108d20 2024-09-21T03:10:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.203+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a41a0730 con 0x7fe1a4108d20 2024-09-21T03:10:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.203+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a410ff10 con 0x7fe1a4108d20 2024-09-21T03:10:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.204+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a4103d10 con 0x7fe1a4108d20 2024-09-21T03:10:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.205+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe19c0054d0 con 0x7fe1a4108d20 2024-09-21T03:10:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.206+0000 7fe19b7fe640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 0x7fe1900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.207+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3604+0+0 (secure 0 0 0) 0x7fe19c0e67c0 con 0x7fe1a4108d20 2024-09-21T03:10:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.207+0000 7fe1a259c640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 0x7fe1900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.207+0000 7fe1a259c640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 0x7fe1900b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe194005fd0 tx=0x7fe19400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:21.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.211+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe19c0ae2b0 con 0x7fe1a4108d20 2024-09-21T03:10:21.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:21.404+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe1a4063800 con 0x7fe1900b6560 2024-09-21T03:10:21.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:10:21.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:10:21.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:10:21.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:21.406 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.407 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:21 smithi142 ceph-mon[34512]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:21 smithi193 ceph-mon[35224]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:10:21.619 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:10:21.619 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-mon[31219]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:10:21.872 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:10:21.617+0000 7fbc72ea9640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:10:21.872 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:10:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:10:21.625+0000 7fbc6ac9f640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi083 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' 2024-09-21T03:10:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi083 2024-09-21T03:10:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-21T03:10:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:10:22.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' 2024-09-21T03:10:22.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi083 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 8544M 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910]' entity='osd.3' 2024-09-21T03:10:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:10:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:10:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] boot 2024-09-21T03:10:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:10:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:23.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:23 smithi142 ceph-mon[34512]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] boot 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:23 smithi193 ceph-mon[35224]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: osd.3 [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] boot 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:23 smithi083 ceph-mon[31219]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/1367683240' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"}]': finished 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:10:24.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/1367683240' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"}]': finished 2024-09-21T03:10:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:10:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/1367683240' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"} : dispatch 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6f14a7-871f-4d09-a704-71022463636c"}]': finished 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:10:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:25.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:25 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/2199006546' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/2199006546' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:25 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/2199006546' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:26.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:26 smithi193 ceph-mon[35224]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:26 smithi083 ceph-mon[31219]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:26 smithi142 ceph-mon[34512]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:28 smithi193 ceph-mon[35224]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:28 smithi083 ceph-mon[31219]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:28 smithi142 ceph-mon[34512]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:30.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:30 smithi142 ceph-mon[34512]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:10:30.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:30 smithi193 ceph-mon[35224]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:10:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:30 smithi083 ceph-mon[31219]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:10:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:32 smithi083 ceph-mon[31219]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:10:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:32 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:10:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:32 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:32 smithi142 ceph-mon[34512]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:10:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:32 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:10:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:32 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:32.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:32 smithi193 ceph-mon[35224]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:10:32.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:32 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:10:32.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:32 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:33 smithi193 ceph-mon[35224]: Deploying daemon osd.4 on smithi142 2024-09-21T03:10:33.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:33 smithi142 ceph-mon[34512]: Deploying daemon osd.4 on smithi142 2024-09-21T03:10:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:33 smithi083 ceph-mon[31219]: Deploying daemon osd.4 on smithi142 2024-09-21T03:10:34.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:34 smithi193 ceph-mon[35224]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:10:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:34 smithi083 ceph-mon[31219]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:10:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:34 smithi142 ceph-mon[34512]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:10:36.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:36 smithi193 ceph-mon[35224]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:10:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:36 smithi083 ceph-mon[31219]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:10:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:36 smithi142 ceph-mon[34512]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:10:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:38 smithi193 ceph-mon[35224]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:38 smithi083 ceph-mon[31219]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:38 smithi142 ceph-mon[34512]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:39.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:39.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:40.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:40 smithi142 ceph-mon[34512]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:40 smithi193 ceph-mon[35224]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:40 smithi083 ceph-mon[31219]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:10:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.368 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:42 smithi193 ceph-mon[35224]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:42 smithi083 ceph-mon[31219]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:42.401 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:42.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:42.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:42.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:42 smithi142 ceph-mon[34512]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:42.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.442+0000 7fe19b7fe640 1 -- 172.21.15.142:0/3248996437 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1a4063800 con 0x7fe1900b6560 2024-09-21T03:10:42.445 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-09-21T03:10:42.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 msgr2=0x7fe1900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:42.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 0x7fe1900b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe194005fd0 tx=0x7fe19400b390 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 msgr2=0x7fe1a419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009fd0 tx=0x7fe19c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 shutdown_connections 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1900b6560 0x7fe1900b8a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a4108d20 0x7fe1a419fd10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1a41083d0 0x7fe1a419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 --2- 172.21.15.142:0/3248996437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1a4102ae0 0x7fe1a419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 >> 172.21.15.142:0/3248996437 conn(0x7fe1a4079790 msgr2=0x7fe1a41064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 shutdown_connections 2024-09-21T03:10:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:10:42.445+0000 7fe1a8daf640 1 -- 172.21.15.142:0/3248996437 wait complete. 2024-09-21T03:10:43.091 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:10:43.093 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-21T03:10:43.093 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:10:43.343 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:10:43.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:43 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:43.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:43 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:43.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:43 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:43 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:43 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:43 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:43.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:43 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:43.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:43 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:43.464 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:43 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:44.220 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:44 smithi142 ceph-mon[34512]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:44.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:44 smithi083 ceph-mon[31219]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:44.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:44 smithi193 ceph-mon[35224]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:44.765 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:10:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:10:44.496+0000 7fb6dd26f640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:10:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:45 smithi083 ceph-mon[31219]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:45 smithi083 ceph-mon[31219]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:45 smithi142 ceph-mon[34512]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:45 smithi142 ceph-mon[34512]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:45 smithi193 ceph-mon[35224]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:45 smithi193 ceph-mon[35224]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:10:45.843 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:10:45.868 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-21T03:10:46.114 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.237 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:46.237 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:46.237 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.427 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:10:46.129+0000 7fb6d91f1640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:10:46.427 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:10:46.135+0000 7fb6d4fef640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:10:46.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:46.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:10:46.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:10:46.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:46.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:46.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:10:46.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- 172.21.15.193:0/1215930487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 msgr2=0x7f12d8103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:46.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 --2- 172.21.15.193:0/1215930487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d8103fc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009900 tx=0x7f12cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:10:46.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- 172.21.15.193:0/1215930487 shutdown_connections 2024-09-21T03:10:46.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 --2- 172.21.15.193:0/1215930487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d8109e40 0x7f12d8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:46.638 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 --2- 172.21.15.193:0/1215930487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d81094f0 0x7f12d8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:46.638 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 --2- 172.21.15.193:0/1215930487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d8103fc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:46.638 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- 172.21.15.193:0/1215930487 >> 172.21.15.193:0/1215930487 conn(0x7f12d80ff600 msgr2=0x7f12d8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:46.638 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- 172.21.15.193:0/1215930487 shutdown_connections 2024-09-21T03:10:46.638 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- 172.21.15.193:0/1215930487 wait complete. 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 Processor -- start 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.636+0000 7f12df0aa640 1 -- start start 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d81094f0 0x7f12d819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d8109e40 0x7f12d819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8113b00 con 0x7f12d8109e40 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8113980 con 0x7f12d81094f0 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12df0aa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8113c80 con 0x7f12d8103bb0 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:47260/0 (socket says 172.21.15.193:47260) 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 -- 172.21.15.193:0/3952111739 learned_addr learned my addr 172.21.15.193:0/3952111739 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12d7fff640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d81094f0 0x7f12d819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dd647640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d8109e40 0x7f12d819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:46.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 -- 172.21.15.193:0/3952111739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d81094f0 msgr2=0x7f12d819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d81094f0 0x7f12d819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 -- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d8109e40 msgr2=0x7f12d819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d8109e40 0x7f12d819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 -- 172.21.15.193:0/3952111739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d81a0440 con 0x7f12d8103bb0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.637+0000 7f12dce46640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12cc005940 tx=0x7f12cc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.638+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc03f030 con 0x7f12d8103bb0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.638+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d81a06d0 con 0x7f12d8103bb0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.638+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12cc003cb0 con 0x7f12d8103bb0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.638+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc035070 con 0x7f12d8103bb0 2024-09-21T03:10:46.641 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.638+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d81a6e90 con 0x7f12d8103bb0 2024-09-21T03:10:46.642 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.639+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d8104e30 con 0x7f12d8103bb0 2024-09-21T03:10:46.642 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.640+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f12cc042030 con 0x7f12d8103bb0 2024-09-21T03:10:46.642 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.641+0000 7f12d5ffb640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 0x7f12c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:46.642 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.641+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4036+0+0 (secure 0 0 0) 0x7f12cc106b70 con 0x7f12d8103bb0 2024-09-21T03:10:46.642 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.641+0000 7f12d7fff640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 0x7f12c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:46.643 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.642+0000 7f12d7fff640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 0x7f12c80b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12d8108ff0 tx=0x7f12c400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:46.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.645+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12cc0ce4b0 con 0x7f12d8103bb0 2024-09-21T03:10:46.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:10:46.834+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f12d8063800 con 0x7f12c80b6560 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi142 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:47.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:47.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi142 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:47.462 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi142 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:10:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] boot 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:10:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] boot 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:10:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:48.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:10:48.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:10:48.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:10:48.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:10:48.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: osd.4 [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] boot 2024-09-21T03:10:48.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:10:48.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:10:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:49 smithi193 ceph-mon[35224]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:10:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:49 smithi142 ceph-mon[34512]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:10:49.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:49 smithi083 ceph-mon[31219]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T03:10:50.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/882876628' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"}]': finished 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:10:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/1206560663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/882876628' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"}]': finished 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:10:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/1206560663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/882876628' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"} : dispatch 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0de58bca-db3b-4600-83bb-9df3665dc790"}]': finished 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:10:50.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/1206560663' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:10:52.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:52 smithi142 ceph-mon[34512]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:52 smithi193 ceph-mon[35224]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:52.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:52 smithi083 ceph-mon[31219]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:54.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:54 smithi142 ceph-mon[34512]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:54 smithi193 ceph-mon[35224]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:54 smithi083 ceph-mon[31219]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:56.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:56 smithi193 ceph-mon[35224]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:56.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:56 smithi142 ceph-mon[34512]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:56 smithi083 ceph-mon[31219]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:58 smithi193 ceph-mon[35224]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:10:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:58.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:58 smithi142 ceph-mon[34512]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:58.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:10:58.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:58 smithi083 ceph-mon[31219]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:10:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:10:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:59.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:10:59 smithi193 ceph-mon[35224]: Deploying daemon osd.5 on smithi193 2024-09-21T03:10:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:10:59 smithi142 ceph-mon[34512]: Deploying daemon osd.5 on smithi193 2024-09-21T03:10:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:10:59 smithi083 ceph-mon[31219]: Deploying daemon osd.5 on smithi193 2024-09-21T03:11:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:00 smithi142 ceph-mon[34512]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:00.500 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:00 smithi193 ceph-mon[35224]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:00.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:00 smithi083 ceph-mon[31219]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:02.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:02 smithi142 ceph-mon[34512]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:02 smithi193 ceph-mon[35224]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:02 smithi083 ceph-mon[31219]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:04.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:04 smithi193 ceph-mon[35224]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:04 smithi083 ceph-mon[31219]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:04 smithi142 ceph-mon[34512]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:05.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:05.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:05.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:06.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:06 smithi193 ceph-mon[35224]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:06 smithi083 ceph-mon[31219]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:06 smithi142 ceph-mon[34512]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:08.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:08.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:08.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.420 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.419+0000 7f12d5ffb640 1 -- 172.21.15.193:0/3952111739 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12d8063800 con 0x7f12c80b6560 2024-09-21T03:11:08.421 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 msgr2=0x7f12c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 0x7f12c80b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12d8108ff0 tx=0x7f12c400b410 comp rx=0 tx=0).stop 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 msgr2=0x7f12d819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12cc005940 tx=0x7f12cc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 shutdown_connections 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12c80b6560 0x7f12c80b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:08.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d8109e40 0x7f12d819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:08.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d81094f0 0x7f12d819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:08.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 --2- 172.21.15.193:0/3952111739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8103bb0 0x7f12d819a4d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:08.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 >> 172.21.15.193:0/3952111739 conn(0x7f12d80ff600 msgr2=0x7f12d8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:08.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 shutdown_connections 2024-09-21T03:11:08.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:11:08.422+0000 7f12df0aa640 1 -- 172.21.15.193:0/3952111739 wait complete. 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:08.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:08.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:08.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:08.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.101 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:11:09.104 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-09-21T03:11:09.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:11:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:09.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:09.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:09.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:09.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:10.393 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:10 smithi193 ceph-mon[35224]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:10.394 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:11:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:11:10.373+0000 7f908131b640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:11:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:10 smithi142 ceph-mon[34512]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:10 smithi083 ceph-mon[31219]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:11.281 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:11 smithi083 ceph-mon[31219]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:11.281 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:11 smithi083 ceph-mon[31219]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:11.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:11 smithi142 ceph-mon[34512]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:11.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:11 smithi142 ceph-mon[34512]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:11 smithi193 ceph-mon[35224]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:11 smithi193 ceph-mon[35224]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:11:12.191 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:11:12.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-09-21T03:11:12.315 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:11:12.145+0000 7f907d29d640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:11:12.315 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:11:12.151+0000 7f907909b640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:11:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:11:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:11:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:12.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:12.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:11:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:11:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:12.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 -- 172.21.15.83:0/1452558477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109eb0 msgr2=0x7fba4c1106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1452558477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109eb0 0x7fba4c1106e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba44009ca0 tx=0x7fba4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 -- 172.21.15.83:0/1452558477 shutdown_connections 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1452558477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109eb0 0x7fba4c1106e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1452558477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109560 0x7fba4c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1452558477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.997+0000 7fba50fb7640 1 -- 172.21.15.83:0/1452558477 >> 172.21.15.83:0/1452558477 conn(0x7fba4c079ec0 msgr2=0x7fba4c07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.998+0000 7fba50fb7640 1 -- 172.21.15.83:0/1452558477 shutdown_connections 2024-09-21T03:11:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.998+0000 7fba50fb7640 1 -- 172.21.15.83:0/1452558477 wait complete. 2024-09-21T03:11:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.998+0000 7fba50fb7640 1 Processor -- start 2024-09-21T03:11:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 -- start start 2024-09-21T03:11:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 0x7fba4c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109eb0 0x7fba4c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c113bf0 con 0x7fba4c109560 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c113a70 con 0x7fba4c102080 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba50fb7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c113d70 con 0x7fba4c109eb0 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba4a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba4a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:43184/0 (socket says 172.21.15.83:43184) 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba4a59c640 1 -- 172.21.15.83:0/1045995973 learned_addr learned my addr 172.21.15.83:0/1045995973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 0x7fba4c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba4ad9d640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109eb0 0x7fba4c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 -- 172.21.15.83:0/1045995973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109eb0 msgr2=0x7fba4c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109eb0 0x7fba4c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 -- 172.21.15.83:0/1045995973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 msgr2=0x7fba4c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:12.999+0000 7fba41d9b640 1 -- 172.21.15.83:0/1045995973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c1a04f0 con 0x7fba4c109560 2024-09-21T03:11:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba4a59c640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba41d9b640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 0x7fba4c19ab20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba4c069f90 tx=0x7fba3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34017070 con 0x7fba4c109560 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba340099a0 con 0x7fba4c109560 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c1a07e0 con 0x7fba4c109560 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340047a0 con 0x7fba4c109560 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.000+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c1a6f40 con 0x7fba4c109560 2024-09-21T03:11:13.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.001+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c06b8f0 con 0x7fba4c109560 2024-09-21T03:11:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.002+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fba34004940 con 0x7fba4c109560 2024-09-21T03:11:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.003+0000 7fba43fff640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 0x7fba380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.003+0000 7fba4a59c640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 0x7fba380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.003+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4468+0+0 (secure 0 0 0) 0x7fba340e6410 con 0x7fba4c109560 2024-09-21T03:11:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.004+0000 7fba4a59c640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 0x7fba380b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009e10 tx=0x7fba3c0091c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:13.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.006+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba34020020 con 0x7fba4c109560 2024-09-21T03:11:13.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:13.198+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fba4c063800 con 0x7fba380b6560 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi193 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:13.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi193 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:13.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi193 2024-09-21T03:11:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T03:11:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:11:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T03:11:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:11:14.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:11:14.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] boot 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:14.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:14 smithi142 ceph-mon[34512]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] boot 2024-09-21T03:11:14.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:14.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:14 smithi193 ceph-mon[35224]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: osd.5 [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] boot 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:14 smithi083 ceph-mon[31219]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:11:16.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:15 smithi193 ceph-mon[35224]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:11:16.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:15 smithi193 ceph-mon[35224]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:16.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:15 smithi083 ceph-mon[31219]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:11:16.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:15 smithi083 ceph-mon[31219]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:16.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:15 smithi142 ceph-mon[34512]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T03:11:16.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:15 smithi142 ceph-mon[34512]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"} : dispatch 2024-09-21T03:11:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"}]': finished 2024-09-21T03:11:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:16 smithi193 ceph-mon[35224]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:11:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2179850594' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:17.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"} : dispatch 2024-09-21T03:11:17.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"}]': finished 2024-09-21T03:11:17.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:16 smithi083 ceph-mon[31219]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:11:17.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:17.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2179850594' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"} : dispatch 2024-09-21T03:11:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1771894978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a5a5708-35f7-4e4b-a46c-3c1afab03a79"}]': finished 2024-09-21T03:11:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:16 smithi142 ceph-mon[34512]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:11:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2179850594' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:18.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:17 smithi083 ceph-mon[31219]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:18.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:17 smithi193 ceph-mon[35224]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:18.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:17 smithi142 ceph-mon[34512]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:20.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:20 smithi083 ceph-mon[31219]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:11:20.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:20 smithi193 ceph-mon[35224]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:11:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:20 smithi142 ceph-mon[34512]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:11:22.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:22 smithi083 ceph-mon[31219]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T03:11:22.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:22 smithi193 ceph-mon[35224]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T03:11:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:22 smithi142 ceph-mon[34512]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T03:11:24.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:24 smithi193 ceph-mon[35224]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:11:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:24 smithi083 ceph-mon[31219]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:11:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:24 smithi142 ceph-mon[34512]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T03:11:25.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:25 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:11:25.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:25 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:25 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:11:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:25 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:25 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:11:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:25 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:26 smithi083 ceph-mon[31219]: Deploying daemon osd.6 on smithi083 2024-09-21T03:11:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:26 smithi083 ceph-mon[31219]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:11:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:26 smithi193 ceph-mon[35224]: Deploying daemon osd.6 on smithi083 2024-09-21T03:11:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:26 smithi193 ceph-mon[35224]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:11:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:26 smithi142 ceph-mon[34512]: Deploying daemon osd.6 on smithi083 2024-09-21T03:11:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:26 smithi142 ceph-mon[34512]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:11:28.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:28 smithi083 ceph-mon[31219]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:11:28.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:28 smithi193 ceph-mon[35224]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:11:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:28 smithi142 ceph-mon[34512]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:11:30.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:30 smithi083 ceph-mon[31219]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:11:30.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:30 smithi193 ceph-mon[35224]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:11:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:30 smithi142 ceph-mon[34512]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:11:31.499 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:31.499 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:31.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:32 smithi083 ceph-mon[31219]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:32 smithi193 ceph-mon[35224]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:32 smithi142 ceph-mon[34512]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.832+0000 7fba43fff640 1 -- 172.21.15.83:0/1045995973 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fba4c063800 con 0x7fba380b6560 2024-09-21T03:11:34.834 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.834+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 msgr2=0x7fba380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.834+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 0x7fba380b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009e10 tx=0x7fba3c0091c0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.834+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 msgr2=0x7fba4c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.834+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 0x7fba4c19ab20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba4c069f90 tx=0x7fba3400dbb0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.834+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 shutdown_connections 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba380b6560 0x7fba380b8a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c109eb0 0x7fba4c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba4c109560 0x7fba4c19ab20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 --2- 172.21.15.83:0/1045995973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba4c102080 0x7fba4c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 >> 172.21.15.83:0/1045995973 conn(0x7fba4c079ec0 msgr2=0x7fba4c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:34.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 shutdown_connections 2024-09-21T03:11:34.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:11:34.835+0000 7fba50fb7640 1 -- 172.21.15.83:0/1045995973 wait complete. 2024-09-21T03:11:35.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:35.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.466 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:11:35.468 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2024-09-21T03:11:35.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:11:35.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:35.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:35.723 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:36.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:36 smithi142 ceph-mon[34512]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:36.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:36 smithi193 ceph-mon[35224]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:36 smithi083 ceph-mon[31219]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:37.622 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:11:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:11:37.223+0000 7f2b13341640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:11:38.214 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:11:38.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-09-21T03:11:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:38 smithi142 ceph-mon[34512]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:38 smithi142 ceph-mon[34512]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:11:38.532 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:38 smithi193 ceph-mon[35224]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:38 smithi193 ceph-mon[35224]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:11:38.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:38 smithi083 ceph-mon[31219]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:38.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:38 smithi083 ceph-mon[31219]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/3445555899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78105190 msgr2=0x7f0a781055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/3445555899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78105190 0x7f0a781055a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009900 tx=0x7f0a6402a410 comp rx=0 tx=0).stop 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/3445555899 shutdown_connections 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/3445555899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 0x7f0a7810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/3445555899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78105190 0x7f0a781055a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/3445555899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a781031c0 0x7f0a781035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/3445555899 >> 172.21.15.142:0/3445555899 conn(0x7f0a78077c60 msgr2=0x7f0a780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/3445555899 shutdown_connections 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.058+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/3445555899 wait complete. 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 Processor -- start 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 -- start start 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 0x7f0a7819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 0x7f0a7819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78112980 con 0x7f0a78105ae0 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78112800 con 0x7f0a781031c0 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7ee8b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78112b00 con 0x7f0a78105190 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7cc27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7cc27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36800/0 (socket says 172.21.15.142:36800) 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.059+0000 7f0a7cc27640 1 -- 172.21.15.142:0/1275387699 learned_addr learned my addr 172.21.15.142:0/1275387699 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-21T03:11:39.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a77fff640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 0x7f0a7819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7d428640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 0x7f0a7819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 -- 172.21.15.142:0/1275387699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 msgr2=0x7f0a7819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 0x7f0a7819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 -- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 msgr2=0x7f0a7819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 0x7f0a7819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 -- 172.21.15.142:0/1275387699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a781a6600 con 0x7f0a781031c0 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a77fff640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 0x7f0a7819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7cc27640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d770 tx=0x7f0a6c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:39.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c017070 con 0x7f0a781031c0 2024-09-21T03:11:39.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a781a68f0 con 0x7f0a781031c0 2024-09-21T03:11:39.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c0054d0 con 0x7f0a781031c0 2024-09-21T03:11:39.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c004c30 con 0x7f0a781031c0 2024-09-21T03:11:39.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.060+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a781a6db0 con 0x7f0a781031c0 2024-09-21T03:11:39.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.062+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a781035d0 con 0x7f0a781031c0 2024-09-21T03:11:39.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.064+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f0a6c004dd0 con 0x7f0a781031c0 2024-09-21T03:11:39.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.065+0000 7f0a75ffb640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 0x7f0a680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:39.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.065+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4884+0+0 (secure 0 0 0) 0x7f0a6c0e6560 con 0x7f0a781031c0 2024-09-21T03:11:39.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.065+0000 7f0a77fff640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 0x7f0a680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:39.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.066+0000 7f0a77fff640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 0x7f0a680b8aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000c00 tx=0x7f0a64032cc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:39.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.068+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a6c0adb30 con 0x7f0a781031c0 2024-09-21T03:11:39.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:39.263+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0a78063800 con 0x7f0a680b6590 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.506 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:39.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:39 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:39.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:39.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:39 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:11:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:11:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:11:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:39.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:40.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:11:39.661+0000 7f2b0fad8640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:11:40.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:11:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:11:39.670+0000 7f2b070b9640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi083 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:40.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:40.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:40.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:40 smithi193 ceph-mon[35224]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' 2024-09-21T03:11:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:11:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:11:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi083 2024-09-21T03:11:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:40.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:40 smithi083 ceph-mon[31219]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi083 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 5696M 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T03:11:40.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:40.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:11:40.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:11:40.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:40.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:40 smithi142 ceph-mon[34512]: from='osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109]' entity='osd.6' 2024-09-21T03:11:41.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:41 smithi193 ceph-mon[35224]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:41 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:41 smithi193 ceph-mon[35224]: osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] boot 2024-09-21T03:11:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:41 smithi193 ceph-mon[35224]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:11:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:41 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:41 smithi083 ceph-mon[31219]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:41 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:41 smithi083 ceph-mon[31219]: osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] boot 2024-09-21T03:11:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:41 smithi083 ceph-mon[31219]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:11:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:41 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:41 smithi142 ceph-mon[34512]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:41 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:41 smithi142 ceph-mon[34512]: osd.6 [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] boot 2024-09-21T03:11:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:41 smithi142 ceph-mon[34512]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:11:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:41 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/4095628594' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"}]': finished 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:11:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:11:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:11:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/4095628594' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"}]': finished 2024-09-21T03:11:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:11:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/4095628594' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"} : dispatch 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07842e54-9f66-42d4-9a74-2458dbdd8c37"}]': finished 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:11:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:11:43.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/2501232370' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/2501232370' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/2501232370' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:11:44.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:44 smithi193 ceph-mon[35224]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:44 smithi083 ceph-mon[31219]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:44 smithi142 ceph-mon[34512]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:45 smithi193 ceph-mon[35224]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:45 smithi083 ceph-mon[31219]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:45 smithi142 ceph-mon[34512]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:48.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:48 smithi142 ceph-mon[34512]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:48.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:48 smithi193 ceph-mon[35224]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:48 smithi083 ceph-mon[31219]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:50.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:50 smithi083 ceph-mon[31219]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:50.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:50 smithi193 ceph-mon[35224]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:50.631 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:50 smithi142 ceph-mon[34512]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:51.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:11:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:51.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:11:51.616 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:11:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:52.227 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:52 smithi142 ceph-mon[34512]: Deploying daemon osd.7 on smithi142 2024-09-21T03:11:52.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:52 smithi142 ceph-mon[34512]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:52 smithi193 ceph-mon[35224]: Deploying daemon osd.7 on smithi142 2024-09-21T03:11:52.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:52 smithi193 ceph-mon[35224]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:52.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:52 smithi083 ceph-mon[31219]: Deploying daemon osd.7 on smithi142 2024-09-21T03:11:52.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:52 smithi083 ceph-mon[31219]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:54.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:54 smithi193 ceph-mon[35224]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:54 smithi083 ceph-mon[31219]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:54 smithi142 ceph-mon[34512]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:56.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:56 smithi193 ceph-mon[35224]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:56 smithi083 ceph-mon[31219]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:56 smithi142 ceph-mon[34512]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:11:58.355 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:11:58 smithi142 ceph-mon[34512]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:11:58 smithi193 ceph-mon[35224]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:11:58 smithi083 ceph-mon[31219]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:11:59.762 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-09-21T03:11:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.760+0000 7f0a75ffb640 1 -- 172.21.15.142:0/1275387699 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0a78063800 con 0x7f0a680b6590 2024-09-21T03:11:59.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.762+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 msgr2=0x7f0a680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.762+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 0x7f0a680b8aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000c00 tx=0x7f0a64032cc0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 msgr2=0x7f0a7819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d770 tx=0x7f0a6c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 shutdown_connections 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a680b6590 0x7f0a680b8aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a78105ae0 0x7f0a7819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78105190 0x7f0a7819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 --2- 172.21.15.142:0/1275387699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a781031c0 0x7f0a7819b370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 >> 172.21.15.142:0/1275387699 conn(0x7f0a78077c60 msgr2=0x7f0a781065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 shutdown_connections 2024-09-21T03:11:59.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:11:59.763+0000 7f0a7ee8b640 1 -- 172.21.15.142:0/1275387699 wait complete. 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.397 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.398 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:00.424 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:12:00.427 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-21T03:12:00.427 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:12:00.683 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:12:02.345 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:02 smithi193 ceph-mon[35224]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:02.551 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:02 smithi142 ceph-mon[34512]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:02 smithi083 ceph-mon[31219]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:02.927 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:12:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:12:02.652+0000 7fb87618d640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:12:03.227 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:12:03.254 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-21T03:12:03.512 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:12:03.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:03 smithi193 ceph-mon[35224]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:03 smithi193 ceph-mon[35224]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:03 smithi083 ceph-mon[31219]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:03 smithi083 ceph-mon[31219]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:03 smithi142 ceph-mon[34512]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:03 smithi142 ceph-mon[34512]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 -- 172.21.15.193:0/37614454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 msgr2=0x7fb8d81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 --2- 172.21.15.193:0/37614454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d81087b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802a410 comp rx=0 tx=0).stop 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 -- 172.21.15.193:0/37614454 shutdown_connections 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 --2- 172.21.15.193:0/37614454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8108cf0 0x7fb8d810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 --2- 172.21.15.193:0/37614454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d81087b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 --2- 172.21.15.193:0/37614454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8102a60 0x7fb8d8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 -- 172.21.15.193:0/37614454 >> 172.21.15.193:0/37614454 conn(0x7fb8d8079730 msgr2=0x7fb8d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 -- 172.21.15.193:0/37614454 shutdown_connections 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.973+0000 7fb8df485640 1 -- 172.21.15.193:0/37614454 wait complete. 2024-09-21T03:12:03.979 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 Processor -- start 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 -- start start 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8102a60 0x7fb8d819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 0x7fb8d819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8112960 con 0x7fb8d8108cf0 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d81127e0 con 0x7fb8d8102a60 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8df485640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8112ae0 con 0x7fb8d81083a0 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:43468/0 (socket says 172.21.15.193:43468) 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 -- 172.21.15.193:0/1789739413 learned_addr learned my addr 172.21.15.193:0/1789739413 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dd221640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8102a60 0x7fb8d819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:03.980 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dda22640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 0x7fb8d819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 -- 172.21.15.193:0/1789739413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8102a60 msgr2=0x7fb8d819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8102a60 0x7fb8d819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 -- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 msgr2=0x7fb8d819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 0x7fb8d819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dca20640 1 -- 172.21.15.193:0/1789739413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d81a0310 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.974+0000 7fb8dda22640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 0x7fb8d819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8dca20640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8006010 tx=0x7fb8c8004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c803f030 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d81a05a0 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c8004490 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8035070 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.975+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a0ad0 con 0x7fb8d81083a0 2024-09-21T03:12:03.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.976+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d8102e70 con 0x7fb8d81083a0 2024-09-21T03:12:03.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.978+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb8c8042030 con 0x7fb8d81083a0 2024-09-21T03:12:03.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.979+0000 7fb8ce7fc640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 0x7fb8c40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:03.982 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.979+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5316+0+0 (secure 0 0 0) 0x7fb8c81070c0 con 0x7fb8d81083a0 2024-09-21T03:12:03.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.979+0000 7fb8dd221640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 0x7fb8c40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:03.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.980+0000 7fb8dd221640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 0x7fb8c40b8aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400abe0 tx=0x7fb8d400a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:03.983 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:03.982+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8d81a0ad0 con 0x7fb8d81083a0 2024-09-21T03:12:04.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:04.175+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb8d8063800 con 0x7fb8c40b6590 2024-09-21T03:12:04.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:04.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:04.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:04.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:12:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:04.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:04.677 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:12:04.207+0000 7fb87210f640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:12:04.677 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:12:04.216+0000 7fb86df0d640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:12:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:04.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:04.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:04.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:04.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:04.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:05.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi142 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] boot 2024-09-21T03:12:05.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:12:05.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi142 2024-09-21T03:12:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-21T03:12:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] boot 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:12:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi142 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:05.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: osd.7 [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] boot 2024-09-21T03:12:05.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:12:05.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:12:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:06 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:12:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:06 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:12:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:06 smithi193 ceph-mon[35224]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:06 smithi193 ceph-mon[35224]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:12:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:06 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:12:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:06 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:12:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:06 smithi083 ceph-mon[31219]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:06 smithi083 ceph-mon[31219]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:12:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:06 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:12:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:06 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:12:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:06 smithi142 ceph-mon[34512]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:12:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:06 smithi142 ceph-mon[34512]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:12:07.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/2126909435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"}]': finished 2024-09-21T03:12:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:12:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:07 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/2300110660' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/2126909435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"}]': finished 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:07 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/2300110660' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/2126909435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"} : dispatch 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558"}]': finished 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:07 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/2300110660' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:08 smithi193 ceph-mon[35224]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:08 smithi193 ceph-mon[35224]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:12:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:08 smithi083 ceph-mon[31219]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:08 smithi083 ceph-mon[31219]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:12:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:08 smithi142 ceph-mon[34512]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:08 smithi142 ceph-mon[34512]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:12:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:10.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:10 smithi193 ceph-mon[35224]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:10 smithi083 ceph-mon[31219]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:10 smithi142 ceph-mon[34512]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:12.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:12 smithi193 ceph-mon[35224]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T03:12:12.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:12 smithi142 ceph-mon[34512]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T03:12:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:12 smithi083 ceph-mon[31219]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T03:12:14.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:14 smithi193 ceph-mon[35224]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:12:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:14 smithi083 ceph-mon[31219]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:12:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:14 smithi142 ceph-mon[34512]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:12:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:16 smithi193 ceph-mon[35224]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:12:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:12:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:16 smithi083 ceph-mon[31219]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:12:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:12:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:16 smithi142 ceph-mon[34512]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:12:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:12:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:17 smithi193 ceph-mon[35224]: Deploying daemon osd.8 on smithi193 2024-09-21T03:12:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:17 smithi083 ceph-mon[31219]: Deploying daemon osd.8 on smithi193 2024-09-21T03:12:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:17 smithi142 ceph-mon[34512]: Deploying daemon osd.8 on smithi193 2024-09-21T03:12:18.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:18 smithi083 ceph-mon[31219]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:12:18.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:18 smithi193 ceph-mon[35224]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:12:18.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:18 smithi142 ceph-mon[34512]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:12:20.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:20 smithi193 ceph-mon[35224]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:12:20.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:20 smithi083 ceph-mon[31219]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:12:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:20 smithi142 ceph-mon[34512]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:12:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:22 smithi193 ceph-mon[35224]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:22 smithi083 ceph-mon[31219]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:22 smithi142 ceph-mon[34512]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:25.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.447+0000 7fb8ce7fc640 1 -- 172.21.15.193:0/1789739413 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8d8063800 con 0x7fb8c40b6590 2024-09-21T03:12:25.450 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-09-21T03:12:25.452 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 msgr2=0x7fb8c40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:25.452 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 0x7fb8c40b8aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400abe0 tx=0x7fb8d400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:12:25.452 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 msgr2=0x7fb8d819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8006010 tx=0x7fb8c8004060 comp rx=0 tx=0).stop 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 shutdown_connections 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb8c40b6590 0x7fb8c40b8aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d8108cf0 0x7fb8d819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8d81083a0 0x7fb8d819aa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 --2- 172.21.15.193:0/1789739413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8d8102a60 0x7fb8d819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 >> 172.21.15.193:0/1789739413 conn(0x7fb8d8079730 msgr2=0x7fb8d806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 shutdown_connections 2024-09-21T03:12:25.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:12:25.450+0000 7fb8df485640 1 -- 172.21.15.193:0/1789739413 wait complete. 2024-09-21T03:12:26.194 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:12:26.197 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-09-21T03:12:26.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:12:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:26.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:26 smithi193 ceph-mon[35224]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:26.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:26.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:26.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:26.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:26 smithi083 ceph-mon[31219]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:26.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:26.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:26.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:26 smithi142 ceph-mon[34512]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:27.814 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:12:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:12:27.711+0000 7fe74a45c640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:12:28.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:28 smithi083 ceph-mon[31219]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:28.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:28 smithi083 ceph-mon[31219]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:28.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:28 smithi083 ceph-mon[31219]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:28 smithi193 ceph-mon[35224]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:28 smithi193 ceph-mon[35224]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:28 smithi193 ceph-mon[35224]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:28 smithi142 ceph-mon[34512]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:28 smithi142 ceph-mon[34512]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:28 smithi142 ceph-mon[34512]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:12:29.219 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:12:29.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-09-21T03:12:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:29.564 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:12:29.265+0000 7fe7463de640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:12:29.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:12:29.275+0000 7fe7421dc640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:29 smithi193 ceph-mon[35224]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:29 smithi083 ceph-mon[31219]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: from='osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:12:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:29 smithi142 ceph-mon[34512]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 -- 172.21.15.83:0/514984905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c103b60 msgr2=0x7efd0c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 --2- 172.21.15.83:0/514984905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c103b60 0x7efd0c103f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009900 tx=0x7efcfc02a410 comp rx=0 tx=0).stop 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 -- 172.21.15.83:0/514984905 shutdown_connections 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 --2- 172.21.15.83:0/514984905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 --2- 172.21.15.83:0/514984905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c1094a0 0x7efd0c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 --2- 172.21.15.83:0/514984905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c103b60 0x7efd0c103f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 -- 172.21.15.83:0/514984905 >> 172.21.15.83:0/514984905 conn(0x7efd0c07b850 msgr2=0x7efd0c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 -- 172.21.15.83:0/514984905 shutdown_connections 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 -- 172.21.15.83:0/514984905 wait complete. 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.986+0000 7efd115d3640 1 Processor -- start 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 -- start start 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c103b60 0x7efd0c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c113b10 con 0x7efd0c1094a0 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c113990 con 0x7efd0c103b60 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd115d3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c113c90 con 0x7efd0c109df0 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38438/0 (socket says 172.21.15.83:38438) 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 -- 172.21.15.83:0/172408455 learned_addr learned my addr 172.21.15.83:0/172408455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0affd640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c103b60 0x7efd0c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 -- 172.21.15.83:0/172408455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 msgr2=0x7efd0c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 -- 172.21.15.83:0/172408455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c103b60 msgr2=0x7efd0c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c103b60 0x7efd0c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0a7fc640 1 -- 172.21.15.83:0/172408455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c1a0470 con 0x7efd0c1094a0 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.987+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efd0a7fc640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c970 tx=0x7efd0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00007bb0 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd00007d50 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0000f450 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c1a0760 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.988+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c1a6ec0 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.989+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c103f70 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.992+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7efd00004eb0 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.992+0000 7efcf3fff640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 0x7efcf40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.992+0000 7efd0affd640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 0x7efcf40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.992+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5764+0+0 (secure 0 0 0) 0x7efd000e6aa0 con 0x7efd0c1094a0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.993+0000 7efd0affd640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 0x7efcf40b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcfc031440 tx=0x7efcfc005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:29.996+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd00014030 con 0x7efd0c1094a0 2024-09-21T03:12:30.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:30.186+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efd0c063800 con 0x7efcf40b6560 2024-09-21T03:12:30.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi193 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:30.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:12:30.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi193 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:30.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:30.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:30.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:30.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:12:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T03:12:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi193 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:30.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:30.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] boot 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:31 smithi193 ceph-mon[35224]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] boot 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:31 smithi083 ceph-mon[31219]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: osd.8 [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] boot 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:12:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:31 smithi142 ceph-mon[34512]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:32 smithi083 ceph-mon[31219]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:12:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:32 smithi193 ceph-mon[35224]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:12:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:32 smithi142 ceph-mon[34512]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:12:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:12:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"} : dispatch 2024-09-21T03:12:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"}]': finished 2024-09-21T03:12:33.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:12:33.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:33.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:33 smithi083 ceph-mon[31219]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"} : dispatch 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"}]': finished 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:33 smithi193 ceph-mon[35224]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"} : dispatch 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1769514925' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6"}]': finished 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:33 smithi142 ceph-mon[34512]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:34.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:34 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/929268714' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:34 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/929268714' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:34 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/929268714' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:35 smithi193 ceph-mon[35224]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:35.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:35 smithi083 ceph-mon[31219]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:35 smithi142 ceph-mon[34512]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:38.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:38 smithi083 ceph-mon[31219]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:12:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:38 smithi193 ceph-mon[35224]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:12:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:38 smithi142 ceph-mon[34512]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:12:40.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:40 smithi083 ceph-mon[31219]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:12:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:40 smithi193 ceph-mon[35224]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:12:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:40 smithi142 ceph-mon[34512]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:12:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:42 smithi083 ceph-mon[31219]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:12:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:12:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:42.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:42 smithi142 ceph-mon[34512]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:12:42.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:12:42.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:42.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:42 smithi193 ceph-mon[35224]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:12:42.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:12:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:43.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:43 smithi083 ceph-mon[31219]: Deploying daemon osd.9 on smithi083 2024-09-21T03:12:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:43 smithi142 ceph-mon[34512]: Deploying daemon osd.9 on smithi083 2024-09-21T03:12:43.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:43 smithi193 ceph-mon[35224]: Deploying daemon osd.9 on smithi083 2024-09-21T03:12:44.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:44 smithi083 ceph-mon[31219]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:12:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:44 smithi193 ceph-mon[35224]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:12:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:44 smithi142 ceph-mon[34512]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:12:46.482 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:46 smithi083 ceph-mon[31219]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:46.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:46 smithi193 ceph-mon[35224]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:46 smithi142 ceph-mon[34512]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:48.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:48 smithi193 ceph-mon[35224]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:48 smithi083 ceph-mon[31219]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:48 smithi142 ceph-mon[34512]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:12:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:50 smithi083 ceph-mon[31219]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:50.461 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.461 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:50 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:50 smithi193 ceph-mon[35224]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:50 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:50 smithi142 ceph-mon[34512]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:50 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.973+0000 7efcf3fff640 1 -- 172.21.15.83:0/172408455 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd0c063800 con 0x7efcf40b6560 2024-09-21T03:12:50.976 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 msgr2=0x7efcf40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 0x7efcf40b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcfc031440 tx=0x7efcfc005e80 comp rx=0 tx=0).stop 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 msgr2=0x7efd0c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c970 tx=0x7efd0000ce30 comp rx=0 tx=0).stop 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 shutdown_connections 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efcf40b6560 0x7efcf40b8a70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd0c109df0 0x7efd0c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c1094a0 0x7efd0c19aaa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 --2- 172.21.15.83:0/172408455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd0c103b60 0x7efd0c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.976+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 >> 172.21.15.83:0/172408455 conn(0x7efd0c07b850 msgr2=0x7efd0c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.977+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 shutdown_connections 2024-09-21T03:12:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:12:50.977+0000 7efd115d3640 1 -- 172.21.15.83:0/172408455 wait complete. 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:51 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:51 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:51 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:51.731 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:12:51.733 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2024-09-21T03:12:51.734 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:12:51.989 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:12:52.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:52 smithi083 ceph-mon[31219]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:52.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:52 smithi142 ceph-mon[34512]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:52 smithi193 ceph-mon[35224]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:53.586 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:12:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:12:53.486+0000 7fa122e6c640 -1 osd.9 0 log_to_monitors true 2024-09-21T03:12:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:54 smithi083 ceph-mon[31219]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:54 smithi083 ceph-mon[31219]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:12:54.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:54 smithi193 ceph-mon[35224]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:54 smithi193 ceph-mon[35224]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:12:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:54 smithi142 ceph-mon[34512]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:54 smithi142 ceph-mon[34512]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:12:54.785 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-21T03:12:54.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-09-21T03:12:55.061 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:12:55.477 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:12:55.477 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.478 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:55 smithi142 ceph-mon[34512]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:55 smithi193 ceph-mon[35224]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:12:55.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 -- 172.21.15.142:0/1830909865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 msgr2=0x7fa47810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:55.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 --2- 172.21.15.142:0/1830909865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47810f540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa470009ca0 tx=0x7fa47002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 -- 172.21.15.142:0/1830909865 shutdown_connections 2024-09-21T03:12:55.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 --2- 172.21.15.142:0/1830909865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47810f540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 --2- 172.21.15.142:0/1830909865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa4781087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 --2- 172.21.15.142:0/1830909865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 0x7fa478102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 -- 172.21.15.142:0/1830909865 >> 172.21.15.142:0/1830909865 conn(0x7fa478079740 msgr2=0x7fa478079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.568+0000 7fa47dfed640 1 -- 172.21.15.142:0/1830909865 shutdown_connections 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.569+0000 7fa47dfed640 1 -- 172.21.15.142:0/1830909865 wait complete. 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.569+0000 7fa47dfed640 1 Processor -- start 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.569+0000 7fa47dfed640 1 -- start start 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 0x7fa47819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa47819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112980 con 0x7fa4781083c0 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112800 con 0x7fa478108d10 2024-09-21T03:12:55.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa47dfed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112b00 con 0x7fa478102a80 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa47819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43594/0 (socket says 172.21.15.142:43594) 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 -- 172.21.15.142:0/2892024816 learned_addr learned my addr 172.21.15.142:0/2892024816 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa4777fe640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 0x7fa47819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 -- 172.21.15.142:0/2892024816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 msgr2=0x7fa47819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 0x7fa47819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 -- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 msgr2=0x7fa47819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa47819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 -- 172.21.15.142:0/2892024816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781a0390 con 0x7fa478108d10 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa476ffd640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa47819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.570+0000 7fa477fff640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098d0 tx=0x7fa4700317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.571+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470003b70 con 0x7fa478108d10 2024-09-21T03:12:55.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.571+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa470003d10 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.571+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781a0620 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.571+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47002fa20 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.571+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a6de0 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.572+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa478103d00 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.574+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa47002fc20 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.575+0000 7fa474ff9640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 0x7fa4600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.575+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6196+0+0 (secure 0 0 0) 0x7fa470107f40 con 0x7fa478108d10 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.575+0000 7fa4777fe640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 0x7fa4600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.575+0000 7fa4777fe640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 0x7fa4600b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa468005fd0 tx=0x7fa46800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:55.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.578+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4700cf010 con 0x7fa478108d10 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:12:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-mon[31219]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-21T03:12:55.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:12:55.766+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa47810bb20 con 0x7fa4600b6560 2024-09-21T03:12:56.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:12:55.649+0000 7fa11edee640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:12:56.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:12:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:12:55.660+0000 7fa116be4640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:12:56.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi083 2024-09-21T03:12:56.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:56.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:56.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi083 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi083 2024-09-21T03:12:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 4272M 2024-09-21T03:12:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662]' entity='osd.9' 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:12:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:12:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:12:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:12:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:57.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] boot 2024-09-21T03:12:57.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:12:57.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] boot 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:12:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: osd.9 [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] boot 2024-09-21T03:12:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:12:57.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/1869357075' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"}]': finished 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:12:58.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/1869357075' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"}]': finished 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:12:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/1869357075' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"} : dispatch 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60a1b3a-0053-460c-848e-aa177514017d"}]': finished 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:12:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:12:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:12:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.142:0/3248291947' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:12:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.142:0/3248291947' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:12:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:12:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.142:0/3248291947' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:13:00.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:00 smithi193 ceph-mon[35224]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:00.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:00 smithi083 ceph-mon[31219]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:00 smithi142 ceph-mon[34512]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:01 smithi193 ceph-mon[35224]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:01 smithi083 ceph-mon[31219]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:01 smithi142 ceph-mon[34512]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:04.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:04 smithi142 ceph-mon[34512]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:04.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:04 smithi193 ceph-mon[35224]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:04 smithi083 ceph-mon[31219]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:06.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:06 smithi142 ceph-mon[34512]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:06.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:06 smithi193 ceph-mon[35224]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:06 smithi083 ceph-mon[31219]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:07.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:13:07.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:07.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:13:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:13:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:08.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:08 smithi142 ceph-mon[34512]: Deploying daemon osd.10 on smithi142 2024-09-21T03:13:08.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:08 smithi142 ceph-mon[34512]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:08 smithi193 ceph-mon[35224]: Deploying daemon osd.10 on smithi142 2024-09-21T03:13:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:08 smithi193 ceph-mon[35224]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:08 smithi083 ceph-mon[31219]: Deploying daemon osd.10 on smithi142 2024-09-21T03:13:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:08 smithi083 ceph-mon[31219]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:10 smithi142 ceph-mon[34512]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:10.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:10 smithi193 ceph-mon[35224]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:10 smithi083 ceph-mon[31219]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:12.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:12 smithi142 ceph-mon[34512]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:12 smithi193 ceph-mon[35224]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:12 smithi083 ceph-mon[31219]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:13.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:13.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:13.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:13 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:13.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:13 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:13 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:14.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:14 smithi193 ceph-mon[35224]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:14.614 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:14 smithi142 ceph-mon[34512]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:14 smithi083 ceph-mon[31219]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:16.363 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:16.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:16.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:16.364 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:16 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.406+0000 7fa474ff9640 1 -- 172.21.15.142:0/2892024816 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa47810bb20 con 0x7fa4600b6560 2024-09-21T03:13:16.408 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 msgr2=0x7fa4600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 0x7fa4600b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa468005fd0 tx=0x7fa46800b390 comp rx=0 tx=0).stop 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 msgr2=0x7fa47819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098d0 tx=0x7fa4700317b0 comp rx=0 tx=0).stop 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 shutdown_connections 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4600b6560 0x7fa4600b8a70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478108d10 0x7fa47819fc60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4781083c0 0x7fa47819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 --2- 172.21.15.142:0/2892024816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa478102a80 0x7fa47819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.408+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 >> 172.21.15.142:0/2892024816 conn(0x7fa478079740 msgr2=0x7fa47806bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.409+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 shutdown_connections 2024-09-21T03:13:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-21T03:13:16.409+0000 7fa47dfed640 1 -- 172.21.15.142:0/2892024816 wait complete. 2024-09-21T03:13:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:16 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:16 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.110 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:13:17.112 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-21T03:13:17.112 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:13:17.366 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:17.366 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.366 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.368 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:13:17.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:17.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:18.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:18 smithi083 ceph-mon[31219]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:18.539 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:18 smithi193 ceph-mon[35224]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:18.552 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:18 smithi142 ceph-mon[34512]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:19.347 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:19 smithi193 ceph-mon[35224]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:19.348 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:19 smithi193 ceph-mon[35224]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:19 smithi142 ceph-mon[34512]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:19 smithi142 ceph-mon[34512]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:19.427 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:13:19 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:13:19.055+0000 7f2cb22dd640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:13:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:19 smithi083 ceph-mon[31219]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:19 smithi083 ceph-mon[31219]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:13:20.097 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T03:13:20.123 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-21T03:13:20.368 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.489 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:20.490 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:20.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:20.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:13:20.245+0000 7f2cae25f640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:13:20.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:13:20.259+0000 7f2caa05d640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:13:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:20.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:20.856 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.839+0000 7f55d41e0640 1 -- 172.21.15.193:0/4094256702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 msgr2=0x7f55cc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.839+0000 7f55d41e0640 1 --2- 172.21.15.193:0/4094256702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc10f550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009ca0 tx=0x7f55c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.839+0000 7f55d41e0640 1 -- 172.21.15.193:0/4094256702 shutdown_connections 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.839+0000 7f55d41e0640 1 --2- 172.21.15.193:0/4094256702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc10f550 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/4094256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 0x7f55cc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/4094256702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 0x7f55cc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/4094256702 >> 172.21.15.193:0/4094256702 conn(0x7f55cc079750 msgr2=0x7f55cc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/4094256702 shutdown_connections 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/4094256702 wait complete. 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 Processor -- start 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.840+0000 7f55d41e0640 1 -- start start 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 0x7f55cc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 0x7f55cc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:20.857 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc112990 con 0x7f55cc102a90 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc112810 con 0x7f55cc1083d0 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc112b10 con 0x7f55cc108d20 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:60194/0 (socket says 172.21.15.193:60194) 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 -- 172.21.15.193:0/326901983 learned_addr learned my addr 172.21.15.193:0/326901983 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d1f7c640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 0x7f55cc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d177b640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 0x7f55cc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 -- 172.21.15.193:0/326901983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 msgr2=0x7f55cc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 0x7f55cc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 -- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 msgr2=0x7f55cc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 0x7f55cc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:20.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 -- 172.21.15.193:0/326901983 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc1a0400 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d277d640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005cc0 tx=0x7f55c402f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4003bf0 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c4003d90 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc1a0690 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.841+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c402fbc0 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.842+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a6e50 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.842+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc103d10 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.844+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f55c402fd60 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.845+0000 7f55c2ffd640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 0x7f55b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.845+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6629+0+0 (secure 0 0 0) 0x7f55c4108340 con 0x7f55cc108d20 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.845+0000 7f55d1f7c640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 0x7f55b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.846+0000 7f55d1f7c640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 0x7f55b80b8a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55c80058c0 tx=0x7f55c800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:20.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:20.848+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55c40cf260 con 0x7f55cc108d20 2024-09-21T03:13:21.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:21.035+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55cc10bb80 con 0x7f55b80b6560 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi142 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:13:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:21 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi142 2024-09-21T03:13:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-21T03:13:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:13:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:13:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:13:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:13:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:21 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi142 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:13:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:13:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:21 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] boot 2024-09-21T03:13:22.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:13:22.507 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:13:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:13:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:22.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] boot 2024-09-21T03:13:22.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:13:22.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: osd.10 [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] boot 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:13:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:13:23.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:23 smithi193 ceph-mon[35224]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:13:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:23 smithi083 ceph-mon[31219]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:13:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:23 smithi142 ceph-mon[34512]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:13:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/2268227270' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"}]': finished 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:24 smithi193 ceph-mon[35224]: from='client.? 172.21.15.193:0/430865899' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/2268227270' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"}]': finished 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:24 smithi083 ceph-mon[31219]: from='client.? 172.21.15.193:0/430865899' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:13:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T03:13:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/2268227270' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"} : dispatch 2024-09-21T03:13:24.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "920f94f1-463c-4ffd-bd02-2470631a6746"}]': finished 2024-09-21T03:13:24.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:13:24.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:24.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:24 smithi142 ceph-mon[34512]: from='client.? 172.21.15.193:0/430865899' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:13:25.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:25 smithi193 ceph-mon[35224]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:25 smithi083 ceph-mon[31219]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:25 smithi142 ceph-mon[34512]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:28.513 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:28 smithi193 ceph-mon[35224]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T03:13:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:28 smithi083 ceph-mon[31219]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T03:13:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:28 smithi142 ceph-mon[34512]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T03:13:30.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:30 smithi193 ceph-mon[35224]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T03:13:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:30 smithi083 ceph-mon[31219]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T03:13:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:30 smithi142 ceph-mon[34512]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-21T03:13:32.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:32 smithi193 ceph-mon[35224]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:13:32.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:32 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:13:32.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:32 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:32 smithi083 ceph-mon[31219]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:13:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:32 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:13:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:32 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:32 smithi142 ceph-mon[34512]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:13:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:32 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:13:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:32 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:33.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:33 smithi193 ceph-mon[35224]: Deploying daemon osd.11 on smithi193 2024-09-21T03:13:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:33 smithi083 ceph-mon[31219]: Deploying daemon osd.11 on smithi193 2024-09-21T03:13:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:33 smithi142 ceph-mon[34512]: Deploying daemon osd.11 on smithi193 2024-09-21T03:13:34.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:34 smithi193 ceph-mon[35224]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:13:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:34 smithi083 ceph-mon[31219]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:13:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:34 smithi142 ceph-mon[34512]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:13:36.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:36 smithi193 ceph-mon[35224]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:13:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:36 smithi083 ceph-mon[31219]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:13:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:36 smithi142 ceph-mon[34512]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:13:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:38 smithi193 ceph-mon[35224]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:13:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:38 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:38 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:38 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:38 smithi083 ceph-mon[31219]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:13:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:38 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:38 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:38 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:38 smithi142 ceph-mon[34512]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:13:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:38 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:38 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:38 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:40 smithi193 ceph-mon[35224]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:40 smithi083 ceph-mon[31219]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:40 smithi142 ceph-mon[34512]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:41.838 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.837+0000 7f55c2ffd640 1 -- 172.21.15.193:0/326901983 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55cc10bb80 con 0x7f55b80b6560 2024-09-21T03:13:41.839 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.839+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 msgr2=0x7f55b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.839+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 0x7f55b80b8a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55c80058c0 tx=0x7f55c800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.839+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 msgr2=0x7f55cc19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.839+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005cc0 tx=0x7f55c402f380 comp rx=0 tx=0).stop 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 shutdown_connections 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55b80b6560 0x7f55b80b8a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:41.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55cc108d20 0x7f55cc19fcd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:41.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55cc1083d0 0x7f55cc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:41.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 --2- 172.21.15.193:0/326901983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55cc102a90 0x7f55cc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:41.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 >> 172.21.15.193:0/326901983 conn(0x7f55cc079750 msgr2=0x7f55cc101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:41.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 shutdown_connections 2024-09-21T03:13:41.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:41.840+0000 7f55d41e0640 1 -- 172.21.15.193:0/326901983 wait complete. 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:42.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:42.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:42 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:42.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:42 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:42.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:42.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:42.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:42.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:42 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:42.566 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:13:42.568 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:13:42.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:13:42.831 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 -- 172.21.15.83:0/2589918791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc1083d0 msgr2=0x7f17fc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 --2- 172.21.15.83:0/2589918791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc1083d0 0x7f17fc1087e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009900 tx=0x7f17f002a410 comp rx=0 tx=0).stop 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 -- 172.21.15.83:0/2589918791 shutdown_connections 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 --2- 172.21.15.83:0/2589918791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc108d20 0x7f17fc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 --2- 172.21.15.83:0/2589918791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc1083d0 0x7f17fc1087e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 --2- 172.21.15.83:0/2589918791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 -- 172.21.15.83:0/2589918791 >> 172.21.15.83:0/2589918791 conn(0x7f17fc079790 msgr2=0x7f17fc079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 -- 172.21.15.83:0/2589918791 shutdown_connections 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.293+0000 7f180113e640 1 -- 172.21.15.83:0/2589918791 wait complete. 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 Processor -- start 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 -- start start 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc1083d0 0x7f17fc110570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 0x7f17fc110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc113b00 con 0x7f17fc108d20 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc113980 con 0x7f17fc102ae0 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.294+0000 7f180113e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc113c80 con 0x7f17fc1083d0 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 0x7f17fc110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:43.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:43434/0 (socket says 172.21.15.83:43434) 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 -- 172.21.15.83:0/2669942095 learned_addr learned my addr 172.21.15.83:0/2669942095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 0x7f17fc110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38878/0 (socket says 172.21.15.83:38878) 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fa59c640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc1083d0 0x7f17fc110570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 -- 172.21.15.83:0/2669942095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc1083d0 msgr2=0x7f17fc110570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc1083d0 0x7f17fc110570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 -- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 msgr2=0x7f17fc110ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 0x7f17fc110ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 -- 172.21.15.83:0/2669942095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc1a8810 con 0x7f17fc102ae0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17fad9d640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9f0 tx=0x7f17ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00cd80 con 0x7f17fc102ae0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17ec0108e0 con 0x7f17fc102ae0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1a8b00 con 0x7f17fc102ae0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec018980 con 0x7f17fc102ae0 2024-09-21T03:13:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.295+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a9010 con 0x7f17fc102ae0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.296+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc102ef0 con 0x7f17fc102ae0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.299+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f17ec010430 con 0x7f17fc102ae0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.299+0000 7f17e3fff640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 0x7f17e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.300+0000 7f17fa59c640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 0x7f17e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.300+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7024+0+0 (secure 0 0 0) 0x7f17ec0e7680 con 0x7f17fc102ae0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.300+0000 7f17fa59c640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 0x7f17e40b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009230 tx=0x7f17f0031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:43.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.302+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17ec0b2780 con 0x7f17fc102ae0 2024-09-21T03:13:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.490+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f17fc02e640 con 0x7f17fc102ae0 2024-09-21T03:13:43.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.490+0000 7f17e3fff640 1 -- 172.21.15.83:0/2669942095 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f17ec0b76c0 con 0x7f17fc102ae0 2024-09-21T03:13:43.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 msgr2=0x7f17e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 0x7f17e40b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009230 tx=0x7f17f0031400 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 msgr2=0x7f17fc110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e9f0 tx=0x7f17ec00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 shutdown_connections 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f17e40b6560 0x7f17e40b8a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17fc108d20 0x7f17fc110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17fc1083d0 0x7f17fc110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 --2- 172.21.15.83:0/2669942095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc102ae0 0x7f17fc110030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 >> 172.21.15.83:0/2669942095 conn(0x7f17fc079790 msgr2=0x7f17fc1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 shutdown_connections 2024-09-21T03:13:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:43.493+0000 7f180113e640 1 -- 172.21.15.83:0/2669942095 wait complete. 2024-09-21T03:13:44.236 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726888401,"num_in_osds":12,"osd_in_since":1726888403,"num_remapped_pgs":0} 2024-09-21T03:13:44.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:44 smithi193 ceph-mon[35224]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:44.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:44 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2669942095' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:44 smithi083 ceph-mon[31219]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:44 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2669942095' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:44 smithi142 ceph-mon[34512]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:44 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2669942095' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:44.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:13:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:13:44.544+0000 7fd1f8331640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:13:45.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:13:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:45 smithi193 ceph-mon[35224]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:45 smithi193 ceph-mon[35224]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:45.609 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:45 smithi083 ceph-mon[31219]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:45.609 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:45 smithi083 ceph-mon[31219]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:45 smithi142 ceph-mon[34512]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:45 smithi142 ceph-mon[34512]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 -- 172.21.15.83:0/2230717916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148108400 msgr2=0x7f2148108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 --2- 172.21.15.83:0/2230717916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148108400 0x7f2148108810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2134009900 tx=0x7f213402a410 comp rx=0 tx=0).stop 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 -- 172.21.15.83:0/2230717916 shutdown_connections 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 --2- 172.21.15.83:0/2230717916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108d50 0x7f214810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 --2- 172.21.15.83:0/2230717916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148108400 0x7f2148108810 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 --2- 172.21.15.83:0/2230717916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148103190 0x7f21481035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 -- 172.21.15.83:0/2230717916 >> 172.21.15.83:0/2230717916 conn(0x7f2148079f90 msgr2=0x7f214807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 -- 172.21.15.83:0/2230717916 shutdown_connections 2024-09-21T03:13:46.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.008+0000 7f214eaeb640 1 -- 172.21.15.83:0/2230717916 wait complete. 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 Processor -- start 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 -- start start 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 0x7f21480ff330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148108d50 0x7f21480ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2148112a30 con 0x7f2148103190 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21481128b0 con 0x7f2148108400 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214eaeb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2148112bb0 con 0x7f2148108d50 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214c887640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214c887640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38896/0 (socket says 172.21.15.83:38896) 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214c887640 1 -- 172.21.15.83:0/668583033 learned_addr learned my addr 172.21.15.83:0/668583033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f2147fff640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 0x7f21480ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:46.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.009+0000 7f214d088640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148108d50 0x7f21480ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 -- 172.21.15.83:0/668583033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148108d50 msgr2=0x7f21480ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148108d50 0x7f21480ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 -- 172.21.15.83:0/668583033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 msgr2=0x7f21480ff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 0x7f21480ff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 -- 172.21.15.83:0/668583033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21481a88d0 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f2147fff640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 0x7f21480ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214c887640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b4a0 tx=0x7f213c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c014070 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21481a8bc0 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f213c010ac0 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c017c30 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.010+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21481a9120 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.011+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21481035a0 con 0x7f2148103190 2024-09-21T03:13:46.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.012+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f213c017070 con 0x7f2148103190 2024-09-21T03:13:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.013+0000 7f2145ffb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 0x7f21380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.013+0000 7f2147fff640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 0x7f21380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.013+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7046+0+0 (secure 0 0 0) 0x7f213c0e6550 con 0x7f2148103190 2024-09-21T03:13:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.014+0000 7f2147fff640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 0x7f21380b8a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2134009230 tx=0x7f2134000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.017+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f213c0ad2d0 con 0x7f2148103190 2024-09-21T03:13:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.196+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f214802e640 con 0x7f2148103190 2024-09-21T03:13:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.197+0000 7f2145ffb640 1 -- 172.21.15.83:0/668583033 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f213c0b16b0 con 0x7f2148103190 2024-09-21T03:13:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:46.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 msgr2=0x7f21380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:46.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 0x7f21380b8a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2134009230 tx=0x7f2134000bc0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 msgr2=0x7f21480fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b4a0 tx=0x7f213c00b960 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 shutdown_connections 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f21380b6560 0x7f21380b8a70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2148108d50 0x7f21480ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148108400 0x7f21480ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 --2- 172.21.15.83:0/668583033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2148103190 0x7f21480fedf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 >> 172.21.15.83:0/668583033 conn(0x7f2148079f90 msgr2=0x7f21481065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 shutdown_connections 2024-09-21T03:13:46.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:46.199+0000 7f214eaeb640 1 -- 172.21.15.83:0/668583033 wait complete. 2024-09-21T03:13:46.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:13:46.226+0000 7fd1f42b3640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:13:46.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:13:46.239+0000 7fd1f00b1640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:46 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/668583033' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.573 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:46.573 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:46.573 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.573 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:46 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/668583033' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:13:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:13:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:46.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:46 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/668583033' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:46.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726888401,"num_in_osds":12,"osd_in_since":1726888403,"num_remapped_pgs":0} 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: Detected new or changed devices on smithi193 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:47 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: Detected new or changed devices on smithi193 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:47 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: Detected new or changed devices on smithi193 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:47 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:47.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:13:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: purged_snaps scrub starts 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: purged_snaps scrub ok 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] boot 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:13:48.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:48 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:48.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: purged_snaps scrub starts 2024-09-21T03:13:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: purged_snaps scrub ok 2024-09-21T03:13:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] boot 2024-09-21T03:13:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:13:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:48 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 -- 172.21.15.83:0/2516694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850103b60 msgr2=0x7f1850103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 --2- 172.21.15.83:0/2516694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850103b60 0x7f1850103f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1844009900 tx=0x7f184402a410 comp rx=0 tx=0).stop 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 -- 172.21.15.83:0/2516694751 shutdown_connections 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 --2- 172.21.15.83:0/2516694751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850109df0 0x7f1850110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 --2- 172.21.15.83:0/2516694751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18501094a0 0x7f18501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 --2- 172.21.15.83:0/2516694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850103b60 0x7f1850103f70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 -- 172.21.15.83:0/2516694751 >> 172.21.15.83:0/2516694751 conn(0x7f185007b850 msgr2=0x7f185007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 -- 172.21.15.83:0/2516694751 shutdown_connections 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.598+0000 7f18570e4640 1 -- 172.21.15.83:0/2516694751 wait complete. 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 Processor -- start 2024-09-21T03:13:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 -- start start 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1850103b60 0x7f185019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18501094a0 0x7f185019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1850113af0 con 0x7f1850109df0 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1850113970 con 0x7f1850103b60 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.599+0000 7f18570e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1850113c70 con 0x7f18501094a0 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38914/0 (socket says 172.21.15.83:38914) 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 -- 172.21.15.83:0/4142684847 learned_addr learned my addr 172.21.15.83:0/4142684847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f184ffff640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18501094a0 0x7f185019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1854e80640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1850103b60 0x7f185019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 -- 172.21.15.83:0/4142684847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18501094a0 msgr2=0x7f185019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18501094a0 0x7f185019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 -- 172.21.15.83:0/4142684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1850103b60 msgr2=0x7f185019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1850103b60 0x7f185019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 -- 172.21.15.83:0/4142684847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501a03b0 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f1855681640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f184800cce0 tx=0x7f184800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800f450 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a06a0 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184800faa0 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.600+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848018b70 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.601+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a6ec0 con 0x7f1850109df0 2024-09-21T03:13:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.601+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850103f70 con 0x7f1850109df0 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.603+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1848018d10 con 0x7f1850109df0 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.604+0000 7f184dffb640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 0x7f18400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.604+0000 7f1854e80640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 0x7f18400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.604+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f18480e7550 con 0x7f1850109df0 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.604+0000 7f1854e80640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 0x7f18400b8aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1844031440 tx=0x7f1844005d10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.607+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1848014030 con 0x7f1850109df0 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: purged_snaps scrub starts 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: purged_snaps scrub ok 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: osd.11 [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] boot 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:13:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:48 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:13:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.787+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f185010bc20 con 0x7f1850109df0 2024-09-21T03:13:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.788+0000 7f184dffb640 1 -- 172.21.15.83:0/4142684847 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f18480ae1e0 con 0x7f1850109df0 2024-09-21T03:13:48.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 msgr2=0x7f18400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 0x7f18400b8aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1844031440 tx=0x7f1844005d10 comp rx=0 tx=0).stop 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 msgr2=0x7f185019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f184800cce0 tx=0x7f184800a620 comp rx=0 tx=0).stop 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 shutdown_connections 2024-09-21T03:13:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18400b6590 0x7f18400b8aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1850109df0 0x7f185019fd40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18501094a0 0x7f185019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 --2- 172.21.15.83:0/4142684847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1850103b60 0x7f185019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.790+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 >> 172.21.15.83:0/4142684847 conn(0x7f185007b850 msgr2=0x7f1850107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.791+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 shutdown_connections 2024-09-21T03:13:48.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:48.791+0000 7f18570e4640 1 -- 172.21.15.83:0/4142684847 wait complete. 2024-09-21T03:13:49.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:49 smithi083 ceph-mon[31219]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:13:49.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:49 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4142684847' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:49.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:49 smithi193 ceph-mon[35224]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:13:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:49 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4142684847' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:49.580 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726888427,"num_in_osds":12,"osd_in_since":1726888403,"num_remapped_pgs":0} 2024-09-21T03:13:49.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:13:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:49 smithi142 ceph-mon[34512]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:13:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:49 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4142684847' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:13:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:50.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4093927204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe4108d10 msgr2=0x7f2fe410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4093927204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe4108d10 0x7f2fe410f540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009900 tx=0x7f2fdc02a410 comp rx=0 tx=0).stop 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4093927204 shutdown_connections 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4093927204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe4108d10 0x7f2fe410f540 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4093927204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe41083c0 0x7f2fe41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4093927204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4102a80 0x7f2fe4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4093927204 >> 172.21.15.83:0/4093927204 conn(0x7f2fe4079740 msgr2=0x7f2fe4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4093927204 shutdown_connections 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.317+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4093927204 wait complete. 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 Processor -- start 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 -- start start 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 0x7f2fe419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:50.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4108d10 0x7f2fe419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4112980 con 0x7f2fe41083c0 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4112800 con 0x7f2fe4102a80 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe8dd0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4112b00 con 0x7f2fe4108d10 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54950/0 (socket says 172.21.15.83:54950) 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 -- 172.21.15.83:0/4292841790 learned_addr learned my addr 172.21.15.83:0/4292841790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe259c640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 0x7f2fe419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 -- 172.21.15.83:0/4292841790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4108d10 msgr2=0x7f2fe419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4108d10 0x7f2fe419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 -- 172.21.15.83:0/4292841790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 msgr2=0x7f2fe419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.318+0000 7f2fe1d9b640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 0x7f2fe419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fe1d9b640 1 -- 172.21.15.83:0/4292841790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe41a64c0 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fe259c640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 0x7f2fe419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fe1d9b640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800eb90 tx=0x7f2fd800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8010630 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd800ee90 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8018ac0 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe41a67b0 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.319+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe41a6cf0 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.320+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4103d00 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.321+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2fd8018c60 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.322+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 0x7f2fcc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.322+0000 7f2fe259c640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 0x7f2fcc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.322+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2fd80e7640 con 0x7f2fe41083c0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.323+0000 7f2fe259c640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 0x7f2fcc0b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40058c0 tx=0x7f2fd4005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:50.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.326+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fd80ae2a0 con 0x7f2fe41083c0 2024-09-21T03:13:50.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.503+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fe402e640 con 0x7f2fe41083c0 2024-09-21T03:13:50.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:50 smithi083 ceph-mon[31219]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.504+0000 7f2fd37fe640 1 -- 172.21.15.83:0/4292841790 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18806 (secure 0 0 0) 0x7f2fd80b2680 con 0x7f2fe41083c0 2024-09-21T03:13:50.506 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:50.506 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:13:48.230472+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.506+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 msgr2=0x7f2fcc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 0x7f2fcc0b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40058c0 tx=0x7f2fd4005d50 comp rx=0 tx=0).stop 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 msgr2=0x7f2fe419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800eb90 tx=0x7f2fd800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 shutdown_connections 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fcc0b6590 0x7f2fcc0b8aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4108d10 0x7f2fe419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe41083c0 0x7f2fe419b770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 --2- 172.21.15.83:0/4292841790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe4102a80 0x7f2fe419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 >> 172.21.15.83:0/4292841790 conn(0x7f2fe4079740 msgr2=0x7f2fe4104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 shutdown_connections 2024-09-21T03:13:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:50.507+0000 7f2fe8dd0640 1 -- 172.21.15.83:0/4292841790 wait complete. 2024-09-21T03:13:50.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:50 smithi193 ceph-mon[35224]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:50 smithi142 ceph-mon[34512]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:51.247 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:09:57.414337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:13:51.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:13:51.491 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:51.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:51 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4292841790' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:13:51.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:51 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4292841790' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:13:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:51 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4292841790' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:13:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- 172.21.15.83:0/1781412087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c109df0 msgr2=0x7f602c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 --2- 172.21.15.83:0/1781412087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c109df0 0x7f602c110620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6024009ca0 tx=0x7f602402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- 172.21.15.83:0/1781412087 shutdown_connections 2024-09-21T03:13:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 --2- 172.21.15.83:0/1781412087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c109df0 0x7f602c110620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 --2- 172.21.15.83:0/1781412087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c1094a0 0x7f602c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 --2- 172.21.15.83:0/1781412087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 0x7f602c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- 172.21.15.83:0/1781412087 >> 172.21.15.83:0/1781412087 conn(0x7f602c07b850 msgr2=0x7f602c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- 172.21.15.83:0/1781412087 shutdown_connections 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- 172.21.15.83:0/1781412087 wait complete. 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 Processor -- start 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.961+0000 7f6032702640 1 -- start start 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 0x7f602c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c109df0 0x7f602c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f602c113ab0 con 0x7f602c1094a0 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f602c113930 con 0x7f602c103b60 2024-09-21T03:13:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6032702640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f602c113c30 con 0x7f602c109df0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54960/0 (socket says 172.21.15.83:54960) 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 -- 172.21.15.83:0/3587423638 learned_addr learned my addr 172.21.15.83:0/3587423638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602bfff640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 0x7f602c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f6030c9f640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c109df0 0x7f602c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 -- 172.21.15.83:0/3587423638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c109df0 msgr2=0x7f602c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c109df0 0x7f602c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 -- 172.21.15.83:0/3587423638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 msgr2=0x7f602c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 0x7f602c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 -- 172.21.15.83:0/3587423638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c1a03b0 con 0x7f602c1094a0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.962+0000 7f602b7fe640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f602000d950 tx=0x7f602000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.963+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020017070 con 0x7f602c1094a0 2024-09-21T03:13:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.963+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6020009c30 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.963+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c1a06a0 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.963+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200047a0 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.963+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c1a6e00 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.964+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c103f70 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.965+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6020004020 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.966+0000 7f60297fa640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 0x7f60140b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.966+0000 7f602bfff640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 0x7f60140b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.966+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f60200e6fe0 con 0x7f602c1094a0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.967+0000 7f602bfff640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 0x7f60140b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f601c0058c0 tx=0x7f601c00a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:51.970+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6020020020 con 0x7f602c1094a0 2024-09-21T03:13:52.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.149+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f602c02e640 con 0x7f602c1094a0 2024-09-21T03:13:52.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.150+0000 7f60297fa640 1 -- 172.21.15.83:0/3587423638 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f60200adca0 con 0x7f602c1094a0 2024-09-21T03:13:52.151 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-09-21T03:13:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 msgr2=0x7f60140b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 0x7f60140b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f601c0058c0 tx=0x7f601c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 msgr2=0x7f602c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f602000d950 tx=0x7f602000de10 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 shutdown_connections 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60140b65e0 0x7f60140b8af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f602c109df0 0x7f602c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c1094a0 0x7f602c19a9e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 --2- 172.21.15.83:0/3587423638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f602c103b60 0x7f602c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 >> 172.21.15.83:0/3587423638 conn(0x7f602c07b850 msgr2=0x7f602c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 shutdown_connections 2024-09-21T03:13:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:52.152+0000 7f6032702640 1 -- 172.21.15.83:0/3587423638 wait complete. 2024-09-21T03:13:52.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:52 smithi083 ceph-mon[31219]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:52.459 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:52 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3587423638' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:13:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:52 smithi193 ceph-mon[35224]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:52 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3587423638' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:13:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:52 smithi142 ceph-mon[34512]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:52 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3587423638' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.b on smithi142 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.e on smithi142 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.h on smithi142 2024-09-21T03:13:52.949 INFO:tasks.cephadm:Adding mds.k on smithi142 2024-09-21T03:13:52.950 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-09-21T03:13:52.950 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-09-21T03:13:52.950 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-09-21T03:13:52.950 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-09-21T03:13:52.950 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-09-21T03:13:53.209 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:13:53.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 -- 172.21.15.193:0/2777598442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 msgr2=0x7f74a0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:53.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2777598442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a0110630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7498009ca0 tx=0x7f749802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 -- 172.21.15.193:0/2777598442 shutdown_connections 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2777598442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a0110630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2777598442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2777598442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 0x7f74a0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 -- 172.21.15.193:0/2777598442 >> 172.21.15.193:0/2777598442 conn(0x7f74a00ff5e0 msgr2=0x7f74a0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 -- 172.21.15.193:0/2777598442 shutdown_connections 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.727+0000 7f74a6e99640 1 -- 172.21.15.193:0/2777598442 wait complete. 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 Processor -- start 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 -- start start 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 0x7f74a019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:53.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113ac0 con 0x7f74a01094b0 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113940 con 0x7f74a0103b70 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.728+0000 7f74a6e99640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113c40 con 0x7f74a0109e00 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53324/0 (socket says 172.21.15.193:53324) 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 -- 172.21.15.193:0/2118242723 learned_addr learned my addr 172.21.15.193:0/2118242723 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a4c35640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 0x7f74a019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f749ffff640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 -- 172.21.15.193:0/2118242723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 msgr2=0x7f74a019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 0x7f74a019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 -- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 msgr2=0x7f74a019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:53.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 -- 172.21.15.193:0/2118242723 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a03c0 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f749ffff640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a5436640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7498005cc0 tx=0x7f749802f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498003bf0 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7498003d90 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a0650 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.729+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749802fbc0 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.730+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a6e10 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.731+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a0103f80 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.732+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f749802fd60 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.733+0000 7f749dffb640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 0x7f74880b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.733+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f7498108440 con 0x7f74a0109e00 2024-09-21T03:13:53.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.733+0000 7f74a4c35640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 0x7f74880b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:53.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.734+0000 7f74a4c35640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 0x7f74880b8af0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7490005fd0 tx=0x7f749000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:53.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.737+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74980cf0a0 con 0x7f74a0109e00 2024-09-21T03:13:53.931 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.929+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7f74a010bc50 con 0x7f74880b65e0 2024-09-21T03:13:53.939 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.938+0000 7f749dffb640 1 -- 172.21.15.193:0/2118242723 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f74a010bc50 con 0x7f74880b65e0 2024-09-21T03:13:53.940 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.940+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 msgr2=0x7f74880b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.940+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 0x7f74880b8af0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7490005fd0 tx=0x7f749000b390 comp rx=0 tx=0).stop 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.940+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 msgr2=0x7f74a019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.940+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7498005cc0 tx=0x7f749802f380 comp rx=0 tx=0).stop 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 shutdown_connections 2024-09-21T03:13:53.942 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74880b65e0 0x7f74880b8af0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a0109e00 0x7f74a019fc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a01094b0 0x7f74a019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 --2- 172.21.15.193:0/2118242723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74a0103b70 0x7f74a019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 >> 172.21.15.193:0/2118242723 conn(0x7f74a00ff5e0 msgr2=0x7f74a0106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 shutdown_connections 2024-09-21T03:13:53.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T03:13:53.941+0000 7f74a6e99640 1 -- 172.21.15.193:0/2118242723 wait complete. 2024-09-21T03:13:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:54 smithi193 ceph-mon[35224]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:54 smithi083 ceph-mon[31219]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:54.650 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:13:54.653 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:13:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:54 smithi142 ceph-mon[34512]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:13:54.696 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:13:54.739 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:13:54.782 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:13:54.784 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:13:54.828 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:13:54.871 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:13:54.913 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:13:54.916 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:13:54.959 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:13:55.002 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:13:55.045 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:13:55.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:13:55.287 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:55.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:55.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2488276639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 msgr2=0x7f6aa810f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2488276639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa810f610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009ca0 tx=0x7f6aa002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2488276639 shutdown_connections 2024-09-21T03:13:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2488276639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa810f610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2488276639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa8105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2488276639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2488276639 >> 172.21.15.83:0/2488276639 conn(0x7f6aa8079ec0 msgr2=0x7f6aa807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2488276639 shutdown_connections 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.762+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2488276639 wait complete. 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 Processor -- start 2024-09-21T03:13:55.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 -- start start 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8112ab0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8112930 con 0x7f6aa81031d0 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aaf6d1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8112c30 con 0x7f6aa8105250 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54990/0 (socket says 172.21.15.83:54990) 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 -- 172.21.15.83:0/2122438513 learned_addr learned my addr 172.21.15.83:0/2122438513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aad46d640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aacc6c640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 -- 172.21.15.83:0/2122438513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 msgr2=0x7f6aa819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 -- 172.21.15.83:0/2122438513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 msgr2=0x7f6aa819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 -- 172.21.15.83:0/2122438513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa81a04e0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aad46d640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aacc6c640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.763+0000 7f6aadc6e640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006010 tx=0x7f6aa00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.764+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003cf0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.764+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa81a0770 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.764+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00044f0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.764+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0030bf0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.764+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81a6f30 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.765+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa81035e0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.766+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6aa0004040 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.767+0000 7f6a9e7fc640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 0x7f6a940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.767+0000 7f6aad46d640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 0x7f6a940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.767+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f6aa01083e0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.767+0000 7f6aad46d640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 0x7f6a940b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6aa806bb30 tx=0x7f6aa4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:55.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.770+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6aa004c3a0 con 0x7f6aa8105ba0 2024-09-21T03:13:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.983+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa81a7100 con 0x7f6aa8105ba0 2024-09-21T03:13:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.983+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2122438513 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f6aa00cf040 con 0x7f6aa8105ba0 2024-09-21T03:13:55.985 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:55.985 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"btime":"2024-09-21T03:06:13:570144+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:13:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-09-21T03:13:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 msgr2=0x7f6a940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 0x7f6a940b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6aa806bb30 tx=0x7f6aa4005f90 comp rx=0 tx=0).stop 2024-09-21T03:13:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 msgr2=0x7f6aa819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006010 tx=0x7f6aa00093d0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 shutdown_connections 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6a940b6560 0x7f6a940b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa8105ba0 0x7f6aa819fdb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8105250 0x7f6aa819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 --2- 172.21.15.83:0/2122438513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aa81031d0 0x7f6aa819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 >> 172.21.15.83:0/2122438513 conn(0x7f6aa8079ec0 msgr2=0x7f6aa81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 shutdown_connections 2024-09-21T03:13:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:55.986+0000 7f6aaf6d1640 1 -- 172.21.15.83:0/2122438513 wait complete. 2024-09-21T03:13:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:56 smithi193 ceph-mon[35224]: Deploying daemon mds.l on smithi193 2024-09-21T03:13:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:56 smithi193 ceph-mon[35224]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:56 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2122438513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:13:56.554 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 systemd[1]: Starting Ceph mds.l for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:13:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:56 smithi083 ceph-mon[31219]: Deploying daemon mds.l on smithi193 2024-09-21T03:13:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:56 smithi083 ceph-mon[31219]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:56 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2122438513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:13:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:56 smithi142 ceph-mon[34512]: Deploying daemon mds.l on smithi193 2024-09-21T03:13:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:56 smithi142 ceph-mon[34512]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:56 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2122438513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:13:56.815 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 podman[63931]: 2024-09-21 03:13:56.455587659 +0000 UTC m=+0.016028870 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:56.815 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 podman[63931]: 2024-09-21 03:13:56.618322369 +0000 UTC m=+0.178763581 container create 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:57.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 podman[63931]: 2024-09-21 03:13:56.854252057 +0000 UTC m=+0.414693267 container init 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:57.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 podman[63931]: 2024-09-21 03:13:56.857087596 +0000 UTC m=+0.417528808 container start 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:13:57.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 ceph-mds[63950]: -- 172.21.15.193:0/2294404982 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56218d71f860 con 0x5621918c7600 2024-09-21T03:13:57.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: starting mds.l at 2024-09-21T03:13:57.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 bash[63931]: 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b 2024-09-21T03:13:57.316 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:13:56 smithi193 systemd[1]: Started Ceph mds.l for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:13:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6832/817680520,v1:172.21.15.193:6833/817680520] up:boot 2024-09-21T03:13:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: fsmap 1 up:standby 2024-09-21T03:13:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:13:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: Deploying daemon mds.k on smithi142 2024-09-21T03:13:58.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:57 smithi142 ceph-mon[34512]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6832/817680520,v1:172.21.15.193:6833/817680520] up:boot 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: fsmap 1 up:standby 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: Deploying daemon mds.k on smithi142 2024-09-21T03:13:58.284 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:57 smithi083 ceph-mon[31219]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6832/817680520,v1:172.21.15.193:6833/817680520] up:boot 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: fsmap 1 up:standby 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: Deploying daemon mds.k on smithi142 2024-09-21T03:13:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:57 smithi193 ceph-mon[35224]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:58.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:13:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:13:59.177 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 systemd[1]: Starting Ceph mds.k for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1643462846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 msgr2=0x7fc474110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1643462846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc474110620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009ca0 tx=0x7fc46c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1643462846 shutdown_connections 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1643462846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc474110620 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1643462846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 0x7fc4741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1643462846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 0x7fc474103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1643462846 >> 172.21.15.83:0/1643462846 conn(0x7fc47407b850 msgr2=0x7fc47407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1643462846 shutdown_connections 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1643462846 wait complete. 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 Processor -- start 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.495+0000 7fc47a8c7640 1 -- start start 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 0x7fc47419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 0x7fc47419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113ab0 con 0x7fc474109df0 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113930 con 0x7fc4741094a0 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc47a8c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113c30 con 0x7fc474103b60 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc4737fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 0x7fc47419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37696/0 (socket says 172.21.15.83:37696) 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 -- 172.21.15.83:0/1479448232 learned_addr learned my addr 172.21.15.83:0/1479448232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc473fff640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 0x7fc47419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 -- 172.21.15.83:0/1479448232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 msgr2=0x7fc47419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 0x7fc47419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 -- 172.21.15.83:0/1479448232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 msgr2=0x7fc47419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 0x7fc47419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.496+0000 7fc478e64640 1 -- 172.21.15.83:0/1479448232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741a03f0 con 0x7fc474109df0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc478e64640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005cc0 tx=0x7fc46c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c003cf0 con 0x7fc474109df0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a0680 con 0x7fc474109df0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c0044f0 con 0x7fc474109df0 2024-09-21T03:13:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c02fd10 con 0x7fc474109df0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.497+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a6e40 con 0x7fc474109df0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.498+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474103f70 con 0x7fc474109df0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.499+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc46c004040 con 0x7fc474109df0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.500+0000 7fc4717fa640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 0x7fc45c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.500+0000 7fc473fff640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 0x7fc45c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.500+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fc46c108520 con 0x7fc474109df0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.500+0000 7fc473fff640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 0x7fc45c0b8a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4640058c0 tx=0x7fc46400a250 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:13:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.504+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc46c0cf180 con 0x7fc474109df0 2024-09-21T03:13:59.559 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 podman[62133]: 2024-09-21 03:13:59.222582348 +0000 UTC m=+0.101378899 container create ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T03:13:59.559 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 podman[62133]: 2024-09-21 03:13:59.136723781 +0000 UTC m=+0.015520281 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.730+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc47402e640 con 0x7fc474109df0 2024-09-21T03:13:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.730+0000 7fc4717fa640 1 -- 172.21.15.83:0/1479448232 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fc46c0d3560 con 0x7fc474109df0 2024-09-21T03:13:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:13:59.733 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"btime":"2024-09-21T03:13:56:895679+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:13:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.732+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 msgr2=0x7fc45c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.732+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 0x7fc45c0b8a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4640058c0 tx=0x7fc46400a250 comp rx=0 tx=0).stop 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.732+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 msgr2=0x7fc47419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.732+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005cc0 tx=0x7fc46c02f470 comp rx=0 tx=0).stop 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 shutdown_connections 2024-09-21T03:13:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc45c0b6560 0x7fc45c0b8a70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc474109df0 0x7fc47419fcc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4741094a0 0x7fc47419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 --2- 172.21.15.83:0/1479448232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc474103b60 0x7fc47419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 >> 172.21.15.83:0/1479448232 conn(0x7fc47407b850 msgr2=0x7fc474105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 shutdown_connections 2024-09-21T03:13:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:13:59.733+0000 7fc47a8c7640 1 -- 172.21.15.83:0/1479448232 wait complete. 2024-09-21T03:13:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:13:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1479448232' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 podman[62133]: 2024-09-21 03:13:59.558551789 +0000 UTC m=+0.437348286 container init ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 podman[62133]: 2024-09-21 03:13:59.561575065 +0000 UTC m=+0.440371565 container start ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-mds[62152]: -- 172.21.15.142:0/674559086 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2dd5b5860 con 0x55d2e175d600 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: starting mds.k at 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 bash[62133]: ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 2024-09-21T03:13:59.928 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:13:59 smithi142 systemd[1]: Started Ceph mds.k for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:00.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:13:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1479448232' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:00.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:00.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:00.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:00.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:00.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:00.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:13:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1479448232' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:01.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:00 smithi083 ceph-mon[31219]: Deploying daemon mds.j on smithi083 2024-09-21T03:14:01.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:00 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6832/1409264490,v1:172.21.15.142:6833/1409264490] up:boot 2024-09-21T03:14:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:00 smithi083 ceph-mon[31219]: fsmap 2 up:standby 2024-09-21T03:14:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:14:01.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:00 smithi193 ceph-mon[35224]: Deploying daemon mds.j on smithi083 2024-09-21T03:14:01.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:00 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6832/1409264490,v1:172.21.15.142:6833/1409264490] up:boot 2024-09-21T03:14:01.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:00 smithi193 ceph-mon[35224]: fsmap 2 up:standby 2024-09-21T03:14:01.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:14:01.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:00 smithi142 ceph-mon[34512]: Deploying daemon mds.j on smithi083 2024-09-21T03:14:01.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:00 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6832/1409264490,v1:172.21.15.142:6833/1409264490] up:boot 2024-09-21T03:14:01.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:00 smithi142 ceph-mon[34512]: fsmap 2 up:standby 2024-09-21T03:14:01.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:14:01.971 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:01 smithi142 ceph-mon[34512]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:02.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:01 smithi193 ceph-mon[35224]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:01 smithi083 ceph-mon[31219]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:02.372 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 systemd[1]: Starting Ceph mds.j for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:14:02.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:02.747 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 podman[72887]: 2024-09-21 03:14:02.353519227 +0000 UTC m=+0.015467298 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:02.747 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 podman[72887]: 2024-09-21 03:14:02.49320834 +0000 UTC m=+0.155156393 container create 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:14:02.747 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 podman[72887]: 2024-09-21 03:14:02.746500221 +0000 UTC m=+0.408448283 container init 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:14:02.916 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:03.039 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 podman[72887]: 2024-09-21 03:14:02.749678005 +0000 UTC m=+0.411626066 container start 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:14:03.039 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 ceph-mds[72936]: -- 172.21.15.83:0/3490136368 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c73e7cf860 con 0x55c742977180 2024-09-21T03:14:03.039 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j[72920]: starting mds.j at 2024-09-21T03:14:03.039 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 bash[72887]: 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b 2024-09-21T03:14:03.039 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:14:02 smithi083 systemd[1]: Started Ceph mds.j for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 -- 172.21.15.83:0/3196192236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800108420 msgr2=0x7f2800108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 --2- 172.21.15.83:0/3196192236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800108420 0x7f2800108830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009900 tx=0x7f27f402a410 comp rx=0 tx=0).stop 2024-09-21T03:14:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 -- 172.21.15.83:0/3196192236 shutdown_connections 2024-09-21T03:14:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 --2- 172.21.15.83:0/3196192236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108d70 0x7f280010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 --2- 172.21.15.83:0/3196192236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800108420 0x7f2800108830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 --2- 172.21.15.83:0/3196192236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800102ae0 0x7f2800102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 -- 172.21.15.83:0/3196192236 >> 172.21.15.83:0/3196192236 conn(0x7f2800079770 msgr2=0x7f2800079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 -- 172.21.15.83:0/3196192236 shutdown_connections 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.447+0000 7f2805b6a640 1 -- 172.21.15.83:0/3196192236 wait complete. 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 Processor -- start 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 -- start start 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 0x7f280019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 0x7f280019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2800113b50 con 0x7f2800102ae0 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28001139d0 con 0x7f2800108d70 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f2805b6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2800113cd0 con 0x7f2800108420 2024-09-21T03:14:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37726/0 (socket says 172.21.15.83:37726) 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 -- 172.21.15.83:0/4178925547 learned_addr learned my addr 172.21.15.83:0/4178925547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27feffd640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 0x7f280019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27fffff640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 0x7f280019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 -- 172.21.15.83:0/4178925547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 msgr2=0x7f280019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 0x7f280019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 -- 172.21.15.83:0/4178925547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 msgr2=0x7f280019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 0x7f280019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27ff7fe640 1 -- 172.21.15.83:0/4178925547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28001a04d0 con 0x7f2800102ae0 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27fffff640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 0x7f280019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.448+0000 7f27feffd640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 0x7f280019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f27ff7fe640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d770 tx=0x7f27f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0017070 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28001a07c0 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f00054d0 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a6f20 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.449+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004c30 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.450+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2800103d60 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.451+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f27f0004eb0 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.452+0000 7f27fcff9640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 0x7f27e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.452+0000 7f27feffd640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 0x7f27e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.452+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f27f00e6b10 con 0x7f2800102ae0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.452+0000 7f27feffd640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 0x7f27e80b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009230 tx=0x7f27f4000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:03.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.456+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27f00ad770 con 0x7f2800102ae0 2024-09-21T03:14:03.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.689+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f280010ab60 con 0x7f2800102ae0 2024-09-21T03:14:03.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.689+0000 7f27fcff9640 1 -- 172.21.15.83:0/4178925547 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f27f00b1b50 con 0x7f2800102ae0 2024-09-21T03:14:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:03.692 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"btime":"2024-09-21T03:14:02:838108+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:14:03.692 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 msgr2=0x7f27e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 0x7f27e80b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009230 tx=0x7f27f4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 msgr2=0x7f280019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d770 tx=0x7f27f000dc30 comp rx=0 tx=0).stop 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 shutdown_connections 2024-09-21T03:14:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27e80b6560 0x7f27e80b8a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2800108d70 0x7f280019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2800108420 0x7f280019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 --2- 172.21.15.83:0/4178925547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2800102ae0 0x7f280019a560 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 >> 172.21.15.83:0/4178925547 conn(0x7f2800079770 msgr2=0x7f2800106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 shutdown_connections 2024-09-21T03:14:03.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:03.691+0000 7f2805b6a640 1 -- 172.21.15.83:0/4178925547 wait complete. 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6834/3297642632,v1:172.21.15.83:6835/3297642632] up:boot 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: fsmap 3 up:standby 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:03.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:03.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: Deploying daemon mds.d on smithi083 2024-09-21T03:14:03.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:03.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:03 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4178925547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6834/3297642632,v1:172.21.15.83:6835/3297642632] up:boot 2024-09-21T03:14:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: fsmap 3 up:standby 2024-09-21T03:14:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:14:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: Deploying daemon mds.d on smithi083 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:03 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4178925547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6834/3297642632,v1:172.21.15.83:6835/3297642632] up:boot 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: fsmap 3 up:standby 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:04.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:04.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:04.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: Deploying daemon mds.d on smithi083 2024-09-21T03:14:04.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:04.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:03 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4178925547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:05.597 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:05 smithi083 systemd[1]: Starting Ceph mds.d for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:14:05.872 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:05 smithi083 podman[73270]: 2024-09-21 03:14:05.693852999 +0000 UTC m=+0.015832163 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:05.872 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:05 smithi083 podman[73270]: 2024-09-21 03:14:05.797307914 +0000 UTC m=+0.119287105 container create 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:14:06.221 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 podman[73270]: 2024-09-21 03:14:06.066695183 +0000 UTC m=+0.388674332 container init 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:14:06.222 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 podman[73270]: 2024-09-21 03:14:06.069461626 +0000 UTC m=+0.391440782 container start 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:14:06.222 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 ceph-mds[73289]: -- 172.21.15.83:0/3646119711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de416e9860 con 0x55de45891180 2024-09-21T03:14:06.222 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d[73285]: starting mds.d at 2024-09-21T03:14:06.222 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 bash[73270]: 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d 2024-09-21T03:14:06.222 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:14:06 smithi083 systemd[1]: Started Ceph mds.d for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:06 smithi193 ceph-mon[35224]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:06 smithi083 ceph-mon[31219]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:06.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:06 smithi142 ceph-mon[34512]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6836/3179576739,v1:172.21.15.83:6837/3179576739] up:boot 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: fsmap 4 up:standby 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:07 smithi193 ceph-mon[35224]: Deploying daemon mds.e on smithi142 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6836/3179576739,v1:172.21.15.83:6837/3179576739] up:boot 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: fsmap 4 up:standby 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:07 smithi083 ceph-mon[31219]: Deploying daemon mds.e on smithi142 2024-09-21T03:14:07.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6836/3179576739,v1:172.21.15.83:6837/3179576739] up:boot 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: fsmap 4 up:standby 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:07.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:07.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:07.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:07.655 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:07 smithi142 ceph-mon[34512]: Deploying daemon mds.e on smithi142 2024-09-21T03:14:07.889 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:08.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:08 smithi083 ceph-mon[31219]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:08.399 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:08 smithi142 ceph-mon[34512]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 -- 172.21.15.83:0/1939250197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330102290 msgr2=0x7fa3301026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 --2- 172.21.15.83:0/1939250197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330102290 0x7fa3301026a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009900 tx=0x7fa32c02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 -- 172.21.15.83:0/1939250197 shutdown_connections 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 --2- 172.21.15.83:0/1939250197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330108520 0x7fa33010ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 --2- 172.21.15.83:0/1939250197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 0x7fa330107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 --2- 172.21.15.83:0/1939250197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330102290 0x7fa3301026a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 -- 172.21.15.83:0/1939250197 >> 172.21.15.83:0/1939250197 conn(0x7fa330079ea0 msgr2=0x7fa33007a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 -- 172.21.15.83:0/1939250197 shutdown_connections 2024-09-21T03:14:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 -- 172.21.15.83:0/1939250197 wait complete. 2024-09-21T03:14:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.418+0000 7fa33879c640 1 Processor -- start 2024-09-21T03:14:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 -- start start 2024-09-21T03:14:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330102290 0x7fa3301991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 0x7fa3301996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112ac0 con 0x7fa330108520 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112940 con 0x7fa330107bd0 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa33879c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112c40 con 0x7fa330102290 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37762/0 (socket says 172.21.15.83:37762) 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 -- 172.21.15.83:0/879944348 learned_addr learned my addr 172.21.15.83:0/879944348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa335d37640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 0x7fa3301996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336538640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330102290 0x7fa3301991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 -- 172.21.15.83:0/879944348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330102290 msgr2=0x7fa3301991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330102290 0x7fa3301991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 -- 172.21.15.83:0/879944348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 msgr2=0x7fa3301996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 0x7fa3301996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.419+0000 7fa336d39640 1 -- 172.21.15.83:0/879944348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3301a8850 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa336d39640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b970 tx=0x7fa32800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328019070 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3301a8b40 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa328014440 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3280075d0 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.420+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a9080 con 0x7fa330108520 2024-09-21T03:14:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.421+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3301026a0 con 0x7fa330108520 2024-09-21T03:14:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.422+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa32801c070 con 0x7fa330108520 2024-09-21T03:14:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.423+0000 7fa3277fe640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 0x7fa31c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.423+0000 7fa336538640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 0x7fa31c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.423+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fa3280e68f0 con 0x7fa330108520 2024-09-21T03:14:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.424+0000 7fa336538640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 0x7fa31c0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa32c000c00 tx=0x7fa32c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.426+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3280b1930 con 0x7fa330108520 2024-09-21T03:14:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:08 smithi193 ceph-mon[35224]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.640+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa33010fb20 con 0x7fa330108520 2024-09-21T03:14:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.640+0000 7fa3277fe640 1 -- 172.21.15.83:0/879944348 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fa33010fb20 con 0x7fa330108520 2024-09-21T03:14:08.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:08.642 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"btime":"2024-09-21T03:14:06:213235+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:14:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-09-21T03:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 msgr2=0x7fa31c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 0x7fa31c0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa32c000c00 tx=0x7fa32c031400 comp rx=0 tx=0).stop 2024-09-21T03:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 msgr2=0x7fa330199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b970 tx=0x7fa32800be30 comp rx=0 tx=0).stop 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 shutdown_connections 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa31c0b6560 0x7fa31c0b8a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330108520 0x7fa330199c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa330107bd0 0x7fa3301996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 --2- 172.21.15.83:0/879944348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330102290 0x7fa3301991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 >> 172.21.15.83:0/879944348 conn(0x7fa330079ea0 msgr2=0x7fa330105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 shutdown_connections 2024-09-21T03:14:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:08.642+0000 7fa33879c640 1 -- 172.21.15.83:0/879944348 wait complete. 2024-09-21T03:14:09.177 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:14:08 smithi142 systemd[1]: Started Ceph mds.e for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:09.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/879944348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:09.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.344 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:09.344 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:09.344 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:09.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/879944348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:09.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/879944348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:10.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:10 smithi193 ceph-mon[35224]: Deploying daemon mds.f on smithi193 2024-09-21T03:14:10.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:10 smithi193 ceph-mon[35224]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:10.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:10 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6834/3055596594,v1:172.21.15.142:6835/3055596594] up:boot 2024-09-21T03:14:10.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:10 smithi193 ceph-mon[35224]: fsmap 5 up:standby 2024-09-21T03:14:10.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:10 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:14:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:10 smithi083 ceph-mon[31219]: Deploying daemon mds.f on smithi193 2024-09-21T03:14:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:10 smithi083 ceph-mon[31219]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:10 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6834/3055596594,v1:172.21.15.142:6835/3055596594] up:boot 2024-09-21T03:14:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:10 smithi083 ceph-mon[31219]: fsmap 5 up:standby 2024-09-21T03:14:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:10 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:14:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:10 smithi142 ceph-mon[34512]: Deploying daemon mds.f on smithi193 2024-09-21T03:14:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:10 smithi142 ceph-mon[34512]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:10 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6834/3055596594,v1:172.21.15.142:6835/3055596594] up:boot 2024-09-21T03:14:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:10 smithi142 ceph-mon[34512]: fsmap 5 up:standby 2024-09-21T03:14:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:10 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:14:10.814 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:10 smithi193 systemd[1]: Starting Ceph mds.f for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:14:11.210 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:10 smithi193 podman[64200]: 2024-09-21 03:14:10.851324113 +0000 UTC m=+0.015632487 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:11.210 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:10 smithi193 podman[64200]: 2024-09-21 03:14:10.964771103 +0000 UTC m=+0.129079472 container create 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:14:11.210 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 podman[64200]: 2024-09-21 03:14:11.209351675 +0000 UTC m=+0.373660044 container init 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:14:11.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:11.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 podman[64200]: 2024-09-21 03:14:11.21231613 +0000 UTC m=+0.376624503 container start 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:14:11.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 ceph-mds[64219]: -- 172.21.15.193:0/3637397383 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c178bbfc20 con 0x55c17cd67180 2024-09-21T03:14:11.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: starting mds.f at 2024-09-21T03:14:11.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 bash[64200]: 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 2024-09-21T03:14:11.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:14:11 smithi193 systemd[1]: Started Ceph mds.f for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:12.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 -- 172.21.15.83:0/1470515338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc075dd0 msgr2=0x7fd8cc0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:12.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/1470515338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc0761e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009900 tx=0x7fd8bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 -- 172.21.15.83:0/1470515338 shutdown_connections 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/1470515338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc114430 0x7fd8cc116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/1470515338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc0761e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/1470515338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc077380 0x7fd8cc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.138+0000 7fd8d1efc640 1 -- 172.21.15.83:0/1470515338 >> 172.21.15.83:0/1470515338 conn(0x7fd8cc07b850 msgr2=0x7fd8cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.139+0000 7fd8d1efc640 1 -- 172.21.15.83:0/1470515338 shutdown_connections 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.139+0000 7fd8d1efc640 1 -- 172.21.15.83:0/1470515338 wait complete. 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.139+0000 7fd8d1efc640 1 Processor -- start 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.139+0000 7fd8d1efc640 1 -- start start 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc077380 0x7fd8cc1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc11a040 con 0x7fd8cc114430 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc119ec0 con 0x7fd8cc077380 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc11a1c0 con 0x7fd8cc075dd0 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42478/0 (socket says 172.21.15.83:42478) 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 -- 172.21.15.83:0/4164741745 learned_addr learned my addr 172.21.15.83:0/4164741745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8caffd640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc077380 0x7fd8cc1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 -- 172.21.15.83:0/4164741745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc075dd0 msgr2=0x7fd8cc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 -- 172.21.15.83:0/4164741745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc077380 msgr2=0x7fd8cc1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc077380 0x7fd8cc1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 -- 172.21.15.83:0/4164741745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1a69c0 con 0x7fd8cc114430 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8cbfff640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400eaa0 tx=0x7fd8c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400cdb0 con 0x7fd8cc114430 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc1a6cb0 con 0x7fd8cc114430 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8c4005020 con 0x7fd8cc114430 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.140+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40116d0 con 0x7fd8cc114430 2024-09-21T03:14:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.141+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc1ad410 con 0x7fd8cc114430 2024-09-21T03:14:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.142+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0761e0 con 0x7fd8cc114430 2024-09-21T03:14:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.143+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd8c40051d0 con 0x7fd8cc114430 2024-09-21T03:14:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.144+0000 7fd8c8ff9640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 0x7fd8b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.144+0000 7fd8cb7fe640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 0x7fd8b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.144+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd8c40e7120 con 0x7fd8cc114430 2024-09-21T03:14:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.144+0000 7fd8cb7fe640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 0x7fd8b80b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005d00 tx=0x7fd8b400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.148+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd8c4016030 con 0x7fd8cc114430 2024-09-21T03:14:12.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:12.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:boot 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: fsmap 6 up:standby 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:12.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:12 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:12.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.363+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8cc1a1b80 con 0x7fd8cc114430 2024-09-21T03:14:12.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:12.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:boot 2024-09-21T03:14:12.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: fsmap 6 up:standby 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:12.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:12 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:12.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.364+0000 7fd8c8ff9640 1 -- 172.21.15.83:0/4164741745 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7fd8c40add80 con 0x7fd8cc114430 2024-09-21T03:14:12.367 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:12.367 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"btime":"2024-09-21T03:14:11:254442+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24601,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T03:14:12.367 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-09-21T03:14:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.366+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 msgr2=0x7fd8b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.366+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 0x7fd8b80b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005d00 tx=0x7fd8b400b410 comp rx=0 tx=0).stop 2024-09-21T03:14:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.366+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 msgr2=0x7fd8cc1a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.366+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400eaa0 tx=0x7fd8c400ef60 comp rx=0 tx=0).stop 2024-09-21T03:14:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 shutdown_connections 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8b80b6560 0x7fd8b80b8a70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc114430 0x7fd8cc1a6290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc077380 0x7fd8cc1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 --2- 172.21.15.83:0/4164741745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc075dd0 0x7fd8cc1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 >> 172.21.15.83:0/4164741745 conn(0x7fd8cc07b850 msgr2=0x7fd8cc112d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 shutdown_connections 2024-09-21T03:14:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:12.367+0000 7fd8d1efc640 1 -- 172.21.15.83:0/4164741745 wait complete. 2024-09-21T03:14:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:boot 2024-09-21T03:14:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: fsmap 6 up:standby 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:12 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:13.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:13 smithi193 ceph-mon[35224]: Deploying daemon mds.c on smithi193 2024-09-21T03:14:13.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:13 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4164741745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:13 smithi083 ceph-mon[31219]: Deploying daemon mds.c on smithi193 2024-09-21T03:14:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:13 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4164741745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:13 smithi142 ceph-mon[34512]: Deploying daemon mds.c on smithi193 2024-09-21T03:14:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:13 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4164741745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:14.266 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:14:13 smithi193 systemd[1]: Started Ceph mds.c for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:14.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:14 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:14 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:14 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:15.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:15.270 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:15 smithi083 ceph-mon[31219]: Deploying daemon mds.h on smithi142 2024-09-21T03:14:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:15 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:boot 2024-09-21T03:14:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:15 smithi083 ceph-mon[31219]: fsmap 7 up:standby 2024-09-21T03:14:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:15 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:14:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:15 smithi193 ceph-mon[35224]: Deploying daemon mds.h on smithi142 2024-09-21T03:14:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:15 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:boot 2024-09-21T03:14:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:15 smithi193 ceph-mon[35224]: fsmap 7 up:standby 2024-09-21T03:14:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:15 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:14:15.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:15 smithi142 ceph-mon[34512]: Deploying daemon mds.h on smithi142 2024-09-21T03:14:15.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:15 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:boot 2024-09-21T03:14:15.580 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:15 smithi142 ceph-mon[34512]: fsmap 7 up:standby 2024-09-21T03:14:15.580 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:15 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:14:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 -- 172.21.15.83:0/2114506714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 msgr2=0x7f323c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 --2- 172.21.15.83:0/2114506714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c10f520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3234009ca0 tx=0x7f323402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 -- 172.21.15.83:0/2114506714 shutdown_connections 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 --2- 172.21.15.83:0/2114506714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c10f520 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 --2- 172.21.15.83:0/2114506714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 0x7f323c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 --2- 172.21.15.83:0/2114506714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 0x7f323c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 -- 172.21.15.83:0/2114506714 >> 172.21.15.83:0/2114506714 conn(0x7f323c079730 msgr2=0x7f323c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:15.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 -- 172.21.15.83:0/2114506714 shutdown_connections 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.797+0000 7f3242555640 1 -- 172.21.15.83:0/2114506714 wait complete. 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 Processor -- start 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 -- start start 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 0x7f323c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 0x7f323c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f323c112960 con 0x7f323c108cf0 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f323c1127e0 con 0x7f323c102a60 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3242555640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f323c112ae0 con 0x7f323c1083a0 2024-09-21T03:14:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3240af2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3240af2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42494/0 (socket says 172.21.15.83:42494) 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f3240af2640 1 -- 172.21.15.83:0/3492439179 learned_addr learned my addr 172.21.15.83:0/3492439179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.798+0000 7f323bfff640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 0x7f323c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3233fff640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 0x7f323c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 -- 172.21.15.83:0/3492439179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 msgr2=0x7f323c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 0x7f323c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 -- 172.21.15.83:0/3492439179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 msgr2=0x7f323c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 0x7f323c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 -- 172.21.15.83:0/3492439179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c1a0410 con 0x7f323c108cf0 2024-09-21T03:14:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3240af2640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32340098d0 tx=0x7f32340317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234003b70 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3234003d10 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c1a06a0 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323402fa20 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.799+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c1a6e60 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.800+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323c103ce0 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.803+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f323402fbe0 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.803+0000 7f3239ffb640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 0x7f32280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.804+0000 7f323bfff640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 0x7f32280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.804+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f3234108470 con 0x7f323c108cf0 2024-09-21T03:14:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.804+0000 7f323bfff640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 0x7f32280b8a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f322c0095f0 tx=0x7f322c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:15.807+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32340cf0d0 con 0x7f323c108cf0 2024-09-21T03:14:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.026+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f323c10baf0 con 0x7f323c108cf0 2024-09-21T03:14:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.027+0000 7f3239ffb640 1 -- 172.21.15.83:0/3492439179 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f32340d34b0 con 0x7f323c108cf0 2024-09-21T03:14:16.029 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:16.029 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"btime":"2024-09-21T03:14:14:746471+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24601,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:14:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-09-21T03:14:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 msgr2=0x7f32280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 0x7f32280b8a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f322c0095f0 tx=0x7f322c005f90 comp rx=0 tx=0).stop 2024-09-21T03:14:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 msgr2=0x7f323c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32340098d0 tx=0x7f32340317b0 comp rx=0 tx=0).stop 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 shutdown_connections 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32280b6560 0x7f32280b8a70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c108cf0 0x7f323c19fce0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c1083a0 0x7f323c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 --2- 172.21.15.83:0/3492439179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c102a60 0x7f323c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 >> 172.21.15.83:0/3492439179 conn(0x7f323c079730 msgr2=0x7f323c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 shutdown_connections 2024-09-21T03:14:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:16.029+0000 7f3242555640 1 -- 172.21.15.83:0/3492439179 wait complete. 2024-09-21T03:14:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:16 smithi193 ceph-mon[35224]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3492439179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:16 smithi083 ceph-mon[31219]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3492439179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:16 smithi142 ceph-mon[34512]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3492439179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:16.677 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:14:16 smithi142 systemd[1]: Started Ceph mds.h for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: Deploying daemon mds.i on smithi193 2024-09-21T03:14:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:17.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:boot 2024-09-21T03:14:17.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: fsmap 8 up:standby 2024-09-21T03:14:17.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:17 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:17.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: Deploying daemon mds.i on smithi193 2024-09-21T03:14:17.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:17.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:boot 2024-09-21T03:14:17.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: fsmap 8 up:standby 2024-09-21T03:14:17.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:17 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:14:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: Deploying daemon mds.i on smithi193 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:boot 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: fsmap 8 up:standby 2024-09-21T03:14:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:17 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:14:18.564 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:18 smithi193 systemd[1]: Starting Ceph mds.i for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:14:18.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:18.987 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:18.990 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:18 smithi193 podman[64752]: 2024-09-21 03:14:18.600353273 +0000 UTC m=+0.015931064 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:18.990 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:18 smithi193 podman[64752]: 2024-09-21 03:14:18.73419804 +0000 UTC m=+0.149775839 container create 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:14:18.990 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:18 smithi193 podman[64752]: 2024-09-21 03:14:18.989086201 +0000 UTC m=+0.404663993 container init 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:14:19.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:18 smithi193 podman[64752]: 2024-09-21 03:14:18.992049844 +0000 UTC m=+0.407627636 container start 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:19.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:19 smithi193 ceph-mds[64772]: -- 172.21.15.193:0/1439672371 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e1e91b860 con 0x563e22ac3a80 2024-09-21T03:14:19.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:19 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: starting mds.i at 2024-09-21T03:14:19.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:19 smithi193 bash[64752]: 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 2024-09-21T03:14:19.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:14:19 smithi193 systemd[1]: Started Ceph mds.i for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 -- 172.21.15.83:0/1882277544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96410de10 msgr2=0x7ff9641102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 --2- 172.21.15.83:0/1882277544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96410de10 0x7ff9641102f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009ca0 tx=0x7ff95c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 -- 172.21.15.83:0/1882277544 shutdown_connections 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 --2- 172.21.15.83:0/1882277544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96410de10 0x7ff9641102f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 --2- 172.21.15.83:0/1882277544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff964105290 0x7ff96410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 --2- 172.21.15.83:0/1882277544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 0x7ff964104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 -- 172.21.15.83:0/1882277544 >> 172.21.15.83:0/1882277544 conn(0x7ff964079730 msgr2=0x7ff964079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.509+0000 7ff96920b640 1 -- 172.21.15.83:0/1882277544 shutdown_connections 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.510+0000 7ff96920b640 1 -- 172.21.15.83:0/1882277544 wait complete. 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.510+0000 7ff96920b640 1 Processor -- start 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.510+0000 7ff96920b640 1 -- start start 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 0x7ff96419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 0x7ff96419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff964113a60 con 0x7ff964105290 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641138e0 con 0x7ff96410de10 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96920b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff964113be0 con 0x7ff964104860 2024-09-21T03:14:19.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55682/0 (socket says 172.21.15.83:55682) 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 -- 172.21.15.83:0/1156511458 learned_addr learned my addr 172.21.15.83:0/1156511458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96359e640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 0x7ff96419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff962d9d640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 0x7ff96419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 -- 172.21.15.83:0/1156511458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 msgr2=0x7ff96419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 0x7ff96419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 -- 172.21.15.83:0/1156511458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 msgr2=0x7ff96419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 0x7ff96419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 -- 172.21.15.83:0/1156511458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9641a0400 con 0x7ff964105290 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96359e640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 0x7ff96419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff96259c640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.511+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95800ccc0 con 0x7ff964105290 2024-09-21T03:14:19.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.512+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9580108e0 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.512+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641a06f0 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.512+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958018860 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.512+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a6e50 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.513+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff964104c70 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.514+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7ff958010430 con 0x7ff964105290 2024-09-21T03:14:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.515+0000 7ff94bfff640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 0x7ff94c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:19.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.515+0000 7ff962d9d640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 0x7ff94c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:19.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.515+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7ff9580e7870 con 0x7ff964105290 2024-09-21T03:14:19.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.516+0000 7ff962d9d640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 0x7ff94c0b8af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff954009aa0 tx=0x7ff954005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:19.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.518+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9580ae430 con 0x7ff964105290 2024-09-21T03:14:19.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.740+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9641a7180 con 0x7ff964105290 2024-09-21T03:14:19.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.741+0000 7ff94bfff640 1 -- 172.21.15.83:0/1156511458 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7ff9580b2810 con 0x7ff964105290 2024-09-21T03:14:19.743 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:19.743 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"btime":"2024-09-21T03:14:19:081082+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24601,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24628,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:14:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 msgr2=0x7ff94c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 0x7ff94c0b8af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff954009aa0 tx=0x7ff954005f90 comp rx=0 tx=0).stop 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 msgr2=0x7ff96419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 shutdown_connections 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff94c0b65e0 0x7ff94c0b8af0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96410de10 0x7ff96419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964105290 0x7ff96419aa30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 --2- 172.21.15.83:0/1156511458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff964104860 0x7ff96419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 >> 172.21.15.83:0/1156511458 conn(0x7ff964079730 msgr2=0x7ff964107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 shutdown_connections 2024-09-21T03:14:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:19.743+0000 7ff96920b640 1 -- 172.21.15.83:0/1156511458 wait complete. 2024-09-21T03:14:20.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:boot 2024-09-21T03:14:20.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: fsmap 9 up:standby 2024-09-21T03:14:20.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:14:20.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: Deploying daemon mds.g on smithi083 2024-09-21T03:14:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1156511458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:boot 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: fsmap 9 up:standby 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: Deploying daemon mds.g on smithi083 2024-09-21T03:14:20.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1156511458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:boot 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: fsmap 9 up:standby 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:20.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:20.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: Deploying daemon mds.g on smithi083 2024-09-21T03:14:20.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1156511458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:21.744 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:21 smithi083 podman[74008]: 2024-09-21 03:14:21.584426721 +0000 UTC m=+0.015359841 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:21.744 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:21 smithi083 podman[74008]: 2024-09-21 03:14:21.743084529 +0000 UTC m=+0.174017645 container create e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T03:14:22.017 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:21 smithi083 podman[74008]: 2024-09-21 03:14:21.980309606 +0000 UTC m=+0.411242730 container init e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:14:22.018 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:21 smithi083 podman[74008]: 2024-09-21 03:14:21.983264271 +0000 UTC m=+0.414197386 container start e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:14:22.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mds[74027]: -- 172.21.15.83:0/2976511950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee60415c20 con 0x55ee645bd600 2024-09-21T03:14:22.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: starting mds.g at 2024-09-21T03:14:22.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:22 smithi083 bash[74008]: e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c 2024-09-21T03:14:22.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:14:22 smithi083 systemd[1]: Started Ceph mds.g for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:22 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:22.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:22 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:22.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:22 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:23.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:23.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:23 smithi083 ceph-mon[31219]: Deploying daemon mds.a on smithi083 2024-09-21T03:14:23.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:23 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6838/3046510766,v1:172.21.15.83:6839/3046510766] up:boot 2024-09-21T03:14:23.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:23 smithi083 ceph-mon[31219]: fsmap 10 up:standby 2024-09-21T03:14:23.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:23 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:14:23.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:23 smithi193 ceph-mon[35224]: Deploying daemon mds.a on smithi083 2024-09-21T03:14:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:23 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6838/3046510766,v1:172.21.15.83:6839/3046510766] up:boot 2024-09-21T03:14:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:23 smithi193 ceph-mon[35224]: fsmap 10 up:standby 2024-09-21T03:14:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:23 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:14:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:23 smithi142 ceph-mon[34512]: Deploying daemon mds.a on smithi083 2024-09-21T03:14:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:23 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6838/3046510766,v1:172.21.15.83:6839/3046510766] up:boot 2024-09-21T03:14:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:23 smithi142 ceph-mon[34512]: fsmap 10 up:standby 2024-09-21T03:14:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:23 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:14:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.312+0000 7f387ee81640 1 -- 172.21.15.83:0/2306369372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 msgr2=0x7f3878102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.312+0000 7f387ee81640 1 --2- 172.21.15.83:0/2306369372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f3878102e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f386c009900 tx=0x7f386c02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.312+0000 7f387ee81640 1 -- 172.21.15.83:0/2306369372 shutdown_connections 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.312+0000 7f387ee81640 1 --2- 172.21.15.83:0/2306369372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3878108d10 0x7f387810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 --2- 172.21.15.83:0/2306369372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38781083c0 0x7f38781087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 --2- 172.21.15.83:0/2306369372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f3878102e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 -- 172.21.15.83:0/2306369372 >> 172.21.15.83:0/2306369372 conn(0x7f3878079740 msgr2=0x7f3878079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 -- 172.21.15.83:0/2306369372 shutdown_connections 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 -- 172.21.15.83:0/2306369372 wait complete. 2024-09-21T03:14:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 Processor -- start 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 -- start start 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38781083c0 0x7f387819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.313+0000 7f387ee81640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 0x7f387819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387ee81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3878112a60 con 0x7f3878102a80 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387ee81640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38781128e0 con 0x7f3878108d10 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387ee81640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3878112be0 con 0x7f38781083c0 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55730/0 (socket says 172.21.15.83:55730) 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 -- 172.21.15.83:0/4074172732 learned_addr learned my addr 172.21.15.83:0/4074172732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387d41e640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 0x7f387819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f3877fff640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38781083c0 0x7f387819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 -- 172.21.15.83:0/4074172732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38781083c0 msgr2=0x7f387819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38781083c0 0x7f387819b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 -- 172.21.15.83:0/4074172732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 msgr2=0x7f387819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 0x7f387819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 -- 172.21.15.83:0/4074172732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38781a6650 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387d41e640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 0x7f387819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387cc1d640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f386c006010 tx=0x7f386c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c03f030 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386c003800 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38781a68e0 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.314+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c035070 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.315+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38781a6e40 con 0x7f3878102a80 2024-09-21T03:14:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.316+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3878103d00 con 0x7f3878102a80 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.317+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f386c003cb0 con 0x7f3878102a80 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.318+0000 7f3875ffb640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 0x7f38680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.318+0000 7f3877fff640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 0x7f38680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.318+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f386c107a40 con 0x7f3878102a80 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.318+0000 7f3877fff640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 0x7f38680b8a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3878079ce0 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.322+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f386c0ce6a0 con 0x7f3878102a80 2024-09-21T03:14:24.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:24 smithi083 ceph-mon[31219]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:24.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.548+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f387806bc50 con 0x7f3878102a80 2024-09-21T03:14:24.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.549+0000 7f3875ffb640 1 -- 172.21.15.83:0/4074172732 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7f386c0d2a80 con 0x7f3878102a80 2024-09-21T03:14:24.551 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:24.551 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"btime":"2024-09-21T03:14:22:216690+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24601,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24628,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:14:24.551 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 msgr2=0x7f38680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 0x7f38680b8a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3878079ce0 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 msgr2=0x7f387819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f386c006010 tx=0x7f386c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 shutdown_connections 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38680b6560 0x7f38680b8a70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3878108d10 0x7f387819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38781083c0 0x7f387819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 --2- 172.21.15.83:0/4074172732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3878102a80 0x7f387819b390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.551+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 >> 172.21.15.83:0/4074172732 conn(0x7f3878079740 msgr2=0x7f3878101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.552+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 shutdown_connections 2024-09-21T03:14:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:24.552+0000 7f387ee81640 1 -- 172.21.15.83:0/4074172732 wait complete. 2024-09-21T03:14:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:24 smithi193 ceph-mon[35224]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:24 smithi142 ceph-mon[34512]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4074172732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:25.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:25 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4074172732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:25 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4074172732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:25.872 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:14:25 smithi083 systemd[1]: Started Ceph mds.a for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6840/2540451962,v1:172.21.15.83:6841/2540451962] up:boot 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: fsmap 11 up:standby 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6840/2540451962,v1:172.21.15.83:6841/2540451962] up:boot 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: fsmap 11 up:standby 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:26.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6840/2540451962,v1:172.21.15.83:6841/2540451962] up:boot 2024-09-21T03:14:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: fsmap 11 up:standby 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:14:26.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:27.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:27 smithi142 ceph-mon[34512]: Deploying daemon mds.b on smithi142 2024-09-21T03:14:27.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:27 smithi193 ceph-mon[35224]: Deploying daemon mds.b on smithi142 2024-09-21T03:14:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:27 smithi083 ceph-mon[31219]: Deploying daemon mds.b on smithi142 2024-09-21T03:14:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 -- 172.21.15.83:0/1445619458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 msgr2=0x7f52c41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 --2- 172.21.15.83:0/1445619458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c41098b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009900 tx=0x7f52b802a410 comp rx=0 tx=0).stop 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 -- 172.21.15.83:0/1445619458 shutdown_connections 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 --2- 172.21.15.83:0/1445619458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4109df0 0x7f52c4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 --2- 172.21.15.83:0/1445619458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c41098b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 --2- 172.21.15.83:0/1445619458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52c4103b60 0x7f52c4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 -- 172.21.15.83:0/1445619458 >> 172.21.15.83:0/1445619458 conn(0x7f52c407b850 msgr2=0x7f52c407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 -- 172.21.15.83:0/1445619458 shutdown_connections 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.186+0000 7f52ca840640 1 -- 172.21.15.83:0/1445619458 wait complete. 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 Processor -- start 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 -- start start 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4103b60 0x7f52c419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52c4109df0 0x7f52c419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4113ab0 con 0x7f52c41094a0 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4113930 con 0x7f52c4109df0 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52ca840640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4113c30 con 0x7f52c4103b60 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55788/0 (socket says 172.21.15.83:55788) 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 -- 172.21.15.83:0/2194386328 learned_addr learned my addr 172.21.15.83:0/2194386328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c3fff640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4103b60 0x7f52c419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 -- 172.21.15.83:0/2194386328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4103b60 msgr2=0x7f52c419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4103b60 0x7f52c419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 -- 172.21.15.83:0/2194386328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52c4109df0 msgr2=0x7f52c419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52c4109df0 0x7f52c419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.187+0000 7f52c37fe640 1 -- 172.21.15.83:0/2194386328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c41a04f0 con 0x7f52c41094a0 2024-09-21T03:14:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52c37fe640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52b802a930 tx=0x7f52b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b803f030 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b8003aa0 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c41a0780 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8035070 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.188+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c41a6f40 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.189+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c4104de0 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.191+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f52b8042030 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.191+0000 7f52c17fa640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 0x7f52ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.191+0000 7f52c3fff640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 0x7f52ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.191+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f52b8107ae0 con 0x7f52c41094a0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.192+0000 7f52c3fff640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 0x7f52ac0b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005fd0 tx=0x7f52b4005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.194+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52b80ce740 con 0x7f52c41094a0 2024-09-21T03:14:28.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:28 smithi083 ceph-mon[31219]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.416+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c4103f70 con 0x7f52c41094a0 2024-09-21T03:14:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.417+0000 7f52c17fa640 1 -- 172.21.15.83:0/2194386328 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f52c4103f70 con 0x7f52c41094a0 2024-09-21T03:14:28.420 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"btime":"2024-09-21T03:14:28:247873+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24601,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24628,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T03:14:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-09-21T03:14:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 msgr2=0x7f52ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 0x7f52ac0b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005fd0 tx=0x7f52b4005850 comp rx=0 tx=0).stop 2024-09-21T03:14:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 msgr2=0x7f52c419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52b802a930 tx=0x7f52b8004270 comp rx=0 tx=0).stop 2024-09-21T03:14:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 shutdown_connections 2024-09-21T03:14:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52ac0b6560 0x7f52ac0b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52c4109df0 0x7f52c419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c41094a0 0x7f52c419ab20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 --2- 172.21.15.83:0/2194386328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c4103b60 0x7f52c419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.419+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 >> 172.21.15.83:0/2194386328 conn(0x7f52c407b850 msgr2=0x7f52c4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.420+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 shutdown_connections 2024-09-21T03:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:28.420+0000 7f52ca840640 1 -- 172.21.15.83:0/2194386328 wait complete. 2024-09-21T03:14:28.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:28 smithi193 ceph-mon[35224]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:28 smithi142 ceph-mon[34512]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:28.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:28 smithi142 systemd[1]: Started Ceph mds.b for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:14:29.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:14:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:boot 2024-09-21T03:14:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: fsmap 12 up:standby 2024-09-21T03:14:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:14:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2194386328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:29.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:29 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:boot 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: fsmap 12 up:standby 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2194386328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:29 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:boot 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: fsmap 12 up:standby 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2194386328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:29 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:30.019 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:30 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- 172.21.15.83:0/1318809891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0107be0 msgr2=0x7fd6c0107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:30.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 --2- 172.21.15.83:0/1318809891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0107be0 0x7fd6c0107ff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009900 tx=0x7fd6bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:30.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- 172.21.15.83:0/1318809891 shutdown_connections 2024-09-21T03:14:30.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 --2- 172.21.15.83:0/1318809891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c0108530 0x7fd6c010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 --2- 172.21.15.83:0/1318809891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0107be0 0x7fd6c0107ff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 --2- 172.21.15.83:0/1318809891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c01022a0 0x7fd6c01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- 172.21.15.83:0/1318809891 >> 172.21.15.83:0/1318809891 conn(0x7fd6c0079ec0 msgr2=0x7fd6c007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- 172.21.15.83:0/1318809891 shutdown_connections 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- 172.21.15.83:0/1318809891 wait complete. 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 Processor -- start 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.495+0000 7fd6c72be640 1 -- start start 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c01022a0 0x7fd6c00feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c0107be0 0x7fd6c00ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111970 con 0x7fd6c0108530 2024-09-21T03:14:30.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01117f0 con 0x7fd6c01022a0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c72be640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111af0 con 0x7fd6c0107be0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49822/0 (socket says 172.21.15.83:49822) 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 -- 172.21.15.83:0/3953888254 learned_addr learned my addr 172.21.15.83:0/3953888254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c4859640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c0107be0 0x7fd6c00ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c505a640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c01022a0 0x7fd6c00feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 -- 172.21.15.83:0/3953888254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c0107be0 msgr2=0x7fd6c00ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c0107be0 0x7fd6c00ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 -- 172.21.15.83:0/3953888254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c01022a0 msgr2=0x7fd6c00feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c01022a0 0x7fd6c00feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 -- 172.21.15.83:0/3953888254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c01a8860 con 0x7fd6c0108530 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.496+0000 7fd6c585b640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800eaa0 tx=0x7fd6b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.497+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800cdb0 con 0x7fd6c0108530 2024-09-21T03:14:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.497+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b8005020 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.497+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c01a8b50 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.497+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80116d0 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.497+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c01a9060 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.498+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c01026b0 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.499+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd6b80051d0 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.500+0000 7fd6b27fc640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 0x7fd6ac0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.500+0000 7fd6c505a640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 0x7fd6ac0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.500+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd6b80e7390 con 0x7fd6c0108530 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.500+0000 7fd6c505a640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 0x7fd6ac0b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0100940 tx=0x7fd6b4010040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:30.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.503+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6b8016030 con 0x7fd6c0108530 2024-09-21T03:14:30.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:30 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:30 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.731+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6c01001f0 con 0x7fd6c0108530 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.732+0000 7fd6b27fc640 1 -- 172.21.15.83:0/3953888254 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fd6c01001f0 con 0x7fd6c0108530 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-09-21T03:14:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:14:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 msgr2=0x7fd6ac0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 0x7fd6ac0b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0100940 tx=0x7fd6b4010040 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 msgr2=0x7fd6c00ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800eaa0 tx=0x7fd6b800ef60 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 shutdown_connections 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6ac0b65e0 0x7fd6ac0b8af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6c0108530 0x7fd6c00ff970 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c0107be0 0x7fd6c00ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 --2- 172.21.15.83:0/3953888254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6c01022a0 0x7fd6c00feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 >> 172.21.15.83:0/3953888254 conn(0x7fd6c0079ec0 msgr2=0x7fd6c0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 shutdown_connections 2024-09-21T03:14:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:30.734+0000 7fd6c72be640 1 -- 172.21.15.83:0/3953888254 wait complete. 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3953888254' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:31 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:31.457 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:14:31.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3953888254' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:31 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3953888254' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:31 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:14:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:32.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:32 smithi083 ceph-mon[31219]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:32 smithi193 ceph-mon[35224]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:32 smithi142 ceph-mon[34512]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/1822600584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 msgr2=0x7f2c98102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/1822600584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c98102e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009900 tx=0x7f2c8c02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/1822600584 shutdown_connections 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/1822600584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98108d10 0x7f2c9810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/1822600584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c981083c0 0x7f2c981087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/1822600584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c98102e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/1822600584 >> 172.21.15.83:0/1822600584 conn(0x7f2c98079740 msgr2=0x7f2c98079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/1822600584 shutdown_connections 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.906+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/1822600584 wait complete. 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 Processor -- start 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 -- start start 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981083c0 0x7f2c9819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 0x7f2c9819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981129e0 con 0x7f2c98102a80 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98112860 con 0x7f2c98108d10 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9f1e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98112b60 con 0x7f2c981083c0 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9cf80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9cf80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49836/0 (socket says 172.21.15.83:49836) 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9cf80640 1 -- 172.21.15.83:0/417722472 learned_addr learned my addr 172.21.15.83:0/417722472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.907+0000 7f2c9d781640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 0x7f2c9819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 -- 172.21.15.83:0/417722472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981083c0 msgr2=0x7f2c9819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981083c0 0x7f2c9819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 -- 172.21.15.83:0/417722472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 msgr2=0x7f2c9819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 0x7f2c9819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 -- 172.21.15.83:0/417722472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981a6590 con 0x7f2c98102a80 2024-09-21T03:14:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9d781640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 0x7f2c9819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9cf80640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005940 tx=0x7f2c8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c03f030 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a6820 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c003cb0 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c035070 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.908+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a6d80 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.909+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c98103d00 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.911+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2c8c042030 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.911+0000 7f2c95ffb640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 0x7f2c880b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.912+0000 7f2c97fff640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 0x7f2c880b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.912+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2c8c107bb0 con 0x7f2c98102a80 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.912+0000 7f2c97fff640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 0x7f2c880b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005fd0 tx=0x7f2c84005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:32.915+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c8c0ce810 con 0x7f2c98102a80 2024-09-21T03:14:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.114+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2c9806bf70 con 0x7f2c98102a80 2024-09-21T03:14:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.573+0000 7f2c95ffb640 1 -- 172.21.15.83:0/417722472 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f2c8c0d2bf0 con 0x7f2c98102a80 2024-09-21T03:14:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-09-21T03:14:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 msgr2=0x7f2c880b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 0x7f2c880b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005fd0 tx=0x7f2c84005850 comp rx=0 tx=0).stop 2024-09-21T03:14:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 msgr2=0x7f2c9819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005940 tx=0x7f2c8c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:14:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 shutdown_connections 2024-09-21T03:14:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c880b6560 0x7f2c880b8a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c98108d10 0x7f2c9819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981083c0 0x7f2c9819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 --2- 172.21.15.83:0/417722472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98102a80 0x7f2c9819b2d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:33.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 >> 172.21.15.83:0/417722472 conn(0x7f2c98079740 msgr2=0x7f2c98101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:33.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 shutdown_connections 2024-09-21T03:14:33.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:33.576+0000 7f2c9f1e4640 1 -- 172.21.15.83:0/417722472 wait complete. 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:33.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:33 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:33.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:33.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:33 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:14:33.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:33 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:14:34.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:14:34.481 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:34.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:34 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:14:34.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:34 smithi083 ceph-mon[31219]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:14:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:34 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:14:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:34 smithi142 ceph-mon[34512]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 -- 172.21.15.83:0/2930598198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 msgr2=0x7fc7d010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2930598198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d010d8d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009900 tx=0x7fc7b802a410 comp rx=0 tx=0).stop 2024-09-21T03:14:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 -- 172.21.15.83:0/2930598198 shutdown_connections 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2930598198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d010de10 0x7fc7d01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2930598198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d010d8d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2930598198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d0104860 0x7fc7d0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 -- 172.21.15.83:0/2930598198 >> 172.21.15.83:0/2930598198 conn(0x7fc7d0079730 msgr2=0x7fc7d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 -- 172.21.15.83:0/2930598198 shutdown_connections 2024-09-21T03:14:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.010+0000 7fc7d6431640 1 -- 172.21.15.83:0/2930598198 wait complete. 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 Processor -- start 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 -- start start 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d0104860 0x7fc7d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 0x7fc7d019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0113aa0 con 0x7fc7d0105290 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0113920 con 0x7fc7d010de10 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7d6431640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0113c20 con 0x7fc7d0104860 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49856/0 (socket says 172.21.15.83:49856) 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.011+0000 7fc7c7fff640 1 -- 172.21.15.83:0/2245192594 learned_addr learned my addr 172.21.15.83:0/2245192594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 -- 172.21.15.83:0/2245192594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d0104860 msgr2=0x7fc7d019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d0104860 0x7fc7d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 -- 172.21.15.83:0/2245192594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 msgr2=0x7fc7d019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7d49ce640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 0x7fc7d019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 0x7fc7d019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 -- 172.21.15.83:0/2245192594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d01a0440 con 0x7fc7d0105290 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7d49ce640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 0x7fc7d019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7c7fff640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802a930 tx=0x7fc7b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:35.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b803f030 con 0x7fc7d0105290 2024-09-21T03:14:35.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b8003cb0 con 0x7fc7d0105290 2024-09-21T03:14:35.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d01a06d0 con 0x7fc7d0105290 2024-09-21T03:14:35.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8035070 con 0x7fc7d0105290 2024-09-21T03:14:35.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.012+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a0bb0 con 0x7fc7d0105290 2024-09-21T03:14:35.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.013+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d0104c70 con 0x7fc7d0105290 2024-09-21T03:14:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.015+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc7b8042030 con 0x7fc7d0105290 2024-09-21T03:14:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.016+0000 7fc7cdffb640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 0x7fc7bc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.016+0000 7fc7cffff640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 0x7fc7bc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.016+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7712+0+0 (secure 0 0 0) 0x7fc7b8107c90 con 0x7fc7d0105290 2024-09-21T03:14:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.016+0000 7fc7cffff640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 0x7fc7bc0b8b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000a990 tx=0x7fc7c0005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.019+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7b80ce700 con 0x7fc7d0105290 2024-09-21T03:14:35.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:34 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/417722472' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:14:35.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:34 smithi193 ceph-mon[35224]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:14:35.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.200+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc7d002e640 con 0x7fc7d0105290 2024-09-21T03:14:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.587+0000 7fc7cdffb640 1 -- 172.21.15.83:0/2245192594 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fc7b80d2ae0 con 0x7fc7d0105290 2024-09-21T03:14:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 msgr2=0x7fc7bc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 0x7fc7bc0b8b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000a990 tx=0x7fc7c0005c70 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 msgr2=0x7fc7d019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802a930 tx=0x7fc7b8004270 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 shutdown_connections 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7bc0b6610 0x7fc7bc0b8b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7d010de10 0x7fc7d019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d0105290 0x7fc7d019ab30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 --2- 172.21.15.83:0/2245192594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d0104860 0x7fc7d019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 >> 172.21.15.83:0/2245192594 conn(0x7fc7d0079730 msgr2=0x7fc7d00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 shutdown_connections 2024-09-21T03:14:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:35.590+0000 7fc7d6431640 1 -- 172.21.15.83:0/2245192594 wait complete. 2024-09-21T03:14:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:35 smithi083 ceph-mon[31219]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:14:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:35 smithi083 ceph-mon[31219]: pgmap v258: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:35 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:14:35.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:35 smithi142 ceph-mon[34512]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:14:35.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:35 smithi142 ceph-mon[34512]: pgmap v258: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:35.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:35 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:14:36.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:35 smithi193 ceph-mon[35224]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:14:36.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:35 smithi193 ceph-mon[35224]: pgmap v258: 33 pgs: 5 creating+peering, 27 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:36.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:35 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:14:36.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:14:36.498 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:36.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:36 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:14:36.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:36 smithi083 ceph-mon[31219]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:14:36.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:36 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:14:36.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:36 smithi142 ceph-mon[34512]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 -- 172.21.15.83:0/2002192558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8410de50 msgr2=0x7f8a84110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 --2- 172.21.15.83:0/2002192558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8410de50 0x7f8a84110320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009ca0 tx=0x7f8a7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 -- 172.21.15.83:0/2002192558 shutdown_connections 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 --2- 172.21.15.83:0/2002192558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8410de50 0x7f8a84110320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 --2- 172.21.15.83:0/2002192558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a84105280 0x7f8a8410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 --2- 172.21.15.83:0/2002192558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 0x7f8a84104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 -- 172.21.15.83:0/2002192558 >> 172.21.15.83:0/2002192558 conn(0x7f8a84079720 msgr2=0x7f8a84079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 -- 172.21.15.83:0/2002192558 shutdown_connections 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.956+0000 7f8a896d5640 1 -- 172.21.15.83:0/2002192558 wait complete. 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 Processor -- start 2024-09-21T03:14:36.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 -- start start 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 0x7f8a841027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 0x7f8a84103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84113b10 con 0x7f8a84105280 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84113990 con 0x7f8a841048a0 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a896d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84113c90 con 0x7f8a8410de50 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49876/0 (socket says 172.21.15.83:49876) 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a827fc640 1 -- 172.21.15.83:0/69631699 learned_addr learned my addr 172.21.15.83:0/69631699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a82ffd640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 0x7f8a841027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.957+0000 7f8a837fe640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 0x7f8a84103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 -- 172.21.15.83:0/69631699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 msgr2=0x7f8a84103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 0x7f8a84103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 -- 172.21.15.83:0/69631699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 msgr2=0x7f8a841027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:36.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 0x7f8a841027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 -- 172.21.15.83:0/69631699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a841a86b0 con 0x7f8a84105280 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a837fe640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 0x7f8a84103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a827fc640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c8c0 tx=0x7f8a6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c007bb0 con 0x7f8a84105280 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841a89a0 con 0x7f8a84105280 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a6c007d50 con 0x7f8a84105280 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0186e0 con 0x7f8a84105280 2024-09-21T03:14:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.958+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a841a8f60 con 0x7f8a84105280 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.959+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84104cb0 con 0x7f8a84105280 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.961+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8a6c00f450 con 0x7f8a84105280 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.961+0000 7f8a7bfff640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 0x7f8a700b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.961+0000 7f8a82ffd640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 0x7f8a700b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.962+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8094+0+0 (secure 0 0 0) 0x7f8a6c0e76b0 con 0x7f8a84105280 2024-09-21T03:14:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.962+0000 7f8a82ffd640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 0x7f8a700b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a84104200 tx=0x7f8a74009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:36.965+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a6c014030 con 0x7f8a84105280 2024-09-21T03:14:37.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:36 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2245192594' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:14:37.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:36 smithi193 ceph-mon[35224]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:14:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.182+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a84103ab0 con 0x7f8a84105280 2024-09-21T03:14:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.596+0000 7f8a7bfff640 1 -- 172.21.15.83:0/69631699 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8a6c0ae0b0 con 0x7f8a84105280 2024-09-21T03:14:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:14:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 msgr2=0x7f8a700b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 0x7f8a700b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a84104200 tx=0x7f8a74009110 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 msgr2=0x7f8a84102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00c8c0 tx=0x7f8a6c00ccc0 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 shutdown_connections 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a700b6560 0x7f8a700b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a8410de50 0x7f8a84103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a84105280 0x7f8a84102cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 --2- 172.21.15.83:0/69631699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841048a0 0x7f8a841027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 >> 172.21.15.83:0/69631699 conn(0x7f8a84079720 msgr2=0x7f8a841079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 shutdown_connections 2024-09-21T03:14:37.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:37.599+0000 7f8a896d5640 1 -- 172.21.15.83:0/69631699 wait complete. 2024-09-21T03:14:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:37 smithi083 ceph-mon[31219]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:14:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:37 smithi083 ceph-mon[31219]: pgmap v261: 97 pgs: 13 active+clean, 11 creating+peering, 73 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:37 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:14:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:37 smithi083 ceph-mon[31219]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:14:37.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:37 smithi142 ceph-mon[34512]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:14:37.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:37 smithi142 ceph-mon[34512]: pgmap v261: 97 pgs: 13 active+clean, 11 creating+peering, 73 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:37.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:37 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:14:37.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:37 smithi142 ceph-mon[34512]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:14:38.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:37 smithi193 ceph-mon[35224]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:14:38.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:37 smithi193 ceph-mon[35224]: pgmap v261: 97 pgs: 13 active+clean, 11 creating+peering, 73 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:14:38.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:37 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:14:38.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:37 smithi193 ceph-mon[35224]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:14:38.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:14:38.520 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:38.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:38 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:14:38.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:38 smithi083 ceph-mon[31219]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:14:38.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:38 smithi083 ceph-mon[31219]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:14:38.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:38 smithi083 ceph-mon[31219]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:14:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:38 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:14:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:38 smithi142 ceph-mon[34512]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:14:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:38 smithi142 ceph-mon[34512]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:14:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:38 smithi142 ceph-mon[34512]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:14:39.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.021+0000 7f6284bd5640 1 -- 172.21.15.83:0/3287277321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280103b60 msgr2=0x7f6280103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:39.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 --2- 172.21.15.83:0/3287277321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280103b60 0x7f6280103f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6270009900 tx=0x7f627002a410 comp rx=0 tx=0).stop 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 -- 172.21.15.83:0/3287277321 shutdown_connections 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 --2- 172.21.15.83:0/3287277321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6280109df0 0x7f6280110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 --2- 172.21.15.83:0/3287277321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62801094a0 0x7f62801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 --2- 172.21.15.83:0/3287277321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280103b60 0x7f6280103f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 -- 172.21.15.83:0/3287277321 >> 172.21.15.83:0/3287277321 conn(0x7f628007b850 msgr2=0x7f628007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 -- 172.21.15.83:0/3287277321 shutdown_connections 2024-09-21T03:14:39.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 -- 172.21.15.83:0/3287277321 wait complete. 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.022+0000 7f6284bd5640 1 Processor -- start 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 -- start start 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6280103b60 0x7f6280075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62801094a0 0x7f6280075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6280113ab0 con 0x7f6280109df0 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6280113930 con 0x7f62801094a0 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f6284bd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6280113c30 con 0x7f6280103b60 2024-09-21T03:14:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49902/0 (socket says 172.21.15.83:49902) 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 -- 172.21.15.83:0/2172855532 learned_addr learned my addr 172.21.15.83:0/2172855532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627dd9b640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62801094a0 0x7f6280075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627e59c640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6280103b60 0x7f6280075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 -- 172.21.15.83:0/2172855532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6280103b60 msgr2=0x7f6280075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6280103b60 0x7f6280075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 -- 172.21.15.83:0/2172855532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62801094a0 msgr2=0x7f6280075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62801094a0 0x7f6280075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.023+0000 7f627ed9d640 1 -- 172.21.15.83:0/2172855532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62801a87e0 con 0x7f6280109df0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f627ed9d640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f627800cad0 tx=0x7f627800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627800ae20 con 0x7f6280109df0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62801a8ad0 con 0x7f6280109df0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6278005020 con 0x7f6280109df0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627800f6b0 con 0x7f6280109df0 2024-09-21T03:14:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.024+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a8fc0 con 0x7f6280109df0 2024-09-21T03:14:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.025+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6280103f70 con 0x7f6280109df0 2024-09-21T03:14:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.026+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f62780051d0 con 0x7f6280109df0 2024-09-21T03:14:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.027+0000 7f626f7fe640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 0x7f62680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.027+0000 7f627e59c640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 0x7f62680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.027+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f62780e7820 con 0x7f6280109df0 2024-09-21T03:14:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.027+0000 7f627e59c640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 0x7f62680b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6270000c00 tx=0x7f6270031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.030+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6278014030 con 0x7f6280109df0 2024-09-21T03:14:39.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:38 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/69631699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:14:39.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:38 smithi193 ceph-mon[35224]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:14:39.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:38 smithi193 ceph-mon[35224]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:14:39.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:38 smithi193 ceph-mon[35224]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:14:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.208+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6280076a10 con 0x7f6280109df0 2024-09-21T03:14:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.210+0000 7f626f7fe640 1 -- 172.21.15.83:0/2172855532 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22617 (secure 0 0 0) 0x7f6280076a10 con 0x7f6280109df0 2024-09-21T03:14:39.212 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:39.213 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":75,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:14:37.183774+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 msgr2=0x7f62680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 0x7f62680b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6270000c00 tx=0x7f6270031400 comp rx=0 tx=0).stop 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 msgr2=0x7f6280076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f627800cad0 tx=0x7f627800cf90 comp rx=0 tx=0).stop 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 shutdown_connections 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62680b6560 0x7f62680b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6280109df0 0x7f6280076190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62801094a0 0x7f6280075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 --2- 172.21.15.83:0/2172855532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6280103b60 0x7f6280075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 >> 172.21.15.83:0/2172855532 conn(0x7f628007b850 msgr2=0x7f62801075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 shutdown_connections 2024-09-21T03:14:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:39.212+0000 7f6284bd5640 1 -- 172.21.15.83:0/2172855532 wait complete. 2024-09-21T03:14:39.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:39 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:active 2024-09-21T03:14:39.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:39 smithi083 ceph-mon[31219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:39.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:39 smithi083 ceph-mon[31219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:39.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:39 smithi083 ceph-mon[31219]: pgmap v263: 97 pgs: 79 active+clean, 6 creating+peering, 12 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-21T03:14:39.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:39 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2172855532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:39.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:14:39.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:39 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:active 2024-09-21T03:14:39.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:39 smithi142 ceph-mon[34512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:39.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:39 smithi142 ceph-mon[34512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:39.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:39 smithi142 ceph-mon[34512]: pgmap v263: 97 pgs: 79 active+clean, 6 creating+peering, 12 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-21T03:14:39.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:39 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2172855532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:40.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:39 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] up:active 2024-09-21T03:14:40.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:39 smithi193 ceph-mon[35224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:40.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:39 smithi193 ceph-mon[35224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:40.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:39 smithi193 ceph-mon[35224]: pgmap v263: 97 pgs: 79 active+clean, 6 creating+peering, 12 unknown; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-21T03:14:40.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:39 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2172855532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.554+0000 7fda5c740640 1 -- 172.21.15.83:0/254607213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54108d70 msgr2=0x7fda5410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.554+0000 7fda5c740640 1 --2- 172.21.15.83:0/254607213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54108d70 0x7fda5410f5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009ca0 tx=0x7fda4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 -- 172.21.15.83:0/254607213 shutdown_connections 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 --2- 172.21.15.83:0/254607213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54108d70 0x7fda5410f5a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 --2- 172.21.15.83:0/254607213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 0x7fda54108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 --2- 172.21.15.83:0/254607213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54102ae0 0x7fda54102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 -- 172.21.15.83:0/254607213 >> 172.21.15.83:0/254607213 conn(0x7fda54079770 msgr2=0x7fda54079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 -- 172.21.15.83:0/254607213 shutdown_connections 2024-09-21T03:14:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 -- 172.21.15.83:0/254607213 wait complete. 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 Processor -- start 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.555+0000 7fda5c740640 1 -- start start 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 0x7fda5419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 0x7fda5419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda54113b40 con 0x7fda54102ae0 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda541139c0 con 0x7fda54108420 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5c740640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda54113cc0 con 0x7fda54108d70 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45248/0 (socket says 172.21.15.83:45248) 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 -- 172.21.15.83:0/3556214103 learned_addr learned my addr 172.21.15.83:0/3556214103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda59cdb640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 0x7fda5419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5acdd640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 0x7fda5419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 -- 172.21.15.83:0/3556214103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 msgr2=0x7fda5419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 0x7fda5419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 -- 172.21.15.83:0/3556214103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 msgr2=0x7fda5419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 0x7fda5419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 -- 172.21.15.83:0/3556214103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541a0420 con 0x7fda54102ae0 2024-09-21T03:14:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5acdd640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 0x7fda5419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda5a4dc640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d770 tx=0x7fda5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.556+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50017070 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.557+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda541a0710 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.557+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda500054d0 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.557+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50004c30 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.557+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5410fde0 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.557+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda54103d60 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.559+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fda50004dd0 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.560+0000 7fda4b7fe640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 0x7fda400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.560+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fda500e7250 con 0x7fda54102ae0 2024-09-21T03:14:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.560+0000 7fda59cdb640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 0x7fda400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.560+0000 7fda59cdb640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 0x7fda400b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda54105fc0 tx=0x7fda4400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.563+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda500adb00 con 0x7fda54102ae0 2024-09-21T03:14:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.742+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda5410ab60 con 0x7fda54102ae0 2024-09-21T03:14:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.743+0000 7fda4b7fe640 1 -- 172.21.15.83:0/3556214103 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22617 (secure 0 0 0) 0x7fda500b1ee0 con 0x7fda54102ae0 2024-09-21T03:14:40.746 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:40.746 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":75,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:14:37.183774+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:14:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.745+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 msgr2=0x7fda400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 0x7fda400b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda54105fc0 tx=0x7fda4400b390 comp rx=0 tx=0).stop 2024-09-21T03:14:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 msgr2=0x7fda5419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d770 tx=0x7fda5000dc30 comp rx=0 tx=0).stop 2024-09-21T03:14:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 shutdown_connections 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fda400b6560 0x7fda400b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda54108d70 0x7fda5419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda54108420 0x7fda5419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 --2- 172.21.15.83:0/3556214103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda54102ae0 0x7fda5419a4b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 >> 172.21.15.83:0/3556214103 conn(0x7fda54079770 msgr2=0x7fda54104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 shutdown_connections 2024-09-21T03:14:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:40.746+0000 7fda5c740640 1 -- 172.21.15.83:0/3556214103 wait complete. 2024-09-21T03:14:40.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:40 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3556214103' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:41.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:40 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3556214103' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:41.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:40 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3556214103' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:14:41.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:14:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:42.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:41 smithi193 ceph-mon[35224]: pgmap v264: 97 pgs: 91 active+clean, 6 creating+peering; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2024-09-21T03:14:42.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:41 smithi142 ceph-mon[34512]: pgmap v264: 97 pgs: 91 active+clean, 6 creating+peering; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2024-09-21T03:14:42.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:41 smithi083 ceph-mon[31219]: pgmap v264: 97 pgs: 91 active+clean, 6 creating+peering; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 -- 172.21.15.83:0/3088554295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea41094c0 msgr2=0x7f0ea41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/3088554295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea41094c0 0x7f0ea41098d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e9802a410 comp rx=0 tx=0).stop 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 -- 172.21.15.83:0/3088554295 shutdown_connections 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/3088554295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ea4109e10 0x7f0ea4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/3088554295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea41094c0 0x7f0ea41098d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/3088554295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4103b80 0x7f0ea4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 -- 172.21.15.83:0/3088554295 >> 172.21.15.83:0/3088554295 conn(0x7f0ea40ff610 msgr2=0x7f0ea4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 -- 172.21.15.83:0/3088554295 shutdown_connections 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 -- 172.21.15.83:0/3088554295 wait complete. 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.294+0000 7f0ea94d8640 1 Processor -- start 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 -- start start 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ea41094c0 0x7f0ea419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4109e10 0x7f0ea419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4113af0 con 0x7f0ea4103b80 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4113970 con 0x7f0ea4109e10 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea94d8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4113c70 con 0x7f0ea41094c0 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:42.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45272/0 (socket says 172.21.15.83:45272) 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 -- 172.21.15.83:0/1362096222 learned_addr learned my addr 172.21.15.83:0/1362096222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea37fe640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4109e10 0x7f0ea419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 -- 172.21.15.83:0/1362096222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ea41094c0 msgr2=0x7f0ea419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ea41094c0 0x7f0ea419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 -- 172.21.15.83:0/1362096222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4109e10 msgr2=0x7f0ea419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4109e10 0x7f0ea419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 -- 172.21.15.83:0/1362096222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea41a0450 con 0x7f0ea4103b80 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.295+0000 7f0ea2ffd640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eab0 tx=0x7f0e9400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.296+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9400ce40 con 0x7f0ea4103b80 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.296+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e94011920 con 0x7f0ea4103b80 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.296+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea41a0740 con 0x7f0ea4103b80 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.296+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94010a30 con 0x7f0ea4103b80 2024-09-21T03:14:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.296+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41a6ea0 con 0x7f0ea4103b80 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.297+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea4103f90 con 0x7f0ea4103b80 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.298+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f0e94010bd0 con 0x7f0ea4103b80 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.299+0000 7f0e8bfff640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 0x7f0e8c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.299+0000 7f0ea27fc640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 0x7f0e8c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.299+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f0e940e7950 con 0x7f0ea4103b80 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.299+0000 7f0ea27fc640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 0x7f0e8c0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e9802a930 tx=0x7f0e98031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:42.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.303+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e94016030 con 0x7f0ea4103b80 2024-09-21T03:14:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.516+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0ea402e640 con 0x7f0ea4103b80 2024-09-21T03:14:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.807+0000 7f0e8bfff640 1 -- 172.21.15.83:0/1362096222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f0e940ae200 con 0x7f0ea4103b80 2024-09-21T03:14:42.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.810+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 msgr2=0x7f0e8c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.810+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 0x7f0e8c0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e9802a930 tx=0x7f0e98031400 comp rx=0 tx=0).stop 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.810+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 msgr2=0x7f0ea419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.810+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eab0 tx=0x7f0e9400ef70 comp rx=0 tx=0).stop 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.810+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 shutdown_connections 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0e8c0b6560 0x7f0e8c0b8a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ea4109e10 0x7f0ea419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ea41094c0 0x7f0ea419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 --2- 172.21.15.83:0/1362096222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103b80 0x7f0ea419a4e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:42.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 >> 172.21.15.83:0/1362096222 conn(0x7f0ea40ff610 msgr2=0x7f0ea4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:42.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 shutdown_connections 2024-09-21T03:14:42.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:42.811+0000 7f0ea94d8640 1 -- 172.21.15.83:0/1362096222 wait complete. 2024-09-21T03:14:43.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:42 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:14:43.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:42 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:14:43.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:42 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:14:43.515 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-21T03:14:43.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T03:14:43.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:14:43.262+0000 7fe74a4b4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:14:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:44.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:14:44.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:43 smithi083 ceph-mon[31219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:44.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:43 smithi083 ceph-mon[31219]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T03:14:44.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:14:44.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:43 smithi142 ceph-mon[34512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:44.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:43 smithi142 ceph-mon[34512]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T03:14:44.185 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1362096222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:14:44.186 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:43 smithi193 ceph-mon[35224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:44.186 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:43 smithi193 ceph-mon[35224]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T03:14:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1833468704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4108d10 msgr2=0x7fa5b410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1833468704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4108d10 0x7fa5b410f540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a402a410 comp rx=0 tx=0).stop 2024-09-21T03:14:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1833468704 shutdown_connections 2024-09-21T03:14:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1833468704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4108d10 0x7fa5b410f540 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1833468704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b41083c0 0x7fa5b41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.295+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1833468704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b4102a80 0x7fa5b4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.296+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1833468704 >> 172.21.15.83:0/1833468704 conn(0x7fa5b4079740 msgr2=0x7fa5b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.296+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1833468704 shutdown_connections 2024-09-21T03:14:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.296+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1833468704 wait complete. 2024-09-21T03:14:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.296+0000 7fa5bba2c640 1 Processor -- start 2024-09-21T03:14:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.296+0000 7fa5bba2c640 1 -- start start 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b41083c0 0x7fa5b419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 0x7fa5b419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4113aa0 con 0x7fa5b4102a80 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4113920 con 0x7fa5b4108d10 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4113c20 con 0x7fa5b41083c0 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45288/0 (socket says 172.21.15.83:45288) 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 -- 172.21.15.83:0/1250650576 learned_addr learned my addr 172.21.15.83:0/1250650576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b8fc7640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b41083c0 0x7fa5b419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b9fc9640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 0x7fa5b419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 -- 172.21.15.83:0/1250650576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b41083c0 msgr2=0x7fa5b419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b41083c0 0x7fa5b419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 -- 172.21.15.83:0/1250650576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 msgr2=0x7fa5b419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 0x7fa5b419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 -- 172.21.15.83:0/1250650576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b41a0480 con 0x7fa5b4102a80 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b9fc9640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 0x7fa5b419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b97c8640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d950 tx=0x7fa5ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac017070 con 0x7fa5b4102a80 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ac009c30 con 0x7fa5b4102a80 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b41a0770 con 0x7fa5b4102a80 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00f620 con 0x7fa5b4102a80 2024-09-21T03:14:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.297+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b41a6ed0 con 0x7fa5b4102a80 2024-09-21T03:14:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.299+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b4103d00 con 0x7fa5b4102a80 2024-09-21T03:14:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.300+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa5ac010460 con 0x7fa5b4102a80 2024-09-21T03:14:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.301+0000 7fa5b27fc640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 0x7fa59c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.301+0000 7fa5b8fc7640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 0x7fa59c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.301+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa5ac0e6fc0 con 0x7fa5b4102a80 2024-09-21T03:14:44.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.301+0000 7fa5b8fc7640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 0x7fa59c0b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4105fd0 tx=0x7fa5a8009680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.305+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa5ac0ad870 con 0x7fa5b4102a80 2024-09-21T03:14:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.520+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fa5b410ab30 con 0x7fa5b4102a80 2024-09-21T03:14:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.818+0000 7fa5b27fc640 1 -- 172.21.15.83:0/1250650576 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fa5b410ab30 con 0x7fa5b4102a80 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 msgr2=0x7fa59c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 0x7fa59c0b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4105fd0 tx=0x7fa5a8009680 comp rx=0 tx=0).stop 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 msgr2=0x7fa5b419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d950 tx=0x7fa5ac00de10 comp rx=0 tx=0).stop 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 shutdown_connections 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa59c0b6560 0x7fa59c0b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5b4108d10 0x7fa5b419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b41083c0 0x7fa5b419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 --2- 172.21.15.83:0/1250650576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5b4102a80 0x7fa5b419a510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.820+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 >> 172.21.15.83:0/1250650576 conn(0x7fa5b4079740 msgr2=0x7fa5b4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.821+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 shutdown_connections 2024-09-21T03:14:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:44.821+0000 7fa5bba2c640 1 -- 172.21.15.83:0/1250650576 wait complete. 2024-09-21T03:14:45.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:44 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:14:45.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:44 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:14:45.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:44 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T03:14:45.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:14:45.777 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:46.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:14:46.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:14:46.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:14:46.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:46.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:14:46.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:14:46.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:45 smithi083 ceph-mon[31219]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:14:46.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:14:46.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:45 smithi142 ceph-mon[34512]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1250650576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:14:46.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:45 smithi193 ceph-mon[35224]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-21T03:14:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 -- 172.21.15.83:0/939970338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc1051a0 msgr2=0x7f82bc1055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/939970338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc1051a0 0x7f82bc1055b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009900 tx=0x7f82ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 -- 172.21.15.83:0/939970338 shutdown_connections 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/939970338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc105af0 0x7f82bc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/939970338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc1051a0 0x7f82bc1055b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/939970338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 0x7f82bc1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 -- 172.21.15.83:0/939970338 >> 172.21.15.83:0/939970338 conn(0x7f82bc079f90 msgr2=0x7f82bc07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 -- 172.21.15.83:0/939970338 shutdown_connections 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 -- 172.21.15.83:0/939970338 wait complete. 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.314+0000 7f82c3bf4640 1 Processor -- start 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 -- start start 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 0x7f82bc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 0x7f82bc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112a10 con 0x7f82bc105af0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112890 con 0x7f82bc1030b0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c3bf4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112b90 con 0x7f82bc1051a0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45308/0 (socket says 172.21.15.83:45308) 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 -- 172.21.15.83:0/3408171746 learned_addr learned my addr 172.21.15.83:0/3408171746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c1990640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 0x7f82bc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c118f640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 0x7f82bc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 -- 172.21.15.83:0/3408171746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 msgr2=0x7f82bc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 0x7f82bc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 -- 172.21.15.83:0/3408171746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 msgr2=0x7f82bc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 0x7f82bc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c2191640 1 -- 172.21.15.83:0/3408171746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc1a0420 con 0x7f82bc105af0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.315+0000 7f82c118f640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 0x7f82bc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82c2191640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82b400b970 tx=0x7f82b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4019070 con 0x7f82bc105af0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc1a0710 con 0x7f82bc105af0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b4014440 con 0x7f82bc105af0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc1a6f30 con 0x7f82bc105af0 2024-09-21T03:14:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.316+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40075d0 con 0x7f82bc105af0 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.317+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc1034c0 con 0x7f82bc105af0 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.319+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f82b401c070 con 0x7f82bc105af0 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.319+0000 7f82b2ffd640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 0x7f82a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.319+0000 7f82c1990640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 0x7f82a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.319+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f82b40e6cb0 con 0x7f82bc105af0 2024-09-21T03:14:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.320+0000 7f82c1990640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 0x7f82a80b8aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82b800abe0 tx=0x7f82b800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.322+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82b40ad560 con 0x7f82bc105af0 2024-09-21T03:14:46.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.538+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f82bc19b6a0 con 0x7f82bc105af0 2024-09-21T03:14:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.838+0000 7f82b2ffd640 1 -- 172.21.15.83:0/3408171746 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f82bc19b6a0 con 0x7f82bc105af0 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 msgr2=0x7f82a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 0x7f82a80b8aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82b800abe0 tx=0x7f82b800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 msgr2=0x7f82bc19fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82b400b970 tx=0x7f82b400bd70 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 shutdown_connections 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f82a80b6590 0x7f82a80b8aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82bc105af0 0x7f82bc19fdb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82bc1051a0 0x7f82bc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 --2- 172.21.15.83:0/3408171746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc1030b0 0x7f82bc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 >> 172.21.15.83:0/3408171746 conn(0x7f82bc079f90 msgr2=0x7f82bc1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 shutdown_connections 2024-09-21T03:14:46.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:46.840+0000 7f82c3bf4640 1 -- 172.21.15.83:0/3408171746 wait complete. 2024-09-21T03:14:47.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:active 2024-09-21T03:14:47.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:14:47.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:14:47.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:14:47.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:14:47.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:46 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:active 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:14:47.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:46 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] up:active 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:14:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:46 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:14:47.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:14:47.841 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:14:47.841 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:active 2024-09-21T03:14:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:48.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:14:48.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:14:48.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:14:48.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:14:48.118 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:47 smithi083 ceph-mon[31219]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T03:14:48.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:14:48.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:active 2024-09-21T03:14:48.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:14:48.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:14:48.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:14:48.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:14:48.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:47 smithi142 ceph-mon[34512]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3408171746' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] up:active 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T03:14:48.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:47 smithi193 ceph-mon[35224]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T03:14:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 -- 172.21.15.83:0/2416372535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620108d70 msgr2=0x7fa62010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 --2- 172.21.15.83:0/2416372535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620108d70 0x7fa62010f5a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa618009ca0 tx=0x7fa61802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 -- 172.21.15.83:0/2416372535 shutdown_connections 2024-09-21T03:14:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 --2- 172.21.15.83:0/2416372535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620108d70 0x7fa62010f5a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 --2- 172.21.15.83:0/2416372535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108420 0x7fa620108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 --2- 172.21.15.83:0/2416372535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620102ae0 0x7fa620102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 -- 172.21.15.83:0/2416372535 >> 172.21.15.83:0/2416372535 conn(0x7fa620079770 msgr2=0x7fa620079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 -- 172.21.15.83:0/2416372535 shutdown_connections 2024-09-21T03:14:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.366+0000 7fa62503c640 1 -- 172.21.15.83:0/2416372535 wait complete. 2024-09-21T03:14:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.367+0000 7fa62503c640 1 Processor -- start 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 -- start start 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620108420 0x7fa62019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108d70 0x7fa6201a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa620112a70 con 0x7fa620102ae0 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201128f0 con 0x7fa620108420 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa62503c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa620112bf0 con 0x7fa620108d70 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45322/0 (socket says 172.21.15.83:45322) 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 -- 172.21.15.83:0/1208318076 learned_addr learned my addr 172.21.15.83:0/1208318076 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61e59c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620108420 0x7fa62019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61f59e640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108d70 0x7fa6201a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 -- 172.21.15.83:0/1208318076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108d70 msgr2=0x7fa6201a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108d70 0x7fa6201a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 -- 172.21.15.83:0/1208318076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620108420 msgr2=0x7fa62019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620108420 0x7fa62019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.368+0000 7fa61ed9d640 1 -- 172.21.15.83:0/1208318076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6201a2660 con 0x7fa620102ae0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa61ed9d640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa61000c890 tx=0x7fa61000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610004220 con 0x7fa620102ae0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6100043c0 con 0x7fa620102ae0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6201a2950 con 0x7fa620102ae0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610004a00 con 0x7fa620102ae0 2024-09-21T03:14:48.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.369+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6201a90b0 con 0x7fa620102ae0 2024-09-21T03:14:48.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.370+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa620103d60 con 0x7fa620102ae0 2024-09-21T03:14:48.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.372+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa6100079b0 con 0x7fa620102ae0 2024-09-21T03:14:48.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.372+0000 7fa617fff640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 0x7fa60c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:48.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.373+0000 7fa61e59c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 0x7fa60c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:48.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.373+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa6100e75a0 con 0x7fa620102ae0 2024-09-21T03:14:48.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.373+0000 7fa61e59c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 0x7fa60c0b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa620108110 tx=0x7fa608008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.376+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa610014030 con 0x7fa620102ae0 2024-09-21T03:14:48.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.592+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa62019d820 con 0x7fa620102ae0 2024-09-21T03:14:48.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:14:48.263+0000 7fe74a4b4640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T03:14:48.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:14:48.263+0000 7fe74a4b4640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T03:14:48.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:14:48.263+0000 7fe74a4b4640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-21T03:14:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.754+0000 7fa617fff640 1 -- 172.21.15.83:0/1208318076 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fa6100ade50 con 0x7fa620102ae0 2024-09-21T03:14:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 msgr2=0x7fa60c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 0x7fa60c0b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa620108110 tx=0x7fa608008040 comp rx=0 tx=0).stop 2024-09-21T03:14:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 msgr2=0x7fa62019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa61000c890 tx=0x7fa61000cd50 comp rx=0 tx=0).stop 2024-09-21T03:14:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 shutdown_connections 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa60c0b6560 0x7fa60c0b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa620108d70 0x7fa6201a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa620108420 0x7fa62019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 --2- 172.21.15.83:0/1208318076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa620102ae0 0x7fa62019c6f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 >> 172.21.15.83:0/1208318076 conn(0x7fa620079770 msgr2=0x7fa620106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 shutdown_connections 2024-09-21T03:14:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:48.758+0000 7fa62503c640 1 -- 172.21.15.83:0/1208318076 wait complete. 2024-09-21T03:14:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:active 2024-09-21T03:14:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:14:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:14:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: Cluster is now healthy 2024-09-21T03:14:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:active 2024-09-21T03:14:49.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:48 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:14:49.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:active 2024-09-21T03:14:49.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:14:49.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:14:49.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: Cluster is now healthy 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:active 2024-09-21T03:14:49.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:48 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] up:active 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: Cluster is now healthy 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1208318076' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] up:active 2024-09-21T03:14:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:48 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:14:49.634 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-21T03:14:49.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:14:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:50.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:49 smithi083 ceph-mon[31219]: pgmap v268: 97 pgs: 97 active+clean; 583 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 619 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:14:50.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:49 smithi142 ceph-mon[34512]: pgmap v268: 97 pgs: 97 active+clean; 583 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 619 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:14:50.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:49 smithi193 ceph-mon[35224]: pgmap v268: 97 pgs: 97 active+clean; 583 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 619 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T03:14:50.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 -- 172.21.15.83:0/3079135669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc1031d0 msgr2=0x7f5dcc1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:50.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/3079135669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc1035e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009900 tx=0x7f5dbc02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:50.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 -- 172.21.15.83:0/3079135669 shutdown_connections 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/3079135669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc108d50 0x7f5dcc10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/3079135669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/3079135669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc1035e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 -- 172.21.15.83:0/3079135669 >> 172.21.15.83:0/3079135669 conn(0x7f5dcc079f90 msgr2=0x7f5dcc07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 -- 172.21.15.83:0/3079135669 shutdown_connections 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.432+0000 7f5dd31c9640 1 -- 172.21.15.83:0/3079135669 wait complete. 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 Processor -- start 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 -- start start 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc112ad0 con 0x7f5dcc108d50 2024-09-21T03:14:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc112950 con 0x7f5dcc1031d0 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd31c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc112c50 con 0x7f5dcc108400 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45914/0 (socket says 172.21.15.83:45914) 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 -- 172.21.15.83:0/4129362716 learned_addr learned my addr 172.21.15.83:0/4129362716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd0f65640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dcbfff640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 -- 172.21.15.83:0/4129362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 msgr2=0x7f5dcc19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 -- 172.21.15.83:0/4129362716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 msgr2=0x7f5dcc19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 -- 172.21.15.83:0/4129362716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc1a27d0 con 0x7f5dcc108d50 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd0f65640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dcbfff640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.433+0000 7f5dd1766640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400b970 tx=0x7f5dc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.434+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4019070 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.434+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc4014440 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.434+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc1a2ac0 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.434+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40075d0 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.434+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a9220 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.435+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc103660 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.436+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f5dc401c070 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.437+0000 7f5dc9ffb640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 0x7f5db40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.437+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f5dc40e6d50 con 0x7f5dcc108d50 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.437+0000 7f5dd0f65640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 0x7f5db40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.437+0000 7f5dd0f65640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 0x7f5db40b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc031440 tx=0x7f5dbc005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:50.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.440+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5dc40ad600 con 0x7f5dcc108d50 2024-09-21T03:14:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.644+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5dcc063800 con 0x7f5db40b6560 2024-09-21T03:14:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.668+0000 7f5dc9ffb640 1 -- 172.21.15.83:0/4129362716 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5dcc063800 con 0x7f5db40b6560 2024-09-21T03:14:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 msgr2=0x7f5db40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 0x7f5db40b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc031440 tx=0x7f5dbc005d50 comp rx=0 tx=0).stop 2024-09-21T03:14:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 msgr2=0x7f5dcc1a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400b970 tx=0x7f5dc400be30 comp rx=0 tx=0).stop 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 shutdown_connections 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5db40b6560 0x7f5db40b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dcc108d50 0x7f5dcc1a20a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dcc108400 0x7f5dcc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 --2- 172.21.15.83:0/4129362716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dcc1031d0 0x7f5dcc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 >> 172.21.15.83:0/4129362716 conn(0x7f5dcc079f90 msgr2=0x7f5dcc1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 shutdown_connections 2024-09-21T03:14:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:50.671+0000 7f5dd31c9640 1 -- 172.21.15.83:0/4129362716 wait complete. 2024-09-21T03:14:50.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:14:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:14:50.649+0000 7f46ad7bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:14:50.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:14:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:14:50.649+0000 7f46ad7bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:14:50.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:14:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:14:50.649+0000 7f46ad7bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:14:50.872 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:14:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:14:50.649+0000 7f46ad7bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:14:50.873 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:14:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:14:50.649+0000 7f46ad7bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:14:51.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:14:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:52.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 -- 172.21.15.83:0/3806567688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c109e00 msgr2=0x7f7a1c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/3806567688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c109e00 0x7f7a1c110630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009ca0 tx=0x7f7a1402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 -- 172.21.15.83:0/3806567688 shutdown_connections 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/3806567688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c109e00 0x7f7a1c110630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/3806567688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/3806567688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 0x7f7a1c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 -- 172.21.15.83:0/3806567688 >> 172.21.15.83:0/3806567688 conn(0x7f7a1c07b910 msgr2=0x7f7a1c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 -- 172.21.15.83:0/3806567688 shutdown_connections 2024-09-21T03:14:52.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.129+0000 7f7a22bc4640 1 -- 172.21.15.83:0/3806567688 wait complete. 2024-09-21T03:14:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 Processor -- start 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 -- start start 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 0x7f7a1c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 0x7f7a1c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c113ac0 con 0x7f7a1c1094b0 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c113940 con 0x7f7a1c109e00 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a22bc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c113c40 con 0x7f7a1c103bc0 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45970/0 (socket says 172.21.15.83:45970) 2024-09-21T03:14:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 -- 172.21.15.83:0/2178327715 learned_addr learned my addr 172.21.15.83:0/2178327715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a21161640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 0x7f7a1c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a20960640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 0x7f7a1c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 -- 172.21.15.83:0/2178327715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 msgr2=0x7f7a1c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 0x7f7a1c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 -- 172.21.15.83:0/2178327715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 msgr2=0x7f7a1c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 0x7f7a1c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 -- 172.21.15.83:0/2178327715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c1a03c0 con 0x7f7a1c1094b0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a21161640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 0x7f7a1c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.130+0000 7f7a1bfff640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d740 tx=0x7f7a1000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.131+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10017070 con 0x7f7a1c1094b0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.131+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1000de50 con 0x7f7a1c1094b0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.131+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1a06b0 con 0x7f7a1c1094b0 2024-09-21T03:14:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.131+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000f620 con 0x7f7a1c1094b0 2024-09-21T03:14:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.131+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a6e10 con 0x7f7a1c1094b0 2024-09-21T03:14:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.132+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c103fd0 con 0x7f7a1c1094b0 2024-09-21T03:14:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.133+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7a1001a070 con 0x7f7a1c1094b0 2024-09-21T03:14:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.134+0000 7f7a19ffb640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 0x7f7a040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.134+0000 7f7a20960640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 0x7f7a040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.134+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f7a100e6f00 con 0x7f7a1c1094b0 2024-09-21T03:14:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.134+0000 7f7a20960640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 0x7f7a040b8b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c005fd0 tx=0x7f7a0c005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:52.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.138+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a100ad650 con 0x7f7a1c1094b0 2024-09-21T03:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.344+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7a1c063800 con 0x7f7a040b6670 2024-09-21T03:14:52.347 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:52 smithi083 ceph-mon[31219]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:52.347 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:52 smithi083 ceph-mon[31219]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-09-21T03:14:52.347 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:52 smithi083 ceph-mon[31219]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:14:52.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.376+0000 7f7a19ffb640 1 -- 172.21.15.83:0/2178327715 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a1c063800 con 0x7f7a040b6670 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 msgr2=0x7f7a040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 0x7f7a040b8b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c005fd0 tx=0x7f7a0c005850 comp rx=0 tx=0).stop 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 msgr2=0x7f7a1c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d740 tx=0x7f7a1000dc00 comp rx=0 tx=0).stop 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 shutdown_connections 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7a040b6670 0x7f7a040b8b80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c109e00 0x7f7a1c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a1c1094b0 0x7f7a1c19a9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 --2- 172.21.15.83:0/2178327715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c103bc0 0x7f7a1c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 >> 172.21.15.83:0/2178327715 conn(0x7f7a1c07b910 msgr2=0x7f7a1c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 shutdown_connections 2024-09-21T03:14:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:52.378+0000 7f7a22bc4640 1 -- 172.21.15.83:0/2178327715 wait complete. 2024-09-21T03:14:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:52 smithi193 ceph-mon[35224]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:52.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:52 smithi193 ceph-mon[35224]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-09-21T03:14:52.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:52 smithi193 ceph-mon[35224]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:14:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:52 smithi142 ceph-mon[34512]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:52 smithi142 ceph-mon[34512]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-09-21T03:14:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:52 smithi142 ceph-mon[34512]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:14:53.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:14:53.307 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:53.677 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:14:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:14:53.263+0000 7fe74a4b4640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T03:14:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 -- 172.21.15.83:0/2737253636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc108d20 msgr2=0x7f30fc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 --2- 172.21.15.83:0/2737253636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc108d20 0x7f30fc10f550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009ca0 tx=0x7f30f402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 -- 172.21.15.83:0/2737253636 shutdown_connections 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 --2- 172.21.15.83:0/2737253636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc108d20 0x7f30fc10f550 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 --2- 172.21.15.83:0/2737253636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 0x7f30fc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 --2- 172.21.15.83:0/2737253636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc102a90 0x7f30fc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.846+0000 7f3102996640 1 -- 172.21.15.83:0/2737253636 >> 172.21.15.83:0/2737253636 conn(0x7f30fc079750 msgr2=0x7f30fc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.847+0000 7f3102996640 1 -- 172.21.15.83:0/2737253636 shutdown_connections 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.847+0000 7f3102996640 1 -- 172.21.15.83:0/2737253636 wait complete. 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.847+0000 7f3102996640 1 Processor -- start 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.847+0000 7f3102996640 1 -- start start 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 0x7f30fc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc108d20 0x7f30fc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc112990 con 0x7f30fc102a90 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc112810 con 0x7f30fc108d20 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc112b10 con 0x7f30fc1083d0 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45990/0 (socket says 172.21.15.83:45990) 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 -- 172.21.15.83:0/2250768082 learned_addr learned my addr 172.21.15.83:0/2250768082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3100f33640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc108d20 0x7f30fc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fb7fe640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 0x7f30fc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 -- 172.21.15.83:0/2250768082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 msgr2=0x7f30fc1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 0x7f30fc1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 -- 172.21.15.83:0/2250768082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc108d20 msgr2=0x7f30fc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc108d20 0x7f30fc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 -- 172.21.15.83:0/2250768082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc19f1f0 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30fbfff640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c920 tx=0x7f30ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec007c70 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ec007e10 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19f4e0 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.848+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec018820 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.849+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a5d00 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.849+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc103d10 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.851+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f30ec00f450 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.852+0000 7f30f97fa640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 0x7f30e40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.852+0000 7f30fb7fe640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 0x7f30e40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.852+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f30ec0e77f0 con 0x7f30fc102a90 2024-09-21T03:14:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.853+0000 7f30fb7fe640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 0x7f30e40b8b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30fc105eb0 tx=0x7f30f000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:53.855+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30ec014030 con 0x7f30fc102a90 2024-09-21T03:14:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.059+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f30fc063800 con 0x7f30e40b6620 2024-09-21T03:14:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.065+0000 7f30f97fa640 1 -- 172.21.15.83:0/2250768082 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f30fc063800 con 0x7f30e40b6620 2024-09-21T03:14:54.066 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/85d61765-43b1-44ea-94d2-792d97d446c9 2024-09-21T03:14:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 msgr2=0x7f30e40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 0x7f30e40b8b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30fc105eb0 tx=0x7f30f000b390 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 msgr2=0x7f30fc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c920 tx=0x7f30ec00cde0 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 shutdown_connections 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30e40b6620 0x7f30e40b8b30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30fc108d20 0x7f30fc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc1083d0 0x7f30fc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 --2- 172.21.15.83:0/2250768082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30fc102a90 0x7f30fc199340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 >> 172.21.15.83:0/2250768082 conn(0x7f30fc079750 msgr2=0x7f30fc1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 shutdown_connections 2024-09-21T03:14:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:54.067+0000 7f3102996640 1 -- 172.21.15.83:0/2250768082 wait complete. 2024-09-21T03:14:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:54 smithi083 ceph-mon[31219]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:54 smithi083 ceph-mon[31219]: pgmap v270: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-21T03:14:54.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:54 smithi193 ceph-mon[35224]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:54 smithi193 ceph-mon[35224]: pgmap v270: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-21T03:14:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:54 smithi142 ceph-mon[34512]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:54 smithi142 ceph-mon[34512]: pgmap v270: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.7 KiB/s wr, 20 op/s 2024-09-21T03:14:54.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:14:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:55.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:55 smithi083 ceph-mon[31219]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:55 smithi193 ceph-mon[35224]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.597+0000 7faf5c610640 1 -- 172.21.15.83:0/3133911166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54102080 msgr2=0x7faf54102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.597+0000 7faf5c610640 1 --2- 172.21.15.83:0/3133911166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54102080 0x7faf54102490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf50009900 tx=0x7faf5002a410 comp rx=0 tx=0).stop 2024-09-21T03:14:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 -- 172.21.15.83:0/3133911166 shutdown_connections 2024-09-21T03:14:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 --2- 172.21.15.83:0/3133911166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109eb0 0x7faf541106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 --2- 172.21.15.83:0/3133911166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54109560 0x7faf54109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 --2- 172.21.15.83:0/3133911166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54102080 0x7faf54102490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 -- 172.21.15.83:0/3133911166 >> 172.21.15.83:0/3133911166 conn(0x7faf54079ec0 msgr2=0x7faf5407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 -- 172.21.15.83:0/3133911166 shutdown_connections 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 -- 172.21.15.83:0/3133911166 wait complete. 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 Processor -- start 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.598+0000 7faf5c610640 1 -- start start 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 0x7faf5419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109560 0x7faf5419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf54113b70 con 0x7faf54109eb0 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf541139f0 con 0x7faf54109560 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf54113cf0 con 0x7faf54102080 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:55.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46004/0 (socket says 172.21.15.83:46004) 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 -- 172.21.15.83:0/57563324 learned_addr learned my addr 172.21.15.83:0/57563324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf59bab640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109560 0x7faf5419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5a3ac640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 0x7faf5419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 -- 172.21.15.83:0/57563324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 msgr2=0x7faf5419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 0x7faf5419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 -- 172.21.15.83:0/57563324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109560 msgr2=0x7faf5419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109560 0x7faf5419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 -- 172.21.15.83:0/57563324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf541a04d0 con 0x7faf54109eb0 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5a3ac640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 0x7faf5419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:14:55.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5abad640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00cad0 tx=0x7faf4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ae20 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf541a07c0 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c005020 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00f6b0 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.599+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a6f20 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.600+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf54102490 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.602+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7faf4c0051d0 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.603+0000 7faf477fe640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 0x7faf400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.603+0000 7faf5a3ac640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 0x7faf400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.603+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7faf4c0e7910 con 0x7faf54109eb0 2024-09-21T03:14:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.603+0000 7faf5a3ac640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 0x7faf400b8b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf50031440 tx=0x7faf50005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.606+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf4c014030 con 0x7faf54109eb0 2024-09-21T03:14:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:55 smithi142 ceph-mon[34512]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.810+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faf5410bce0 con 0x7faf400b65f0 2024-09-21T03:14:55.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.840+0000 7faf477fe640 1 -- 172.21.15.83:0/57563324 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf5410bce0 con 0x7faf400b65f0 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 msgr2=0x7faf400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 0x7faf400b8b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf50031440 tx=0x7faf50005d50 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 msgr2=0x7faf5419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00cad0 tx=0x7faf4c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 shutdown_connections 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faf400b65f0 0x7faf400b8b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf54109eb0 0x7faf5419fda0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf54109560 0x7faf5419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 --2- 172.21.15.83:0/57563324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54102080 0x7faf5419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 >> 172.21.15.83:0/57563324 conn(0x7faf54079ec0 msgr2=0x7faf541075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 shutdown_connections 2024-09-21T03:14:55.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:55.842+0000 7faf5c610640 1 -- 172.21.15.83:0/57563324 wait complete. 2024-09-21T03:14:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:56 smithi193 ceph-mon[35224]: pgmap v271: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T03:14:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:56 smithi083 ceph-mon[31219]: pgmap v271: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T03:14:56.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:14:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:56 smithi142 ceph-mon[34512]: pgmap v271: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T03:14:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:57.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:57 smithi083 ceph-mon[31219]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 -- 172.21.15.83:0/3147932454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac102a60 msgr2=0x7f50ac102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 --2- 172.21.15.83:0/3147932454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac102a60 0x7f50ac102e70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f509c009900 tx=0x7f509c02a410 comp rx=0 tx=0).stop 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 -- 172.21.15.83:0/3147932454 shutdown_connections 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 --2- 172.21.15.83:0/3147932454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac108cf0 0x7f50ac10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 --2- 172.21.15.83:0/3147932454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac1083a0 0x7f50ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 --2- 172.21.15.83:0/3147932454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac102a60 0x7f50ac102e70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 -- 172.21.15.83:0/3147932454 >> 172.21.15.83:0/3147932454 conn(0x7f50ac079730 msgr2=0x7f50ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 -- 172.21.15.83:0/3147932454 shutdown_connections 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.408+0000 7f50b159f640 1 -- 172.21.15.83:0/3147932454 wait complete. 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 Processor -- start 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 -- start start 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 0x7f50ac104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac1083a0 0x7f50ac104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1129e0 con 0x7f50ac108cf0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac112860 con 0x7f50ac102a60 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50b159f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac112b60 con 0x7f50ac1083a0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46016/0 (socket says 172.21.15.83:46016) 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50ab7fe640 1 -- 172.21.15.83:0/157767619 learned_addr learned my addr 172.21.15.83:0/157767619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50aaffd640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 0x7f50ac104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50a3fff640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac1083a0 0x7f50ac104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.409+0000 7f50ab7fe640 1 -- 172.21.15.83:0/157767619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac1083a0 msgr2=0x7f50ac104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50ab7fe640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac1083a0 0x7f50ac104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50ab7fe640 1 -- 172.21.15.83:0/157767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 msgr2=0x7f50ac104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50ab7fe640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 0x7f50ac104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50ab7fe640 1 -- 172.21.15.83:0/157767619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac1a8860 con 0x7f50ac108cf0 2024-09-21T03:14:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50aaffd640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 0x7f50ac104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50ab7fe640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50a400cad0 tx=0x7f50a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a400ae20 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a4005020 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1a8b50 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a400f6b0 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.410+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a90e0 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.411+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac102e70 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.414+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f50a40051d0 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.414+0000 7f50a8ff9640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 0x7f50980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.415+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f50a40e7930 con 0x7f50ac108cf0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.415+0000 7f50aaffd640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 0x7f50980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.415+0000 7f50aaffd640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 0x7f50980b8b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f509c000c00 tx=0x7f509c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.418+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50a4014030 con 0x7f50ac108cf0 2024-09-21T03:14:57.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:57 smithi193 ceph-mon[35224]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:57.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.624+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f50ac06aeb0 con 0x7f50980b6670 2024-09-21T03:14:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.630+0000 7f50a8ff9640 1 -- 172.21.15.83:0/157767619 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f50ac06aeb0 con 0x7f50980b6670 2024-09-21T03:14:57.632 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/5a50105c-b77d-4a93-a145-d0128cc748da 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 msgr2=0x7f50980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 0x7f50980b8b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f509c000c00 tx=0x7f509c031400 comp rx=0 tx=0).stop 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 msgr2=0x7f50ac104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50a400cad0 tx=0x7f50a400cf90 comp rx=0 tx=0).stop 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 shutdown_connections 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50980b6670 0x7f50980b8b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac108cf0 0x7f50ac104cd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50ac1083a0 0x7f50ac104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 --2- 172.21.15.83:0/157767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac102a60 0x7f50ac104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 >> 172.21.15.83:0/157767619 conn(0x7f50ac079730 msgr2=0x7f50ac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 shutdown_connections 2024-09-21T03:14:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:57.632+0000 7f50b159f640 1 -- 172.21.15.83:0/157767619 wait complete. 2024-09-21T03:14:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:57 smithi142 ceph-mon[34512]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:58.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:58 smithi083 ceph-mon[31219]: pgmap v272: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-09-21T03:14:58.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:14:58.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:58 smithi193 ceph-mon[35224]: pgmap v272: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-09-21T03:14:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:14:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:58 smithi142 ceph-mon[34512]: pgmap v272: 97 pgs: 97 active+clean; 591 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-09-21T03:14:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 -- 172.21.15.83:0/3175329310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 msgr2=0x7f9f64107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/3175329310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f64107f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009900 tx=0x7f9f5802a410 comp rx=0 tx=0).stop 2024-09-21T03:14:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 -- 172.21.15.83:0/3175329310 shutdown_connections 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/3175329310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f641084c0 0x7f9f6410ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/3175329310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f64107f80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/3175329310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f64102230 0x7f9f64102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 -- 172.21.15.83:0/3175329310 >> 172.21.15.83:0/3175329310 conn(0x7f9f64079e10 msgr2=0x7f9f6407a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 -- 172.21.15.83:0/3175329310 shutdown_connections 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.016+0000 7f9f6c53e640 1 -- 172.21.15.83:0/3175329310 wait complete. 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 Processor -- start 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 -- start start 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 0x7f9f6419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f641084c0 0x7f9f6419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64111900 con 0x7f9f64107b70 2024-09-21T03:14:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64111780 con 0x7f9f641084c0 2024-09-21T03:14:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.017+0000 7f9f6c53e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64111a80 con 0x7f9f64102230 2024-09-21T03:14:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.018+0000 7f9f69ad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.018+0000 7f9f6a2da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 0x7f9f6419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.018+0000 7f9f6aadb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f641084c0 0x7f9f6419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.018+0000 7f9f69ad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46026/0 (socket says 172.21.15.83:46026) 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.018+0000 7f9f69ad9640 1 -- 172.21.15.83:0/1394725799 learned_addr learned my addr 172.21.15.83:0/1394725799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 -- 172.21.15.83:0/1394725799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f641084c0 msgr2=0x7f9f6419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f641084c0 0x7f9f6419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 -- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 msgr2=0x7f9f6419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 -- 172.21.15.83:0/1394725799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f641a0400 con 0x7f9f64102230 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f69ad9640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6a2da640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 0x7f9f6419a490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e9f0 tx=0x7f9f5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400cd80 con 0x7f9f64102230 2024-09-21T03:14:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f641a06f0 con 0x7f9f64102230 2024-09-21T03:14:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f54011920 con 0x7f9f64102230 2024-09-21T03:14:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54010970 con 0x7f9f64102230 2024-09-21T03:14:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.019+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641a6e50 con 0x7f9f64102230 2024-09-21T03:14:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.020+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f64102640 con 0x7f9f64102230 2024-09-21T03:14:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.022+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9f54011470 con 0x7f9f64102230 2024-09-21T03:14:59.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.023+0000 7f9f637fe640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 0x7f9f4c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:14:59.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.023+0000 7f9f69ad9640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 0x7f9f4c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:14:59.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.023+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f9f540e79e0 con 0x7f9f64102230 2024-09-21T03:14:59.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.023+0000 7f9f69ad9640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 0x7f9f4c0b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f6407a690 tx=0x7f9f58032ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:14:59.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.026+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f54016030 con 0x7f9f64102230 2024-09-21T03:14:59.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.241+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6419b5c0 con 0x7f9f64102230 2024-09-21T03:14:59.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.242+0000 7f9f637fe640 1 -- 172.21.15.83:0/1394725799 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f9f540ae0d0 con 0x7f9f64102230 2024-09-21T03:14:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:14:59.244 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:14:59.244 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 msgr2=0x7f9f4c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 0x7f9f4c0b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f6407a690 tx=0x7f9f58032ed0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 msgr2=0x7f9f6419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 0x7f9f6419a490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e9f0 tx=0x7f9f5400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 shutdown_connections 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9f4c0b65f0 0x7f9f4c0b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f641084c0 0x7f9f6419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f64107b70 0x7f9f6419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 --2- 172.21.15.83:0/1394725799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f64102230 0x7f9f6419a490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:14:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.244+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 >> 172.21.15.83:0/1394725799 conn(0x7f9f64079e10 msgr2=0x7f9f64103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:14:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.245+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 shutdown_connections 2024-09-21T03:14:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:14:59.245+0000 7f9f6c53e640 1 -- 172.21.15.83:0/1394725799 wait complete. 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:14:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T03:14:59.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:14:59.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T03:14:59.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:14:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1394725799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T03:14:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:14:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1394725799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:14:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T03:14:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T03:14:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:14:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T03:14:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-21T03:14:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]} : dispatch 2024-09-21T03:14:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:14:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1394725799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:14:59.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:15:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: pgmap v273: 97 pgs: 97 active+clean; 627 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 24 op/s 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T03:15:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:00 smithi193 ceph-mon[35224]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: pgmap v273: 97 pgs: 97 active+clean; 627 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 24 op/s 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T03:15:00.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:00 smithi083 ceph-mon[31219]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:15:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: pgmap v273: 97 pgs: 97 active+clean; 627 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 24 op/s 2024-09-21T03:15:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T03:15:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T03:15:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T03:15:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:15:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T03:15:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-21T03:15:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-21T03:15:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:00 smithi142 ceph-mon[34512]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:15:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 -- 172.21.15.83:0/1508658416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 msgr2=0x7fa298103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 --2- 172.21.15.83:0/1508658416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa298103600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa288009900 tx=0x7fa28802a410 comp rx=0 tx=0).stop 2024-09-21T03:15:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 -- 172.21.15.83:0/1508658416 shutdown_connections 2024-09-21T03:15:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 --2- 172.21.15.83:0/1508658416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 --2- 172.21.15.83:0/1508658416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 0x7fa298108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 --2- 172.21.15.83:0/1508658416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa298103600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 -- 172.21.15.83:0/1508658416 >> 172.21.15.83:0/1508658416 conn(0x7fa298079ec0 msgr2=0x7fa29807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 -- 172.21.15.83:0/1508658416 shutdown_connections 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.668+0000 7fa29c973640 1 -- 172.21.15.83:0/1508658416 wait complete. 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 Processor -- start 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 -- start start 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 0x7fa29819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa298112a70 con 0x7fa2981031f0 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2981128f0 con 0x7fa298108460 2024-09-21T03:15:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29c973640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa298112bf0 con 0x7fa298108db0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57494/0 (socket says 172.21.15.83:57494) 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.669+0000 7fa29659c640 1 -- 172.21.15.83:0/4183030260 learned_addr learned my addr 172.21.15.83:0/4183030260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa28dd9b640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 0x7fa29819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa296d9d640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 -- 172.21.15.83:0/4183030260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 msgr2=0x7fa29819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 -- 172.21.15.83:0/4183030260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 msgr2=0x7fa29819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 0x7fa29819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 -- 172.21.15.83:0/4183030260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2981a65f0 con 0x7fa2981031f0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa296d9d640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29659c640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa288005940 tx=0x7fa2880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28803f030 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2981a6880 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2880044c0 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288035070 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.670+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2981a6d60 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.671+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa298103600 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.673+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa288042030 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.673+0000 7fa28ffff640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 0x7fa2840b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.673+0000 7fa28dd9b640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 0x7fa2840b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.673+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa288107b90 con 0x7fa2981031f0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.674+0000 7fa28dd9b640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 0x7fa2840b8b30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa298108190 tx=0x7fa280005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.676+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2880ce220 con 0x7fa2981031f0 2024-09-21T03:15:00.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.858+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa298102260 con 0x7fa2981031f0 2024-09-21T03:15:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.860+0000 7fa28ffff640 1 -- 172.21.15.83:0/4183030260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7fa2880d2600 con 0x7fa2981031f0 2024-09-21T03:15:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:15:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 msgr2=0x7fa2840b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 0x7fa2840b8b30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa298108190 tx=0x7fa280005f90 comp rx=0 tx=0).stop 2024-09-21T03:15:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 msgr2=0x7fa29819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa288005940 tx=0x7fa2880040c0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 shutdown_connections 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2840b6620 0x7fa2840b8b30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa298108db0 0x7fa29819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa298108460 0x7fa29819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 --2- 172.21.15.83:0/4183030260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2981031f0 0x7fa29819b360 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 >> 172.21.15.83:0/4183030260 conn(0x7fa298079ec0 msgr2=0x7fa2981064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 shutdown_connections 2024-09-21T03:15:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:00.862+0000 7fa29c973640 1 -- 172.21.15.83:0/4183030260 wait complete. 2024-09-21T03:15:01.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:01 smithi083 ceph-mon[31219]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:15:01.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:01 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4183030260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:01.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:01 smithi193 ceph-mon[35224]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:15:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:01 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4183030260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:01.579 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:15:01.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:01 smithi142 ceph-mon[34512]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:15:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:01 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4183030260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:01.826 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- 172.21.15.83:0/48572255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4102820 msgr2=0x7fb4a4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/48572255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4102820 0x7fb4a4102c30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb498009930 tx=0x7fb49802a420 comp rx=0 tx=0).stop 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- 172.21.15.83:0/48572255 shutdown_connections 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/48572255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4107c60 0x7fb4a4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/48572255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4103170 0x7fb4a41076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/48572255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4102820 0x7fb4a4102c30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- 172.21.15.83:0/48572255 >> 172.21.15.83:0/48572255 conn(0x7fb4a4079770 msgr2=0x7fb4a4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- 172.21.15.83:0/48572255 shutdown_connections 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- 172.21.15.83:0/48572255 wait complete. 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 Processor -- start 2024-09-21T03:15:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.328+0000 7fb4ab12a640 1 -- start start 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102820 0x7fb4a41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4103170 0x7fb4a41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a411a110 con 0x7fb4a4107c60 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4119f90 con 0x7fb4a4103170 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4ab12a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a411a290 con 0x7fb4a4102820 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57520/0 (socket says 172.21.15.83:57520) 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 -- 172.21.15.83:0/3936253977 learned_addr learned my addr 172.21.15.83:0/3936253977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a3fff640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4103170 0x7fb4a41a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a8ec6640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102820 0x7fb4a41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 -- 172.21.15.83:0/3936253977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102820 msgr2=0x7fb4a41a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102820 0x7fb4a41a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 -- 172.21.15.83:0/3936253977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4103170 msgr2=0x7fb4a41a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4103170 0x7fb4a41a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 -- 172.21.15.83:0/3936253977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a41a6b10 con 0x7fb4a4107c60 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a96c7640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00eaa0 tx=0x7fb49c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.329+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00cdb0 con 0x7fb4a4107c60 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.330+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a41a6e00 con 0x7fb4a4107c60 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.330+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c005020 con 0x7fb4a4107c60 2024-09-21T03:15:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.330+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0116d0 con 0x7fb4a4107c60 2024-09-21T03:15:02.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.330+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a41ad560 con 0x7fb4a4107c60 2024-09-21T03:15:02.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.331+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a406c260 con 0x7fb4a4107c60 2024-09-21T03:15:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.332+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb49c0051d0 con 0x7fb4a4107c60 2024-09-21T03:15:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.333+0000 7fb4a1ffb640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.333+0000 7fb4a8ec6640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.333+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb49c0e7b60 con 0x7fb4a4107c60 2024-09-21T03:15:02.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.333+0000 7fb4a8ec6640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 0x7fb4940b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:02.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.336+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb49c016030 con 0x7fb4a4107c60 2024-09-21T03:15:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.509+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4a4106d90 con 0x7fb4940b65f0 2024-09-21T03:15:02.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:02 smithi083 ceph-mon[31219]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 628 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-21T03:15:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.514+0000 7fb4a1ffb640 1 -- 172.21.15.83:0/3936253977 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258865 (secure 0 0 0) 0x7fb4a4106d90 con 0x7fb4940b65f0 2024-09-21T03:15:02.518 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:02.519 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 msgr2=0x7fb4940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 0x7fb4940b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498031400 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 msgr2=0x7fb4a41a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00eaa0 tx=0x7fb49c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 shutdown_connections 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a4107c60 0x7fb4a41a63e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a4103170 0x7fb4a41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 --2- 172.21.15.83:0/3936253977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a4102820 0x7fb4a41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 >> 172.21.15.83:0/3936253977 conn(0x7fb4a4079770 msgr2=0x7fb4a4105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 shutdown_connections 2024-09-21T03:15:02.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:02.517+0000 7fb4ab12a640 1 -- 172.21.15.83:0/3936253977 wait complete. 2024-09-21T03:15:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:02 smithi193 ceph-mon[35224]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 628 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-21T03:15:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:02 smithi142 ceph-mon[34512]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 628 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-21T03:15:03.206 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-21T03:15:01.173985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642618,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":213,"num_read_kb":190,"num_write":352,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351088,"kb_used_data":10152,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590736,"statfs":{"total":1151940427776,"available":1151580913664,"internally_reserved":0,"allocated":10395648,"data_stored":6717374,"data_compressed":113345,"data_compressed_allocated":143360,"data_compressed_original":1785856,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37225,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":62,"num_write":18,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.619027+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:14:37.619027+0000","last_peered":"2024-09-21T03:14:37.619027+0000","last_clean":"2024-09-21T03:14:37.619027+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:14:37.619027+0000","last_undegraded":"2024-09-21T03:14:37.619027+0000","last_fullsized":"2024-09-21T03:14:37.619027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612598+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:14:37.612598+0000","last_peered":"2024-09-21T03:14:37.612598+0000","last_clean":"2024-09-21T03:14:37.612598+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:14:37.612598+0000","last_undegraded":"2024-09-21T03:14:37.612598+0000","last_fullsized":"2024-09-21T03:14:37.612598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612501+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:14:37.612501+0000","last_peered":"2024-09-21T03:14:37.612501+0000","last_clean":"2024-09-21T03:14:37.612501+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:14:37.612501+0000","last_undegraded":"2024-09-21T03:14:37.612501+0000","last_fullsized":"2024-09-21T03:14:37.612501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.614073+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:14:37.614073+0000","last_peered":"2024-09-21T03:14:37.614073+0000","last_clean":"2024-09-21T03:14:37.614073+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:14:37.614073+0000","last_undegraded":"2024-09-21T03:14:37.614073+0000","last_fullsized":"2024-09-21T03:14:37.614073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.602260+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:14:37.602260+0000","last_peered":"2024-09-21T03:14:37.602260+0000","last_clean":"2024-09-21T03:14:37.602260+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:14:37.602260+0000","last_undegraded":"2024-09-21T03:14:37.602260+0000","last_fullsized":"2024-09-21T03:14:37.602260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261516+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:14:59.261516+0000","last_peered":"2024-09-21T03:14:59.261516+0000","last_clean":"2024-09-21T03:14:59.261516+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:14:59.261516+0000","last_undegraded":"2024-09-21T03:14:59.261516+0000","last_fullsized":"2024-09-21T03:14:59.261516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609145+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:14:37.609145+0000","last_peered":"2024-09-21T03:14:37.609145+0000","last_clean":"2024-09-21T03:14:37.609145+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:14:37.609145+0000","last_undegraded":"2024-09-21T03:14:37.609145+0000","last_fullsized":"2024-09-21T03:14:37.609145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609110+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:14:37.609110+0000","last_peered":"2024-09-21T03:14:37.609110+0000","last_clean":"2024-09-21T03:14:37.609110+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:14:37.609110+0000","last_undegraded":"2024-09-21T03:14:37.609110+0000","last_fullsized":"2024-09-21T03:14:37.609110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.602861+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:14:37.602861+0000","last_peered":"2024-09-21T03:14:37.602861+0000","last_clean":"2024-09-21T03:14:37.602861+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:14:37.602861+0000","last_undegraded":"2024-09-21T03:14:37.602861+0000","last_fullsized":"2024-09-21T03:14:37.602861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261730+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:14:59.261730+0000","last_peered":"2024-09-21T03:14:59.261730+0000","last_clean":"2024-09-21T03:14:59.261730+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:14:59.261730+0000","last_undegraded":"2024-09-21T03:14:59.261730+0000","last_fullsized":"2024-09-21T03:14:59.261730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.614105+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:14:37.614105+0000","last_peered":"2024-09-21T03:14:37.614105+0000","last_clean":"2024-09-21T03:14:37.614105+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:14:37.614105+0000","last_undegraded":"2024-09-21T03:14:37.614105+0000","last_fullsized":"2024-09-21T03:14:37.614105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599499+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:14:37.599499+0000","last_peered":"2024-09-21T03:14:37.599499+0000","last_clean":"2024-09-21T03:14:37.599499+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:14:37.599499+0000","last_undegraded":"2024-09-21T03:14:37.599499+0000","last_fullsized":"2024-09-21T03:14:37.599499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612477+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:14:37.612477+0000","last_peered":"2024-09-21T03:14:37.612477+0000","last_clean":"2024-09-21T03:14:37.612477+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:14:37.612477+0000","last_undegraded":"2024-09-21T03:14:37.612477+0000","last_fullsized":"2024-09-21T03:14:37.612477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612344+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:14:37.612344+0000","last_peered":"2024-09-21T03:14:37.612344+0000","last_clean":"2024-09-21T03:14:37.612344+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:14:37.612344+0000","last_undegraded":"2024-09-21T03:14:37.612344+0000","last_fullsized":"2024-09-21T03:14:37.612344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.608444+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:14:37.608444+0000","last_peered":"2024-09-21T03:14:37.608444+0000","last_clean":"2024-09-21T03:14:37.608444+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:14:37.608444+0000","last_undegraded":"2024-09-21T03:14:37.608444+0000","last_fullsized":"2024-09-21T03:14:37.608444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.619289+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:14:37.619289+0000","last_peered":"2024-09-21T03:14:37.619289+0000","last_clean":"2024-09-21T03:14:37.619289+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:14:37.619289+0000","last_undegraded":"2024-09-21T03:14:37.619289+0000","last_fullsized":"2024-09-21T03:14:37.619289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.611651+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:14:37.611651+0000","last_peered":"2024-09-21T03:14:37.611651+0000","last_clean":"2024-09-21T03:14:37.611651+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:14:37.611651+0000","last_undegraded":"2024-09-21T03:14:37.611651+0000","last_fullsized":"2024-09-21T03:14:37.611651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.600172+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:14:37.600172+0000","last_peered":"2024-09-21T03:14:37.600172+0000","last_clean":"2024-09-21T03:14:37.600172+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:14:37.600172+0000","last_undegraded":"2024-09-21T03:14:37.600172+0000","last_fullsized":"2024-09-21T03:14:37.600172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.618427+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:14:37.618427+0000","last_peered":"2024-09-21T03:14:37.618427+0000","last_clean":"2024-09-21T03:14:37.618427+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:14:37.618427+0000","last_undegraded":"2024-09-21T03:14:37.618427+0000","last_fullsized":"2024-09-21T03:14:37.618427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599328+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:14:37.599328+0000","last_peered":"2024-09-21T03:14:37.599328+0000","last_clean":"2024-09-21T03:14:37.599328+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:14:37.599328+0000","last_undegraded":"2024-09-21T03:14:37.599328+0000","last_fullsized":"2024-09-21T03:14:37.599328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.608827+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:14:37.608827+0000","last_peered":"2024-09-21T03:14:37.608827+0000","last_clean":"2024-09-21T03:14:37.608827+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:14:37.608827+0000","last_undegraded":"2024-09-21T03:14:37.608827+0000","last_fullsized":"2024-09-21T03:14:37.608827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609065+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:14:37.609065+0000","last_peered":"2024-09-21T03:14:37.609065+0000","last_clean":"2024-09-21T03:14:37.609065+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:14:37.609065+0000","last_undegraded":"2024-09-21T03:14:37.609065+0000","last_fullsized":"2024-09-21T03:14:37.609065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613252+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:14:37.613252+0000","last_peered":"2024-09-21T03:14:37.613252+0000","last_clean":"2024-09-21T03:14:37.613252+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:14:37.613252+0000","last_undegraded":"2024-09-21T03:14:37.613252+0000","last_fullsized":"2024-09-21T03:14:37.613252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610245+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:14:37.610245+0000","last_peered":"2024-09-21T03:14:37.610245+0000","last_clean":"2024-09-21T03:14:37.610245+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:14:37.610245+0000","last_undegraded":"2024-09-21T03:14:37.610245+0000","last_fullsized":"2024-09-21T03:14:37.610245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.600275+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:14:37.600275+0000","last_peered":"2024-09-21T03:14:37.600275+0000","last_clean":"2024-09-21T03:14:37.600275+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:14:37.600275+0000","last_undegraded":"2024-09-21T03:14:37.600275+0000","last_fullsized":"2024-09-21T03:14:37.600275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.618520+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:14:37.618520+0000","last_peered":"2024-09-21T03:14:37.618520+0000","last_clean":"2024-09-21T03:14:37.618520+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:14:37.618520+0000","last_undegraded":"2024-09-21T03:14:37.618520+0000","last_fullsized":"2024-09-21T03:14:37.618520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599395+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:14:37.599395+0000","last_peered":"2024-09-21T03:14:37.599395+0000","last_clean":"2024-09-21T03:14:37.599395+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:14:37.599395+0000","last_undegraded":"2024-09-21T03:14:37.599395+0000","last_fullsized":"2024-09-21T03:14:37.599395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612577+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:14:37.612577+0000","last_peered":"2024-09-21T03:14:37.612577+0000","last_clean":"2024-09-21T03:14:37.612577+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:14:37.612577+0000","last_undegraded":"2024-09-21T03:14:37.612577+0000","last_fullsized":"2024-09-21T03:14:37.612577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599733+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:14:37.599733+0000","last_peered":"2024-09-21T03:14:37.599733+0000","last_clean":"2024-09-21T03:14:37.599733+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:14:37.599733+0000","last_undegraded":"2024-09-21T03:14:37.599733+0000","last_fullsized":"2024-09-21T03:14:37.599733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.600225+0000","last_change":"2024-09-21T03:14:36.619313+0000","last_active":"2024-09-21T03:14:37.600225+0000","last_peered":"2024-09-21T03:14:37.600225+0000","last_clean":"2024-09-21T03:14:37.600225+0000","last_became_active":"2024-09-21T03:14:36.618846+0000","last_became_peered":"2024-09-21T03:14:36.618846+0000","last_unstale":"2024-09-21T03:14:37.600225+0000","last_undegraded":"2024-09-21T03:14:37.600225+0000","last_fullsized":"2024-09-21T03:14:37.600225+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.298482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.617679+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:14:37.617679+0000","last_peered":"2024-09-21T03:14:37.617679+0000","last_clean":"2024-09-21T03:14:37.617679+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:14:37.617679+0000","last_undegraded":"2024-09-21T03:14:37.617679+0000","last_fullsized":"2024-09-21T03:14:37.617679+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:57.778835+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:14:57.778835+0000","last_peered":"2024-09-21T03:14:57.778835+0000","last_clean":"2024-09-21T03:14:57.778835+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:14:57.778835+0000","last_undegraded":"2024-09-21T03:14:57.778835+0000","last_fullsized":"2024-09-21T03:14:57.778835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613437+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:14:37.613437+0000","last_peered":"2024-09-21T03:14:37.613437+0000","last_clean":"2024-09-21T03:14:37.613437+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:14:37.613437+0000","last_undegraded":"2024-09-21T03:14:37.613437+0000","last_fullsized":"2024-09-21T03:14:37.613437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:46.854490+0000","last_change":"2024-09-21T03:14:34.595602+0000","last_active":"2024-09-21T03:14:46.854490+0000","last_peered":"2024-09-21T03:14:46.854490+0000","last_clean":"2024-09-21T03:14:46.854490+0000","last_became_active":"2024-09-21T03:14:34.595205+0000","last_became_peered":"2024-09-21T03:14:34.595205+0000","last_unstale":"2024-09-21T03:14:46.854490+0000","last_undegraded":"2024-09-21T03:14:46.854490+0000","last_fullsized":"2024-09-21T03:14:46.854490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:19.594684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:45.844831+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:14:45.844831+0000","last_peered":"2024-09-21T03:14:45.844831+0000","last_clean":"2024-09-21T03:14:45.844831+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:14:45.844831+0000","last_undegraded":"2024-09-21T03:14:45.844831+0000","last_fullsized":"2024-09-21T03:14:45.844831+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609387+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:14:37.609387+0000","last_peered":"2024-09-21T03:14:37.609387+0000","last_clean":"2024-09-21T03:14:37.609387+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:14:37.609387+0000","last_undegraded":"2024-09-21T03:14:37.609387+0000","last_fullsized":"2024-09-21T03:14:37.609387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.867886+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:14:47.867886+0000","last_peered":"2024-09-21T03:14:47.867886+0000","last_clean":"2024-09-21T03:14:47.867886+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:14:47.867886+0000","last_undegraded":"2024-09-21T03:14:47.867886+0000","last_fullsized":"2024-09-21T03:14:47.867886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.611391+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:14:37.611391+0000","last_peered":"2024-09-21T03:14:37.611391+0000","last_clean":"2024-09-21T03:14:37.611391+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:14:37.611391+0000","last_undegraded":"2024-09-21T03:14:37.611391+0000","last_fullsized":"2024-09-21T03:14:37.611391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:56.776554+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:14:56.776554+0000","last_peered":"2024-09-21T03:14:56.776554+0000","last_clean":"2024-09-21T03:14:56.776554+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:14:56.776554+0000","last_undegraded":"2024-09-21T03:14:56.776554+0000","last_fullsized":"2024-09-21T03:14:56.776554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609114+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:14:37.609114+0000","last_peered":"2024-09-21T03:14:37.609114+0000","last_clean":"2024-09-21T03:14:37.609114+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:14:37.609114+0000","last_undegraded":"2024-09-21T03:14:37.609114+0000","last_fullsized":"2024-09-21T03:14:37.609114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261577+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:14:59.261577+0000","last_peered":"2024-09-21T03:14:59.261577+0000","last_clean":"2024-09-21T03:14:59.261577+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:14:59.261577+0000","last_undegraded":"2024-09-21T03:14:59.261577+0000","last_fullsized":"2024-09-21T03:14:59.261577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612929+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:14:37.612929+0000","last_peered":"2024-09-21T03:14:37.612929+0000","last_clean":"2024-09-21T03:14:37.612929+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:14:37.612929+0000","last_undegraded":"2024-09-21T03:14:37.612929+0000","last_fullsized":"2024-09-21T03:14:37.612929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:46.853121+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:14:46.853121+0000","last_peered":"2024-09-21T03:14:46.853121+0000","last_clean":"2024-09-21T03:14:46.853121+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:14:46.853121+0000","last_undegraded":"2024-09-21T03:14:46.853121+0000","last_fullsized":"2024-09-21T03:14:46.853121+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.608706+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:14:37.608706+0000","last_peered":"2024-09-21T03:14:37.608706+0000","last_clean":"2024-09-21T03:14:37.608706+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:14:37.608706+0000","last_undegraded":"2024-09-21T03:14:37.608706+0000","last_fullsized":"2024-09-21T03:14:37.608706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'23","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:56.779147+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:14:56.779147+0000","last_peered":"2024-09-21T03:14:56.779147+0000","last_clean":"2024-09-21T03:14:56.779147+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:14:56.779147+0000","last_undegraded":"2024-09-21T03:14:56.779147+0000","last_fullsized":"2024-09-21T03:14:56.779147+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":55,"num_write":25,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:48.761901+0000","last_change":"2024-09-21T03:14:34.592814+0000","last_active":"2024-09-21T03:14:48.761901+0000","last_peered":"2024-09-21T03:14:48.761901+0000","last_clean":"2024-09-21T03:14:48.761901+0000","last_became_active":"2024-09-21T03:14:34.592384+0000","last_became_peered":"2024-09-21T03:14:34.592384+0000","last_unstale":"2024-09-21T03:14:48.761901+0000","last_undegraded":"2024-09-21T03:14:48.761901+0000","last_fullsized":"2024-09-21T03:14:48.761901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:05.118577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599152+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:14:37.599152+0000","last_peered":"2024-09-21T03:14:37.599152+0000","last_clean":"2024-09-21T03:14:37.599152+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:14:37.599152+0000","last_undegraded":"2024-09-21T03:14:37.599152+0000","last_fullsized":"2024-09-21T03:14:37.599152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.868082+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:14:47.868082+0000","last_peered":"2024-09-21T03:14:47.868082+0000","last_clean":"2024-09-21T03:14:47.868082+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:14:47.868082+0000","last_undegraded":"2024-09-21T03:14:47.868082+0000","last_fullsized":"2024-09-21T03:14:47.868082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261836+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:14:59.261836+0000","last_peered":"2024-09-21T03:14:59.261836+0000","last_clean":"2024-09-21T03:14:59.261836+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:14:59.261836+0000","last_undegraded":"2024-09-21T03:14:59.261836+0000","last_fullsized":"2024-09-21T03:14:59.261836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":36,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:56.777287+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:14:56.777287+0000","last_peered":"2024-09-21T03:14:56.777287+0000","last_clean":"2024-09-21T03:14:56.777287+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:14:56.777287+0000","last_undegraded":"2024-09-21T03:14:56.777287+0000","last_fullsized":"2024-09-21T03:14:56.777287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262027+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:14:59.262027+0000","last_peered":"2024-09-21T03:14:59.262027+0000","last_clean":"2024-09-21T03:14:59.262027+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:14:59.262027+0000","last_undegraded":"2024-09-21T03:14:59.262027+0000","last_fullsized":"2024-09-21T03:14:59.262027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612590+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:14:37.612590+0000","last_peered":"2024-09-21T03:14:37.612590+0000","last_clean":"2024-09-21T03:14:37.612590+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:14:37.612590+0000","last_undegraded":"2024-09-21T03:14:37.612590+0000","last_fullsized":"2024-09-21T03:14:37.612590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.618109+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:14:37.618109+0000","last_peered":"2024-09-21T03:14:37.618109+0000","last_clean":"2024-09-21T03:14:37.618109+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:14:37.618109+0000","last_undegraded":"2024-09-21T03:14:37.618109+0000","last_fullsized":"2024-09-21T03:14:37.618109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.866951+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:14:47.866951+0000","last_peered":"2024-09-21T03:14:47.866951+0000","last_clean":"2024-09-21T03:14:47.866951+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:14:47.866951+0000","last_undegraded":"2024-09-21T03:14:47.866951+0000","last_fullsized":"2024-09-21T03:14:47.866951+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261784+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:14:59.261784+0000","last_peered":"2024-09-21T03:14:59.261784+0000","last_clean":"2024-09-21T03:14:59.261784+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:14:59.261784+0000","last_undegraded":"2024-09-21T03:14:59.261784+0000","last_fullsized":"2024-09-21T03:14:59.261784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.868479+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:14:47.868479+0000","last_peered":"2024-09-21T03:14:47.868479+0000","last_clean":"2024-09-21T03:14:47.868479+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:14:47.868479+0000","last_undegraded":"2024-09-21T03:14:47.868479+0000","last_fullsized":"2024-09-21T03:14:47.868479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"peering","last_fresh":"2024-09-21T03:14:55.835362+0000","last_change":"2024-09-21T03:14:59.262045+0000","last_active":"2024-09-21T03:14:59.262045+0000","last_peered":"2024-09-21T03:14:55.835362+0000","last_clean":"2024-09-21T03:14:55.835362+0000","last_became_active":"2024-09-21T03:14:36.620675+0000","last_became_peered":"2024-09-21T03:14:36.620675+0000","last_unstale":"2024-09-21T03:14:55.835362+0000","last_undegraded":"2024-09-21T03:14:55.835362+0000","last_fullsized":"2024-09-21T03:14:55.835362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:55.775763+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:14:55.775763+0000","last_peered":"2024-09-21T03:14:55.775763+0000","last_clean":"2024-09-21T03:14:55.775763+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:14:55.775763+0000","last_undegraded":"2024-09-21T03:14:55.775763+0000","last_fullsized":"2024-09-21T03:14:55.775763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261861+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:14:59.261861+0000","last_peered":"2024-09-21T03:14:59.261861+0000","last_clean":"2024-09-21T03:14:59.261861+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:14:59.261861+0000","last_undegraded":"2024-09-21T03:14:59.261861+0000","last_fullsized":"2024-09-21T03:14:59.261861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262217+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:14:59.262217+0000","last_peered":"2024-09-21T03:14:59.262217+0000","last_clean":"2024-09-21T03:14:59.262217+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:14:59.262217+0000","last_undegraded":"2024-09-21T03:14:59.262217+0000","last_fullsized":"2024-09-21T03:14:59.262217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:48.763112+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:14:48.763112+0000","last_peered":"2024-09-21T03:14:48.763112+0000","last_clean":"2024-09-21T03:14:48.763112+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:14:48.763112+0000","last_undegraded":"2024-09-21T03:14:48.763112+0000","last_fullsized":"2024-09-21T03:14:48.763112+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609578+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:14:37.609578+0000","last_peered":"2024-09-21T03:14:37.609578+0000","last_clean":"2024-09-21T03:14:37.609578+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:14:37.609578+0000","last_undegraded":"2024-09-21T03:14:37.609578+0000","last_fullsized":"2024-09-21T03:14:37.609578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262234+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:14:59.262234+0000","last_peered":"2024-09-21T03:14:59.262234+0000","last_clean":"2024-09-21T03:14:59.262234+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:14:59.262234+0000","last_undegraded":"2024-09-21T03:14:59.262234+0000","last_fullsized":"2024-09-21T03:14:59.262234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599057+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:14:37.599057+0000","last_peered":"2024-09-21T03:14:37.599057+0000","last_clean":"2024-09-21T03:14:37.599057+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:14:37.599057+0000","last_undegraded":"2024-09-21T03:14:37.599057+0000","last_fullsized":"2024-09-21T03:14:37.599057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:54.774122+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:14:54.774122+0000","last_peered":"2024-09-21T03:14:54.774122+0000","last_clean":"2024-09-21T03:14:54.774122+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:14:54.774122+0000","last_undegraded":"2024-09-21T03:14:54.774122+0000","last_fullsized":"2024-09-21T03:14:54.774122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612922+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:14:37.612922+0000","last_peered":"2024-09-21T03:14:37.612922+0000","last_clean":"2024-09-21T03:14:37.612922+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:14:37.612922+0000","last_undegraded":"2024-09-21T03:14:37.612922+0000","last_fullsized":"2024-09-21T03:14:37.612922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262288+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:14:59.262288+0000","last_peered":"2024-09-21T03:14:59.262288+0000","last_clean":"2024-09-21T03:14:59.262288+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:14:59.262288+0000","last_undegraded":"2024-09-21T03:14:59.262288+0000","last_fullsized":"2024-09-21T03:14:59.262288+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.602357+0000","last_change":"2024-09-21T03:14:36.624084+0000","last_active":"2024-09-21T03:14:37.602357+0000","last_peered":"2024-09-21T03:14:37.602357+0000","last_clean":"2024-09-21T03:14:37.602357+0000","last_became_active":"2024-09-21T03:14:36.623492+0000","last_became_peered":"2024-09-21T03:14:36.623492+0000","last_unstale":"2024-09-21T03:14:37.602357+0000","last_undegraded":"2024-09-21T03:14:37.602357+0000","last_fullsized":"2024-09-21T03:14:37.602357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:48.864575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.611846+0000","last_change":"2024-09-21T03:14:35.075737+0000","last_active":"2024-09-21T03:14:37.611846+0000","last_peered":"2024-09-21T03:14:37.611846+0000","last_clean":"2024-09-21T03:14:37.611846+0000","last_became_active":"2024-09-21T03:14:35.075215+0000","last_became_peered":"2024-09-21T03:14:35.075215+0000","last_unstale":"2024-09-21T03:14:37.611846+0000","last_undegraded":"2024-09-21T03:14:37.611846+0000","last_fullsized":"2024-09-21T03:14:37.611846+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:20:22.255850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.602609+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:14:37.602609+0000","last_peered":"2024-09-21T03:14:37.602609+0000","last_clean":"2024-09-21T03:14:37.602609+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:14:37.602609+0000","last_undegraded":"2024-09-21T03:14:37.602609+0000","last_fullsized":"2024-09-21T03:14:37.602609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.599621+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:14:37.599621+0000","last_peered":"2024-09-21T03:14:37.599621+0000","last_clean":"2024-09-21T03:14:37.599621+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:14:37.599621+0000","last_undegraded":"2024-09-21T03:14:37.599621+0000","last_fullsized":"2024-09-21T03:14:37.599621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:53.605501+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:14:53.605501+0000","last_peered":"2024-09-21T03:14:53.605501+0000","last_clean":"2024-09-21T03:14:53.605501+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:14:53.605501+0000","last_undegraded":"2024-09-21T03:14:53.605501+0000","last_fullsized":"2024-09-21T03:14:53.605501+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610283+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:14:37.610283+0000","last_peered":"2024-09-21T03:14:37.610283+0000","last_clean":"2024-09-21T03:14:37.610283+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:14:37.610283+0000","last_undegraded":"2024-09-21T03:14:37.610283+0000","last_fullsized":"2024-09-21T03:14:37.610283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.869249+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:14:47.869249+0000","last_peered":"2024-09-21T03:14:47.869249+0000","last_clean":"2024-09-21T03:14:47.869249+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:14:47.869249+0000","last_undegraded":"2024-09-21T03:14:47.869249+0000","last_fullsized":"2024-09-21T03:14:47.869249+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612799+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:14:37.612799+0000","last_peered":"2024-09-21T03:14:37.612799+0000","last_clean":"2024-09-21T03:14:37.612799+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:14:37.612799+0000","last_undegraded":"2024-09-21T03:14:37.612799+0000","last_fullsized":"2024-09-21T03:14:37.612799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:44.836014+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:14:44.836014+0000","last_peered":"2024-09-21T03:14:44.836014+0000","last_clean":"2024-09-21T03:14:44.836014+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:14:44.836014+0000","last_undegraded":"2024-09-21T03:14:44.836014+0000","last_fullsized":"2024-09-21T03:14:44.836014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.867743+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:14:47.867743+0000","last_peered":"2024-09-21T03:14:47.867743+0000","last_clean":"2024-09-21T03:14:47.867743+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:14:47.867743+0000","last_undegraded":"2024-09-21T03:14:47.867743+0000","last_fullsized":"2024-09-21T03:14:47.867743+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.618140+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:14:37.618140+0000","last_peered":"2024-09-21T03:14:37.618140+0000","last_clean":"2024-09-21T03:14:37.618140+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:14:37.618140+0000","last_undegraded":"2024-09-21T03:14:37.618140+0000","last_fullsized":"2024-09-21T03:14:37.618140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:46.856165+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:14:46.856165+0000","last_peered":"2024-09-21T03:14:46.856165+0000","last_clean":"2024-09-21T03:14:46.856165+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:14:46.856165+0000","last_undegraded":"2024-09-21T03:14:46.856165+0000","last_fullsized":"2024-09-21T03:14:46.856165+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610006+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:14:37.610006+0000","last_peered":"2024-09-21T03:14:37.610006+0000","last_clean":"2024-09-21T03:14:37.610006+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:14:37.610006+0000","last_undegraded":"2024-09-21T03:14:37.610006+0000","last_fullsized":"2024-09-21T03:14:37.610006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262017+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:14:59.262017+0000","last_peered":"2024-09-21T03:14:59.262017+0000","last_clean":"2024-09-21T03:14:59.262017+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:14:59.262017+0000","last_undegraded":"2024-09-21T03:14:59.262017+0000","last_fullsized":"2024-09-21T03:14:59.262017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613567+0000","last_change":"2024-09-21T03:14:36.621002+0000","last_active":"2024-09-21T03:14:37.613567+0000","last_peered":"2024-09-21T03:14:37.613567+0000","last_clean":"2024-09-21T03:14:37.613567+0000","last_became_active":"2024-09-21T03:14:36.620640+0000","last_became_peered":"2024-09-21T03:14:36.620640+0000","last_unstale":"2024-09-21T03:14:37.613567+0000","last_undegraded":"2024-09-21T03:14:37.613567+0000","last_fullsized":"2024-09-21T03:14:37.613567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.471195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.608477+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:14:37.608477+0000","last_peered":"2024-09-21T03:14:37.608477+0000","last_clean":"2024-09-21T03:14:37.608477+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:14:37.608477+0000","last_undegraded":"2024-09-21T03:14:37.608477+0000","last_fullsized":"2024-09-21T03:14:37.608477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612699+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:14:37.612699+0000","last_peered":"2024-09-21T03:14:37.612699+0000","last_clean":"2024-09-21T03:14:37.612699+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:14:37.612699+0000","last_undegraded":"2024-09-21T03:14:37.612699+0000","last_fullsized":"2024-09-21T03:14:37.612699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:38.202124+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:14:38.202124+0000","last_peered":"2024-09-21T03:14:38.202124+0000","last_clean":"2024-09-21T03:14:38.202124+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:14:38.202124+0000","last_undegraded":"2024-09-21T03:14:38.202124+0000","last_fullsized":"2024-09-21T03:14:38.202124+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609934+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:14:37.609934+0000","last_peered":"2024-09-21T03:14:37.609934+0000","last_clean":"2024-09-21T03:14:37.609934+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:14:37.609934+0000","last_undegraded":"2024-09-21T03:14:37.609934+0000","last_fullsized":"2024-09-21T03:14:37.609934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:55.774459+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:14:55.774459+0000","last_peered":"2024-09-21T03:14:55.774459+0000","last_clean":"2024-09-21T03:14:55.774459+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:14:55.774459+0000","last_undegraded":"2024-09-21T03:14:55.774459+0000","last_fullsized":"2024-09-21T03:14:55.774459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":52016,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":132,"num_read_kb":121,"num_write":227,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":156048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":399243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1035084,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070491,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":407960,"data_compressed":2875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":1000087,"data_compressed":38740,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":827392,"data_stored":398499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":990965,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":398360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":445226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":444615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":847872,"data_stored":400229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":46955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":47566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":46939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:03.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:03.457 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 -- 172.21.15.83:0/1950005897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0109df0 msgr2=0x7f33f0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 --2- 172.21.15.83:0/1950005897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0109df0 0x7f33f0110620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009ca0 tx=0x7f33e802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 -- 172.21.15.83:0/1950005897 shutdown_connections 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 --2- 172.21.15.83:0/1950005897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0109df0 0x7f33f0110620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 --2- 172.21.15.83:0/1950005897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 0x7f33f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 --2- 172.21.15.83:0/1950005897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0103b60 0x7f33f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 -- 172.21.15.83:0/1950005897 >> 172.21.15.83:0/1950005897 conn(0x7f33f007b850 msgr2=0x7f33f007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 -- 172.21.15.83:0/1950005897 shutdown_connections 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.929+0000 7f33f6160640 1 -- 172.21.15.83:0/1950005897 wait complete. 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 Processor -- start 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 -- start start 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 0x7f33f019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 0x7f33f019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0113ab0 con 0x7f33f0103b60 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0113930 con 0x7f33f0109df0 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.930+0000 7f33f6160640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0113c30 con 0x7f33f01094a0 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57542/0 (socket says 172.21.15.83:57542) 2024-09-21T03:15:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 -- 172.21.15.83:0/3283172987 learned_addr learned my addr 172.21.15.83:0/3283172987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33eeffd640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 0x7f33f019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33effff640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 0x7f33f019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 -- 172.21.15.83:0/3283172987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 msgr2=0x7f33f019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 0x7f33f019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 -- 172.21.15.83:0/3283172987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 msgr2=0x7f33f019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 0x7f33f019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 -- 172.21.15.83:0/3283172987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f01a03b0 con 0x7f33f0103b60 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33effff640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 0x7f33f019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ef7fe640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c890 tx=0x7f33e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004220 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e00043c0 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004a00 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f01a06a0 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.931+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f01a6e00 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.932+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f0103f70 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.934+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f33e00079b0 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.935+0000 7f33ecff9640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 0x7f33d80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.935+0000 7f33eeffd640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 0x7f33d80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.935+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f33e00e7690 con 0x7f33f0103b60 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.935+0000 7f33eeffd640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 0x7f33d80b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f33f0106fc0 tx=0x7f33e400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:03.938+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33e0014030 con 0x7f33f0103b60 2024-09-21T03:15:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.113+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33f0063800 con 0x7f33d80b6670 2024-09-21T03:15:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.117+0000 7f33ecff9640 1 -- 172.21.15.83:0/3283172987 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258957 (secure 0 0 0) 0x7f33f0063800 con 0x7f33d80b6670 2024-09-21T03:15:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:04.122 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 msgr2=0x7f33d80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 0x7f33d80b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f33f0106fc0 tx=0x7f33e400b410 comp rx=0 tx=0).stop 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 msgr2=0x7f33f019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c890 tx=0x7f33e000cd50 comp rx=0 tx=0).stop 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 shutdown_connections 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33d80b6670 0x7f33d80b8b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109df0 0x7f33f019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:04.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f01094a0 0x7f33f019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 --2- 172.21.15.83:0/3283172987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f0103b60 0x7f33f019a440 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.120+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 >> 172.21.15.83:0/3283172987 conn(0x7f33f007b850 msgr2=0x7f33f0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.121+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 shutdown_connections 2024-09-21T03:15:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:04.121+0000 7f33f6160640 1 -- 172.21.15.83:0/3283172987 wait complete. 2024-09-21T03:15:04.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:04 smithi083 ceph-mon[31219]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:04.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:04 smithi083 ceph-mon[31219]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:04.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:04 smithi193 ceph-mon[35224]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:04.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:04 smithi193 ceph-mon[35224]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:04 smithi142 ceph-mon[34512]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:04 smithi142 ceph-mon[34512]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:04.929 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-21T03:15:03.174569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651670,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":254,"num_read_kb":224,"num_write":361,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351376,"kb_used_data":10384,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590448,"statfs":{"total":1151940427776,"available":1151580618752,"internally_reserved":0,"allocated":10633216,"data_stored":6884207,"data_compressed":159401,"data_compressed_allocated":208896,"data_compressed_original":1916928,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46277,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":96,"num_write":27,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002029"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261516+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:14:59.261516+0000","last_peered":"2024-09-21T03:14:59.261516+0000","last_clean":"2024-09-21T03:14:59.261516+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:14:59.261516+0000","last_undegraded":"2024-09-21T03:14:59.261516+0000","last_fullsized":"2024-09-21T03:14:59.261516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609145+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:14:37.609145+0000","last_peered":"2024-09-21T03:14:37.609145+0000","last_clean":"2024-09-21T03:14:37.609145+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:14:37.609145+0000","last_undegraded":"2024-09-21T03:14:37.609145+0000","last_fullsized":"2024-09-21T03:14:37.609145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261730+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:14:59.261730+0000","last_peered":"2024-09-21T03:14:59.261730+0000","last_clean":"2024-09-21T03:14:59.261730+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:14:59.261730+0000","last_undegraded":"2024-09-21T03:14:59.261730+0000","last_fullsized":"2024-09-21T03:14:59.261730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609065+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:14:37.609065+0000","last_peered":"2024-09-21T03:14:37.609065+0000","last_clean":"2024-09-21T03:14:37.609065+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:14:37.609065+0000","last_undegraded":"2024-09-21T03:14:37.609065+0000","last_fullsized":"2024-09-21T03:14:37.609065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613252+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:14:37.613252+0000","last_peered":"2024-09-21T03:14:37.613252+0000","last_clean":"2024-09-21T03:14:37.613252+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:14:37.613252+0000","last_undegraded":"2024-09-21T03:14:37.613252+0000","last_fullsized":"2024-09-21T03:14:37.613252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610245+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:14:37.610245+0000","last_peered":"2024-09-21T03:14:37.610245+0000","last_clean":"2024-09-21T03:14:37.610245+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:14:37.610245+0000","last_undegraded":"2024-09-21T03:14:37.610245+0000","last_fullsized":"2024-09-21T03:14:37.610245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:57.778835+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:14:57.778835+0000","last_peered":"2024-09-21T03:14:57.778835+0000","last_clean":"2024-09-21T03:14:57.778835+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:14:57.778835+0000","last_undegraded":"2024-09-21T03:14:57.778835+0000","last_fullsized":"2024-09-21T03:14:57.778835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613437+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:14:37.613437+0000","last_peered":"2024-09-21T03:14:37.613437+0000","last_clean":"2024-09-21T03:14:37.613437+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:14:37.613437+0000","last_undegraded":"2024-09-21T03:14:37.613437+0000","last_fullsized":"2024-09-21T03:14:37.613437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609387+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:14:37.609387+0000","last_peered":"2024-09-21T03:14:37.609387+0000","last_clean":"2024-09-21T03:14:37.609387+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:14:37.609387+0000","last_undegraded":"2024-09-21T03:14:37.609387+0000","last_fullsized":"2024-09-21T03:14:37.609387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.867886+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:14:47.867886+0000","last_peered":"2024-09-21T03:14:47.867886+0000","last_clean":"2024-09-21T03:14:47.867886+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:14:47.867886+0000","last_undegraded":"2024-09-21T03:14:47.867886+0000","last_fullsized":"2024-09-21T03:14:47.867886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.783925+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:01.783925+0000","last_peered":"2024-09-21T03:15:01.783925+0000","last_clean":"2024-09-21T03:15:01.783925+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:01.783925+0000","last_undegraded":"2024-09-21T03:15:01.783925+0000","last_fullsized":"2024-09-21T03:15:01.783925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261577+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:14:59.261577+0000","last_peered":"2024-09-21T03:14:59.261577+0000","last_clean":"2024-09-21T03:14:59.261577+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:14:59.261577+0000","last_undegraded":"2024-09-21T03:14:59.261577+0000","last_fullsized":"2024-09-21T03:14:59.261577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:46.853121+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:14:46.853121+0000","last_peered":"2024-09-21T03:14:46.853121+0000","last_clean":"2024-09-21T03:14:46.853121+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:14:46.853121+0000","last_undegraded":"2024-09-21T03:14:46.853121+0000","last_fullsized":"2024-09-21T03:14:46.853121+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.787083+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:01.787083+0000","last_peered":"2024-09-21T03:15:01.787083+0000","last_clean":"2024-09-21T03:15:01.787083+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:01.787083+0000","last_undegraded":"2024-09-21T03:15:01.787083+0000","last_fullsized":"2024-09-21T03:15:01.787083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54255,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":69,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.868082+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:14:47.868082+0000","last_peered":"2024-09-21T03:14:47.868082+0000","last_clean":"2024-09-21T03:14:47.868082+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:14:47.868082+0000","last_undegraded":"2024-09-21T03:14:47.868082+0000","last_fullsized":"2024-09-21T03:14:47.868082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261836+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:14:59.261836+0000","last_peered":"2024-09-21T03:14:59.261836+0000","last_clean":"2024-09-21T03:14:59.261836+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:14:59.261836+0000","last_undegraded":"2024-09-21T03:14:59.261836+0000","last_fullsized":"2024-09-21T03:14:59.261836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.783908+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:01.783908+0000","last_peered":"2024-09-21T03:15:01.783908+0000","last_clean":"2024-09-21T03:15:01.783908+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:01.783908+0000","last_undegraded":"2024-09-21T03:15:01.783908+0000","last_fullsized":"2024-09-21T03:15:01.783908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262027+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:14:59.262027+0000","last_peered":"2024-09-21T03:14:59.262027+0000","last_clean":"2024-09-21T03:14:59.262027+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:14:59.262027+0000","last_undegraded":"2024-09-21T03:14:59.262027+0000","last_fullsized":"2024-09-21T03:14:59.262027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612590+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:14:37.612590+0000","last_peered":"2024-09-21T03:14:37.612590+0000","last_clean":"2024-09-21T03:14:37.612590+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:14:37.612590+0000","last_undegraded":"2024-09-21T03:14:37.612590+0000","last_fullsized":"2024-09-21T03:14:37.612590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261784+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:14:59.261784+0000","last_peered":"2024-09-21T03:14:59.261784+0000","last_clean":"2024-09-21T03:14:59.261784+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:14:59.261784+0000","last_undegraded":"2024-09-21T03:14:59.261784+0000","last_fullsized":"2024-09-21T03:14:59.261784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.868479+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:14:47.868479+0000","last_peered":"2024-09-21T03:14:47.868479+0000","last_clean":"2024-09-21T03:14:47.868479+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:14:47.868479+0000","last_undegraded":"2024-09-21T03:14:47.868479+0000","last_fullsized":"2024-09-21T03:14:47.868479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"peering","last_fresh":"2024-09-21T03:14:55.835362+0000","last_change":"2024-09-21T03:14:59.262045+0000","last_active":"2024-09-21T03:14:59.262045+0000","last_peered":"2024-09-21T03:14:55.835362+0000","last_clean":"2024-09-21T03:14:55.835362+0000","last_became_active":"2024-09-21T03:14:36.620675+0000","last_became_peered":"2024-09-21T03:14:36.620675+0000","last_unstale":"2024-09-21T03:14:55.835362+0000","last_undegraded":"2024-09-21T03:14:55.835362+0000","last_fullsized":"2024-09-21T03:14:55.835362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":46,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.782974+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:00.782974+0000","last_peered":"2024-09-21T03:15:00.782974+0000","last_clean":"2024-09-21T03:15:00.782974+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:00.782974+0000","last_undegraded":"2024-09-21T03:15:00.782974+0000","last_fullsized":"2024-09-21T03:15:00.782974+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261861+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:14:59.261861+0000","last_peered":"2024-09-21T03:14:59.261861+0000","last_clean":"2024-09-21T03:14:59.261861+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:14:59.261861+0000","last_undegraded":"2024-09-21T03:14:59.261861+0000","last_fullsized":"2024-09-21T03:14:59.261861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262217+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:14:59.262217+0000","last_peered":"2024-09-21T03:14:59.262217+0000","last_clean":"2024-09-21T03:14:59.262217+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:14:59.262217+0000","last_undegraded":"2024-09-21T03:14:59.262217+0000","last_fullsized":"2024-09-21T03:14:59.262217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609578+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:14:37.609578+0000","last_peered":"2024-09-21T03:14:37.609578+0000","last_clean":"2024-09-21T03:14:37.609578+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:14:37.609578+0000","last_undegraded":"2024-09-21T03:14:37.609578+0000","last_fullsized":"2024-09-21T03:14:37.609578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262234+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:14:59.262234+0000","last_peered":"2024-09-21T03:14:59.262234+0000","last_clean":"2024-09-21T03:14:59.262234+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:14:59.262234+0000","last_undegraded":"2024-09-21T03:14:59.262234+0000","last_fullsized":"2024-09-21T03:14:59.262234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265682+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:00.265682+0000","last_peered":"2024-09-21T03:15:00.265682+0000","last_clean":"2024-09-21T03:15:00.265682+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:00.265682+0000","last_undegraded":"2024-09-21T03:15:00.265682+0000","last_fullsized":"2024-09-21T03:15:00.265682+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.612922+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:14:37.612922+0000","last_peered":"2024-09-21T03:14:37.612922+0000","last_clean":"2024-09-21T03:14:37.612922+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:14:37.612922+0000","last_undegraded":"2024-09-21T03:14:37.612922+0000","last_fullsized":"2024-09-21T03:14:37.612922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262288+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:14:59.262288+0000","last_peered":"2024-09-21T03:14:59.262288+0000","last_clean":"2024-09-21T03:14:59.262288+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:14:59.262288+0000","last_undegraded":"2024-09-21T03:14:59.262288+0000","last_fullsized":"2024-09-21T03:14:59.262288+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'13","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261762+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:00.261762+0000","last_peered":"2024-09-21T03:15:00.261762+0000","last_clean":"2024-09-21T03:15:00.261762+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:00.261762+0000","last_undegraded":"2024-09-21T03:15:00.261762+0000","last_fullsized":"2024-09-21T03:15:00.261762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610283+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:14:37.610283+0000","last_peered":"2024-09-21T03:14:37.610283+0000","last_clean":"2024-09-21T03:14:37.610283+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:14:37.610283+0000","last_undegraded":"2024-09-21T03:14:37.610283+0000","last_fullsized":"2024-09-21T03:14:37.610283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:47.867743+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:14:47.867743+0000","last_peered":"2024-09-21T03:14:47.867743+0000","last_clean":"2024-09-21T03:14:47.867743+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:14:47.867743+0000","last_undegraded":"2024-09-21T03:14:47.867743+0000","last_fullsized":"2024-09-21T03:14:47.867743+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.610006+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:14:37.610006+0000","last_peered":"2024-09-21T03:14:37.610006+0000","last_clean":"2024-09-21T03:14:37.610006+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:14:37.610006+0000","last_undegraded":"2024-09-21T03:14:37.610006+0000","last_fullsized":"2024-09-21T03:14:37.610006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262017+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:14:59.262017+0000","last_peered":"2024-09-21T03:14:59.262017+0000","last_clean":"2024-09-21T03:14:59.262017+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:14:59.262017+0000","last_undegraded":"2024-09-21T03:14:59.262017+0000","last_fullsized":"2024-09-21T03:14:59.262017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.613567+0000","last_change":"2024-09-21T03:14:36.621002+0000","last_active":"2024-09-21T03:14:37.613567+0000","last_peered":"2024-09-21T03:14:37.613567+0000","last_clean":"2024-09-21T03:14:37.613567+0000","last_became_active":"2024-09-21T03:14:36.620640+0000","last_became_peered":"2024-09-21T03:14:36.620640+0000","last_unstale":"2024-09-21T03:14:37.613567+0000","last_undegraded":"2024-09-21T03:14:37.613567+0000","last_fullsized":"2024-09-21T03:14:37.613567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:39.471195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:38.202124+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:14:38.202124+0000","last_peered":"2024-09-21T03:14:38.202124+0000","last_clean":"2024-09-21T03:14:38.202124+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:14:38.202124+0000","last_undegraded":"2024-09-21T03:14:38.202124+0000","last_fullsized":"2024-09-21T03:14:38.202124+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T03:14:37.609934+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:14:37.609934+0000","last_peered":"2024-09-21T03:14:37.609934+0000","last_clean":"2024-09-21T03:14:37.609934+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:14:37.609934+0000","last_undegraded":"2024-09-21T03:14:37.609934+0000","last_fullsized":"2024-09-21T03:14:37.609934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.781443+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:00.781443+0000","last_peered":"2024-09-21T03:15:00.781443+0000","last_clean":"2024-09-21T03:15:00.781443+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:00.781443+0000","last_undegraded":"2024-09-21T03:15:00.781443+0000","last_fullsized":"2024-09-21T03:15:00.781443+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61068,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":173,"num_read_kb":155,"num_write":236,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":183204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743441,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939671,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1061581,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070492,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":407960,"data_compressed":2875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":1000087,"data_compressed":38740,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":990965,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":398324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":471655,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":471112,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:04.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:05.184 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:05.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:05 smithi193 ceph-mon[35224]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:05.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:05 smithi083 ceph-mon[31219]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:05 smithi142 ceph-mon[34512]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- 172.21.15.83:0/1859176628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14108420 msgr2=0x7f5a14108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1859176628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14108420 0x7f5a14108830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009900 tx=0x7f5a0002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- 172.21.15.83:0/1859176628 shutdown_connections 2024-09-21T03:15:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1859176628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108d70 0x7f5a1410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1859176628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14108420 0x7f5a14108830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1859176628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14102ae0 0x7f5a14102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- 172.21.15.83:0/1859176628 >> 172.21.15.83:0/1859176628 conn(0x7f5a14079770 msgr2=0x7f5a14079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- 172.21.15.83:0/1859176628 shutdown_connections 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- 172.21.15.83:0/1859176628 wait complete. 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 Processor -- start 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.665+0000 7f5a1b122640 1 -- start start 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108420 0x7f5a1419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 0x7f5a1419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14112a50 con 0x7f5a14102ae0 2024-09-21T03:15:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141128d0 con 0x7f5a14108420 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14112bd0 con 0x7f5a14108d70 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57564/0 (socket says 172.21.15.83:57564) 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 -- 172.21.15.83:0/1359700411 learned_addr learned my addr 172.21.15.83:0/1359700411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a196bf640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 0x7f5a1419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a13fff640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108420 0x7f5a1419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 -- 172.21.15.83:0/1359700411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 msgr2=0x7f5a1419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 0x7f5a1419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 -- 172.21.15.83:0/1359700411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108420 msgr2=0x7f5a1419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108420 0x7f5a1419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 -- 172.21.15.83:0/1359700411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a141a0520 con 0x7f5a14102ae0 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a196bf640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 0x7f5a1419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a18ebe640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4a0 tx=0x7f5a0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:05.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08014070 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08010ac0 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141a0810 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.666+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017c30 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.667+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141a6f70 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.668+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a14103d60 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.669+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.670+0000 7f5a11ffb640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 0x7f5a040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.670+0000 7f5a13fff640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 0x7f5a040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.670+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f5a080e6ce0 con 0x7f5a14102ae0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.670+0000 7f5a13fff640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 0x7f5a040b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a00000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.673+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a080ad370 con 0x7f5a14102ae0 2024-09-21T03:15:05.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.851+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a1410bbd0 con 0x7f5a040b65f0 2024-09-21T03:15:05.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.856+0000 7f5a11ffb640 1 -- 172.21.15.83:0/1359700411 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258968 (secure 0 0 0) 0x7f5a1410bbd0 con 0x7f5a040b65f0 2024-09-21T03:15:05.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:05.860 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:05.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 msgr2=0x7f5a040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 0x7f5a040b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006010 tx=0x7f5a00000bc0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 msgr2=0x7f5a1419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4a0 tx=0x7f5a0800b960 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 shutdown_connections 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5a040b65f0 0x7f5a040b8b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a14108d70 0x7f5a1419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a14108420 0x7f5a1419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 --2- 172.21.15.83:0/1359700411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102ae0 0x7f5a1419a5b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 >> 172.21.15.83:0/1359700411 conn(0x7f5a14079770 msgr2=0x7f5a14101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 shutdown_connections 2024-09-21T03:15:05.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:05.859+0000 7f5a1b122640 1 -- 172.21.15.83:0/1359700411 wait complete. 2024-09-21T03:15:06.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:06 smithi193 ceph-mon[35224]: pgmap v278: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:06 smithi193 ceph-mon[35224]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:06.617 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:06 smithi083 ceph-mon[31219]: pgmap v278: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:06.618 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:06 smithi083 ceph-mon[31219]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:06.647 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-21T03:15:05.175097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651670,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":259,"num_read_kb":224,"num_write":361,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351436,"kb_used_data":10428,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590388,"statfs":{"total":1151940427776,"available":1151580557312,"internally_reserved":0,"allocated":10678272,"data_stored":6918980,"data_compressed":170915,"data_compressed_allocated":225280,"data_compressed_original":1949696,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46277,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":91,"num_write":27,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002079"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261516+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:14:59.261516+0000","last_peered":"2024-09-21T03:14:59.261516+0000","last_clean":"2024-09-21T03:14:59.261516+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:14:59.261516+0000","last_undegraded":"2024-09-21T03:14:59.261516+0000","last_fullsized":"2024-09-21T03:14:59.261516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258368+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:00.258368+0000","last_peered":"2024-09-21T03:15:00.258368+0000","last_clean":"2024-09-21T03:15:00.258368+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:00.258368+0000","last_undegraded":"2024-09-21T03:15:00.258368+0000","last_fullsized":"2024-09-21T03:15:00.258368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261730+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:14:59.261730+0000","last_peered":"2024-09-21T03:14:59.261730+0000","last_clean":"2024-09-21T03:14:59.261730+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:14:59.261730+0000","last_undegraded":"2024-09-21T03:14:59.261730+0000","last_fullsized":"2024-09-21T03:14:59.261730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258439+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:00.258439+0000","last_peered":"2024-09-21T03:15:00.258439+0000","last_clean":"2024-09-21T03:15:00.258439+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:00.258439+0000","last_undegraded":"2024-09-21T03:15:00.258439+0000","last_fullsized":"2024-09-21T03:15:00.258439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263181+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:00.263181+0000","last_peered":"2024-09-21T03:15:00.263181+0000","last_clean":"2024-09-21T03:15:00.263181+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:00.263181+0000","last_undegraded":"2024-09-21T03:15:00.263181+0000","last_fullsized":"2024-09-21T03:15:00.263181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258788+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:00.258788+0000","last_peered":"2024-09-21T03:15:00.258788+0000","last_clean":"2024-09-21T03:15:00.258788+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:00.258788+0000","last_undegraded":"2024-09-21T03:15:00.258788+0000","last_fullsized":"2024-09-21T03:15:00.258788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":46,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:02.786143+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:15:02.786143+0000","last_peered":"2024-09-21T03:15:02.786143+0000","last_clean":"2024-09-21T03:15:02.786143+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:15:02.786143+0000","last_undegraded":"2024-09-21T03:15:02.786143+0000","last_fullsized":"2024-09-21T03:15:02.786143+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263742+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:00.263742+0000","last_peered":"2024-09-21T03:15:00.263742+0000","last_clean":"2024-09-21T03:15:00.263742+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:00.263742+0000","last_undegraded":"2024-09-21T03:15:00.263742+0000","last_fullsized":"2024-09-21T03:15:00.263742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258447+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:00.258447+0000","last_peered":"2024-09-21T03:15:00.258447+0000","last_clean":"2024-09-21T03:15:00.258447+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:00.258447+0000","last_undegraded":"2024-09-21T03:15:00.258447+0000","last_fullsized":"2024-09-21T03:15:00.258447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.783925+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:01.783925+0000","last_peered":"2024-09-21T03:15:01.783925+0000","last_clean":"2024-09-21T03:15:01.783925+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:01.783925+0000","last_undegraded":"2024-09-21T03:15:01.783925+0000","last_fullsized":"2024-09-21T03:15:01.783925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261577+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:14:59.261577+0000","last_peered":"2024-09-21T03:14:59.261577+0000","last_clean":"2024-09-21T03:14:59.261577+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:14:59.261577+0000","last_undegraded":"2024-09-21T03:14:59.261577+0000","last_fullsized":"2024-09-21T03:14:59.261577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.787083+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:01.787083+0000","last_peered":"2024-09-21T03:15:01.787083+0000","last_clean":"2024-09-21T03:15:01.787083+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:01.787083+0000","last_undegraded":"2024-09-21T03:15:01.787083+0000","last_fullsized":"2024-09-21T03:15:01.787083+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54255,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":69,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261836+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:14:59.261836+0000","last_peered":"2024-09-21T03:14:59.261836+0000","last_clean":"2024-09-21T03:14:59.261836+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:14:59.261836+0000","last_undegraded":"2024-09-21T03:14:59.261836+0000","last_fullsized":"2024-09-21T03:14:59.261836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:01.783908+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:01.783908+0000","last_peered":"2024-09-21T03:15:01.783908+0000","last_clean":"2024-09-21T03:15:01.783908+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:01.783908+0000","last_undegraded":"2024-09-21T03:15:01.783908+0000","last_fullsized":"2024-09-21T03:15:01.783908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262027+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:14:59.262027+0000","last_peered":"2024-09-21T03:14:59.262027+0000","last_clean":"2024-09-21T03:14:59.262027+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:14:59.262027+0000","last_undegraded":"2024-09-21T03:14:59.262027+0000","last_fullsized":"2024-09-21T03:14:59.262027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263368+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:00.263368+0000","last_peered":"2024-09-21T03:15:00.263368+0000","last_clean":"2024-09-21T03:15:00.263368+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:00.263368+0000","last_undegraded":"2024-09-21T03:15:00.263368+0000","last_fullsized":"2024-09-21T03:15:00.263368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261784+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:14:59.261784+0000","last_peered":"2024-09-21T03:14:59.261784+0000","last_clean":"2024-09-21T03:14:59.261784+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:14:59.261784+0000","last_undegraded":"2024-09-21T03:14:59.261784+0000","last_fullsized":"2024-09-21T03:14:59.261784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"peering","last_fresh":"2024-09-21T03:14:55.835362+0000","last_change":"2024-09-21T03:14:59.262045+0000","last_active":"2024-09-21T03:14:59.262045+0000","last_peered":"2024-09-21T03:14:55.835362+0000","last_clean":"2024-09-21T03:14:55.835362+0000","last_became_active":"2024-09-21T03:14:36.620675+0000","last_became_peered":"2024-09-21T03:14:36.620675+0000","last_unstale":"2024-09-21T03:14:55.835362+0000","last_undegraded":"2024-09-21T03:14:55.835362+0000","last_fullsized":"2024-09-21T03:14:55.835362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":46,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.782974+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:00.782974+0000","last_peered":"2024-09-21T03:15:00.782974+0000","last_clean":"2024-09-21T03:15:00.782974+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:00.782974+0000","last_undegraded":"2024-09-21T03:15:00.782974+0000","last_fullsized":"2024-09-21T03:15:00.782974+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.261861+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:14:59.261861+0000","last_peered":"2024-09-21T03:14:59.261861+0000","last_clean":"2024-09-21T03:14:59.261861+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:14:59.261861+0000","last_undegraded":"2024-09-21T03:14:59.261861+0000","last_fullsized":"2024-09-21T03:14:59.261861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262217+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:14:59.262217+0000","last_peered":"2024-09-21T03:14:59.262217+0000","last_clean":"2024-09-21T03:14:59.262217+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:14:59.262217+0000","last_undegraded":"2024-09-21T03:14:59.262217+0000","last_fullsized":"2024-09-21T03:14:59.262217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258614+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:00.258614+0000","last_peered":"2024-09-21T03:15:00.258614+0000","last_clean":"2024-09-21T03:15:00.258614+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:00.258614+0000","last_undegraded":"2024-09-21T03:15:00.258614+0000","last_fullsized":"2024-09-21T03:15:00.258614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262234+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:14:59.262234+0000","last_peered":"2024-09-21T03:14:59.262234+0000","last_clean":"2024-09-21T03:14:59.262234+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:14:59.262234+0000","last_undegraded":"2024-09-21T03:14:59.262234+0000","last_fullsized":"2024-09-21T03:14:59.262234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265682+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:00.265682+0000","last_peered":"2024-09-21T03:15:00.265682+0000","last_clean":"2024-09-21T03:15:00.265682+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:00.265682+0000","last_undegraded":"2024-09-21T03:15:00.265682+0000","last_fullsized":"2024-09-21T03:15:00.265682+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264186+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:00.264186+0000","last_peered":"2024-09-21T03:15:00.264186+0000","last_clean":"2024-09-21T03:15:00.264186+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:00.264186+0000","last_undegraded":"2024-09-21T03:15:00.264186+0000","last_fullsized":"2024-09-21T03:15:00.264186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262288+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:14:59.262288+0000","last_peered":"2024-09-21T03:14:59.262288+0000","last_clean":"2024-09-21T03:14:59.262288+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:14:59.262288+0000","last_undegraded":"2024-09-21T03:14:59.262288+0000","last_fullsized":"2024-09-21T03:14:59.262288+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'13","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261762+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:00.261762+0000","last_peered":"2024-09-21T03:15:00.261762+0000","last_clean":"2024-09-21T03:15:00.261762+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:00.261762+0000","last_undegraded":"2024-09-21T03:15:00.261762+0000","last_fullsized":"2024-09-21T03:15:00.261762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259126+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:00.259126+0000","last_peered":"2024-09-21T03:15:00.259126+0000","last_clean":"2024-09-21T03:15:00.259126+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:00.259126+0000","last_undegraded":"2024-09-21T03:15:00.259126+0000","last_fullsized":"2024-09-21T03:15:00.259126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258634+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:00.258634+0000","last_peered":"2024-09-21T03:15:00.258634+0000","last_clean":"2024-09-21T03:15:00.258634+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:00.258634+0000","last_undegraded":"2024-09-21T03:15:00.258634+0000","last_fullsized":"2024-09-21T03:15:00.258634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:14:59.262017+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:14:59.262017+0000","last_peered":"2024-09-21T03:14:59.262017+0000","last_clean":"2024-09-21T03:14:59.262017+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:14:59.262017+0000","last_undegraded":"2024-09-21T03:14:59.262017+0000","last_fullsized":"2024-09-21T03:14:59.262017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.267843+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:00.267843+0000","last_peered":"2024-09-21T03:15:00.267843+0000","last_clean":"2024-09-21T03:15:00.267843+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:00.267843+0000","last_undegraded":"2024-09-21T03:15:00.267843+0000","last_fullsized":"2024-09-21T03:15:00.267843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259025+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259025+0000","last_peered":"2024-09-21T03:15:00.259025+0000","last_clean":"2024-09-21T03:15:00.259025+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259025+0000","last_undegraded":"2024-09-21T03:15:00.259025+0000","last_fullsized":"2024-09-21T03:15:00.259025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258780+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:00.258780+0000","last_peered":"2024-09-21T03:15:00.258780+0000","last_clean":"2024-09-21T03:15:00.258780+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:00.258780+0000","last_undegraded":"2024-09-21T03:15:00.258780+0000","last_fullsized":"2024-09-21T03:15:00.258780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.781443+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:00.781443+0000","last_peered":"2024-09-21T03:15:00.781443+0000","last_clean":"2024-09-21T03:15:00.781443+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:00.781443+0000","last_undegraded":"2024-09-21T03:15:00.781443+0000","last_fullsized":"2024-09-21T03:15:00.781443+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61068,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":178,"num_read_kb":155,"num_write":236,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":183204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743441,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939671,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1061581,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070492,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":407960,"data_compressed":2875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":1000087,"data_compressed":38740,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822705,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":471655,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":471112,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":55991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:06.657 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-21T03:15:06.657 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.c', 'version': "75'1", 'reported_seq': 16, 'reported_epoch': 75, 'state': 'peering', 'last_fresh': '2024-09-21T03:14:55.835362+0000', 'last_change': '2024-09-21T03:14:59.262045+0000', 'last_active': '2024-09-21T03:14:59.262045+0000', 'last_peered': '2024-09-21T03:14:55.835362+0000', 'last_clean': '2024-09-21T03:14:55.835362+0000', 'last_became_active': '2024-09-21T03:14:36.620675+0000', 'last_became_peered': '2024-09-21T03:14:36.620675+0000', 'last_unstale': '2024-09-21T03:14:55.835362+0000', 'last_undegraded': '2024-09-21T03:14:55.835362+0000', 'last_fullsized': '2024-09-21T03:14:55.835362+0000', 'mapping_epoch': 76, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 74, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T03:14:35.582522+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T03:14:35.582522+0000', 'last_clean_scrub_stamp': '2024-09-21T03:14:35.582522+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 117, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 9, 5], 'acting': [7, 9, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 5, 9], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T03:15:06.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-21T03:15:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:06 smithi142 ceph-mon[34512]: pgmap v278: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 15 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:06 smithi142 ceph-mon[34512]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:06.909 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- 172.21.15.83:0/1525232254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac108530 msgr2=0x7fbfac10ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 --2- 172.21.15.83:0/1525232254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac108530 0x7fbfac10ed60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009ca0 tx=0x7fbfa402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- 172.21.15.83:0/1525232254 shutdown_connections 2024-09-21T03:15:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 --2- 172.21.15.83:0/1525232254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac108530 0x7fbfac10ed60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 --2- 172.21.15.83:0/1525232254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac107be0 0x7fbfac107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 --2- 172.21.15.83:0/1525232254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 0x7fbfac1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- 172.21.15.83:0/1525232254 >> 172.21.15.83:0/1525232254 conn(0x7fbfac079ec0 msgr2=0x7fbfac07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- 172.21.15.83:0/1525232254 shutdown_connections 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- 172.21.15.83:0/1525232254 wait complete. 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 Processor -- start 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.425+0000 7fbfb2907640 1 -- start start 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 0x7fbfac110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 0x7fbfac1110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac113b60 con 0x7fbfac107be0 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1139e0 con 0x7fbfac108530 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac113ce0 con 0x7fbfac1022a0 2024-09-21T03:15:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57584/0 (socket says 172.21.15.83:57584) 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 -- 172.21.15.83:0/2421185268 learned_addr learned my addr 172.21.15.83:0/2421185268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfabfff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 0x7fbfac110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb0ea4640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 0x7fbfac1110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 -- 172.21.15.83:0/2421185268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 msgr2=0x7fbfac110650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 0x7fbfac110650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 -- 172.21.15.83:0/2421185268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 msgr2=0x7fbfac1110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 0x7fbfac1110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 -- 172.21.15.83:0/2421185268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac1a87b0 con 0x7fbfac107be0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb0ea4640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 0x7fbfac1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa3fff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c970 tx=0x7fbf9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94007bb0 con 0x7fbfac107be0 2024-09-21T03:15:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf94007d50 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac1a8aa0 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.426+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400f450 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.427+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a9030 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.428+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac1026b0 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.430+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbf94004eb0 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.431+0000 7fbfa9ffb640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 0x7fbf980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.431+0000 7fbfabfff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 0x7fbf980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.431+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fbf940e7720 con 0x7fbfac107be0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.431+0000 7fbfabfff640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 0x7fbf980b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005fd0 tx=0x7fbf9c005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.434+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf94014030 con 0x7fbfac107be0 2024-09-21T03:15:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:07 smithi193 ceph-mon[35224]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:07 smithi083 ceph-mon[31219]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:07 smithi142 ceph-mon[34512]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 23 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.681+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbfac111950 con 0x7fbfac107be0 2024-09-21T03:15:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.681+0000 7fbfa9ffb640 1 -- 172.21.15.83:0/2421185268 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3349 (secure 0 0 0) 0x7fbfac111950 con 0x7fbfac107be0 2024-09-21T03:15:07.683 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:07.683 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":437,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726888427,"num_in_osds":12,"osd_in_since":1726888403,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":651670,"bytes_used":359890944,"bytes_avail":1151580536832,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":23,"recovering_keys_per_sec":1,"num_objects_recovered":10,"num_bytes_recovered":185,"num_keys_recovered":10,"read_bytes_sec":4990,"write_bytes_sec":1663,"read_op_per_sec":7,"write_op_per_sec":1},"fsmap":{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14838},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14850},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24569},{"filesystem_id":1,"rank":1,"name":"l","status":"up:standby-replay","gid":24571},{"filesystem_id":1,"rank":0,"name":"e","status":"up:standby-replay","gid":24593},{"filesystem_id":1,"rank":4,"name":"f","status":"up:active","gid":24601},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24613},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24620},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24628},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24647}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"services":{"dashboard":"https://172.21.15.83:8443/"}},"servicemap":{"epoch":18,"modified":"2024-09-21T03:15:07.175727+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 msgr2=0x7fbf980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 0x7fbf980b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005fd0 tx=0x7fbf9c005850 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 msgr2=0x7fbfac110b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c970 tx=0x7fbf9400ce30 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 shutdown_connections 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbf980b6670 0x7fbf980b8b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfac108530 0x7fbfac1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfac107be0 0x7fbfac110b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 --2- 172.21.15.83:0/2421185268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfac1022a0 0x7fbfac110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 >> 172.21.15.83:0/2421185268 conn(0x7fbfac079ec0 msgr2=0x7fbfac103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.683+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 shutdown_connections 2024-09-21T03:15:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:07.684+0000 7fbfb2907640 1 -- 172.21.15.83:0/2421185268 wait complete. 2024-09-21T03:15:08.427 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T03:15:08.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:08 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2421185268' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T03:15:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:08 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2421185268' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T03:15:08.676 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:08 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2421185268' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T03:15:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 -- 172.21.15.83:0/3182467088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b8102a80 msgr2=0x7f74b8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 --2- 172.21.15.83:0/3182467088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b8102a80 0x7f74b8102e90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009900 tx=0x7f74a802a410 comp rx=0 tx=0).stop 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 -- 172.21.15.83:0/3182467088 shutdown_connections 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 --2- 172.21.15.83:0/3182467088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 0x7f74b810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 --2- 172.21.15.83:0/3182467088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b81083c0 0x7f74b81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 --2- 172.21.15.83:0/3182467088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b8102a80 0x7f74b8102e90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 -- 172.21.15.83:0/3182467088 >> 172.21.15.83:0/3182467088 conn(0x7f74b8079740 msgr2=0x7f74b8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 -- 172.21.15.83:0/3182467088 shutdown_connections 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.188+0000 7f74bd390640 1 -- 172.21.15.83:0/3182467088 wait complete. 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 Processor -- start 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 -- start start 2024-09-21T03:15:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 0x7f74b8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 0x7f74b819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8112a60 con 0x7f74b81083c0 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81128e0 con 0x7f74b8108d10 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74bd390640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8112be0 con 0x7f74b8102a80 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57608/0 (socket says 172.21.15.83:57608) 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.189+0000 7f74b67fc640 1 -- 172.21.15.83:0/2296195461 learned_addr learned my addr 172.21.15.83:0/2296195461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b6ffd640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 0x7f74b8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b77fe640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 0x7f74b819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 -- 172.21.15.83:0/2296195461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 msgr2=0x7f74b8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 0x7f74b8199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 -- 172.21.15.83:0/2296195461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 msgr2=0x7f74b819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 0x7f74b819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 -- 172.21.15.83:0/2296195461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b819f4e0 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b6ffd640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 0x7f74b8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74b67fc640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b450 tx=0x7f74ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac015020 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b819f7d0 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac011d40 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00f340 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.190+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b806bb40 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.191+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b8103d00 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.193+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f74ac018020 con 0x7f74b81083c0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.193+0000 7f749ffff640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 0x7f74a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.193+0000 7f74b6ffd640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 0x7f74a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.193+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f74ac0e7060 con 0x7f74b81083c0 2024-09-21T03:15:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.194+0000 7f74b6ffd640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 0x7f74a00b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74a8031440 tx=0x7f74a8005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.198+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74ac01f020 con 0x7f74b81083c0 2024-09-21T03:15:09.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:09 smithi083 ceph-mon[31219]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.379+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74b806be00 con 0x7f74a00b65f0 2024-09-21T03:15:09.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.383+0000 7f749ffff640 1 -- 172.21.15.83:0/2296195461 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259014 (secure 0 0 0) 0x7f74b806be00 con 0x7f74a00b65f0 2024-09-21T03:15:09.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:09.387 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 msgr2=0x7f74a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 0x7f74a00b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74a8031440 tx=0x7f74a8005d50 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 msgr2=0x7f74b8199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b450 tx=0x7f74ac00b910 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 shutdown_connections 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74a00b65f0 0x7f74a00b8b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74b8108d10 0x7f74b819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b81083c0 0x7f74b8199b10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 --2- 172.21.15.83:0/2296195461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74b8102a80 0x7f74b8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 >> 172.21.15.83:0/2296195461 conn(0x7f74b8079740 msgr2=0x7f74b8101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:09.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 shutdown_connections 2024-09-21T03:15:09.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:09.386+0000 7f74bd390640 1 -- 172.21.15.83:0/2296195461 wait complete. 2024-09-21T03:15:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:09 smithi193 ceph-mon[35224]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:09 smithi142 ceph-mon[34512]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T03:15:10.191 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-21T03:15:09.176064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660184,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":310,"num_read_kb":258,"num_write":366,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351440,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590384,"statfs":{"total":1151940427776,"available":1151580553216,"internally_reserved":0,"allocated":10727424,"data_stored":6961812,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17683,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":102,"num_read_kb":68,"num_write":15,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":185,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.925097"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259613+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:00.259613+0000","last_peered":"2024-09-21T03:15:00.259613+0000","last_clean":"2024-09-21T03:15:00.259613+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:00.259613+0000","last_undegraded":"2024-09-21T03:15:00.259613+0000","last_fullsized":"2024-09-21T03:15:00.259613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258368+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:00.258368+0000","last_peered":"2024-09-21T03:15:00.258368+0000","last_clean":"2024-09-21T03:15:00.258368+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:00.258368+0000","last_undegraded":"2024-09-21T03:15:00.258368+0000","last_fullsized":"2024-09-21T03:15:00.258368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260247+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:00.260247+0000","last_peered":"2024-09-21T03:15:00.260247+0000","last_clean":"2024-09-21T03:15:00.260247+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:00.260247+0000","last_undegraded":"2024-09-21T03:15:00.260247+0000","last_fullsized":"2024-09-21T03:15:00.260247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258439+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:00.258439+0000","last_peered":"2024-09-21T03:15:00.258439+0000","last_clean":"2024-09-21T03:15:00.258439+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:00.258439+0000","last_undegraded":"2024-09-21T03:15:00.258439+0000","last_fullsized":"2024-09-21T03:15:00.258439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263181+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:00.263181+0000","last_peered":"2024-09-21T03:15:00.263181+0000","last_clean":"2024-09-21T03:15:00.263181+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:00.263181+0000","last_undegraded":"2024-09-21T03:15:00.263181+0000","last_fullsized":"2024-09-21T03:15:00.263181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258788+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:00.258788+0000","last_peered":"2024-09-21T03:15:00.258788+0000","last_clean":"2024-09-21T03:15:00.258788+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:00.258788+0000","last_undegraded":"2024-09-21T03:15:00.258788+0000","last_fullsized":"2024-09-21T03:15:00.258788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:07.793019+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:15:07.793019+0000","last_peered":"2024-09-21T03:15:07.793019+0000","last_clean":"2024-09-21T03:15:07.793019+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:15:07.793019+0000","last_undegraded":"2024-09-21T03:15:07.793019+0000","last_fullsized":"2024-09-21T03:15:07.793019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263742+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:00.263742+0000","last_peered":"2024-09-21T03:15:00.263742+0000","last_clean":"2024-09-21T03:15:00.263742+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:00.263742+0000","last_undegraded":"2024-09-21T03:15:00.263742+0000","last_fullsized":"2024-09-21T03:15:00.263742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258447+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:00.258447+0000","last_peered":"2024-09-21T03:15:00.258447+0000","last_clean":"2024-09-21T03:15:00.258447+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:00.258447+0000","last_undegraded":"2024-09-21T03:15:00.258447+0000","last_fullsized":"2024-09-21T03:15:00.258447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:06.790843+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:06.790843+0000","last_peered":"2024-09-21T03:15:06.790843+0000","last_clean":"2024-09-21T03:15:06.790843+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:06.790843+0000","last_undegraded":"2024-09-21T03:15:06.790843+0000","last_fullsized":"2024-09-21T03:15:06.790843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:06.793510+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:06.793510+0000","last_peered":"2024-09-21T03:15:06.793510+0000","last_clean":"2024-09-21T03:15:06.793510+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:06.793510+0000","last_undegraded":"2024-09-21T03:15:06.793510+0000","last_fullsized":"2024-09-21T03:15:06.793510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62769,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":83,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260024+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:00.260024+0000","last_peered":"2024-09-21T03:15:00.260024+0000","last_clean":"2024-09-21T03:15:00.260024+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:00.260024+0000","last_undegraded":"2024-09-21T03:15:00.260024+0000","last_fullsized":"2024-09-21T03:15:00.260024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:06.791113+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:06.791113+0000","last_peered":"2024-09-21T03:15:06.791113+0000","last_clean":"2024-09-21T03:15:06.791113+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:06.791113+0000","last_undegraded":"2024-09-21T03:15:06.791113+0000","last_fullsized":"2024-09-21T03:15:06.791113+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259547+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:00.259547+0000","last_peered":"2024-09-21T03:15:00.259547+0000","last_clean":"2024-09-21T03:15:00.259547+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:00.259547+0000","last_undegraded":"2024-09-21T03:15:00.259547+0000","last_fullsized":"2024-09-21T03:15:00.259547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263368+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:00.263368+0000","last_peered":"2024-09-21T03:15:00.263368+0000","last_clean":"2024-09-21T03:15:00.263368+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:00.263368+0000","last_undegraded":"2024-09-21T03:15:00.263368+0000","last_fullsized":"2024-09-21T03:15:00.263368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260269+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:00.260269+0000","last_peered":"2024-09-21T03:15:00.260269+0000","last_clean":"2024-09-21T03:15:00.260269+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:00.260269+0000","last_undegraded":"2024-09-21T03:15:00.260269+0000","last_fullsized":"2024-09-21T03:15:00.260269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.271029+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:00.271029+0000","last_peered":"2024-09-21T03:15:00.271029+0000","last_clean":"2024-09-21T03:15:00.271029+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:00.271029+0000","last_undegraded":"2024-09-21T03:15:00.271029+0000","last_fullsized":"2024-09-21T03:15:00.271029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:05.790115+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:05.790115+0000","last_peered":"2024-09-21T03:15:05.790115+0000","last_clean":"2024-09-21T03:15:05.790115+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:05.790115+0000","last_undegraded":"2024-09-21T03:15:05.790115+0000","last_fullsized":"2024-09-21T03:15:05.790115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260427+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:00.260427+0000","last_peered":"2024-09-21T03:15:00.260427+0000","last_clean":"2024-09-21T03:15:00.260427+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:00.260427+0000","last_undegraded":"2024-09-21T03:15:00.260427+0000","last_fullsized":"2024-09-21T03:15:00.260427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259765+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259765+0000","last_peered":"2024-09-21T03:15:00.259765+0000","last_clean":"2024-09-21T03:15:00.259765+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259765+0000","last_undegraded":"2024-09-21T03:15:00.259765+0000","last_fullsized":"2024-09-21T03:15:00.259765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258614+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:00.258614+0000","last_peered":"2024-09-21T03:15:00.258614+0000","last_clean":"2024-09-21T03:15:00.258614+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:00.258614+0000","last_undegraded":"2024-09-21T03:15:00.258614+0000","last_fullsized":"2024-09-21T03:15:00.258614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259707+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259707+0000","last_peered":"2024-09-21T03:15:00.259707+0000","last_clean":"2024-09-21T03:15:00.259707+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259707+0000","last_undegraded":"2024-09-21T03:15:00.259707+0000","last_fullsized":"2024-09-21T03:15:00.259707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:04.787883+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:04.787883+0000","last_peered":"2024-09-21T03:15:04.787883+0000","last_clean":"2024-09-21T03:15:04.787883+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:04.787883+0000","last_undegraded":"2024-09-21T03:15:04.787883+0000","last_fullsized":"2024-09-21T03:15:04.787883+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264186+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:00.264186+0000","last_peered":"2024-09-21T03:15:00.264186+0000","last_clean":"2024-09-21T03:15:00.264186+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:00.264186+0000","last_undegraded":"2024-09-21T03:15:00.264186+0000","last_fullsized":"2024-09-21T03:15:00.264186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":46,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:03.786787+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:15:03.786787+0000","last_peered":"2024-09-21T03:15:03.786787+0000","last_clean":"2024-09-21T03:15:03.786787+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:15:03.786787+0000","last_undegraded":"2024-09-21T03:15:03.786787+0000","last_fullsized":"2024-09-21T03:15:03.786787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:02.607478+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:02.607478+0000","last_peered":"2024-09-21T03:15:02.607478+0000","last_clean":"2024-09-21T03:15:02.607478+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:02.607478+0000","last_undegraded":"2024-09-21T03:15:02.607478+0000","last_fullsized":"2024-09-21T03:15:02.607478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259126+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:00.259126+0000","last_peered":"2024-09-21T03:15:00.259126+0000","last_clean":"2024-09-21T03:15:00.259126+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:00.259126+0000","last_undegraded":"2024-09-21T03:15:00.259126+0000","last_fullsized":"2024-09-21T03:15:00.259126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258634+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:00.258634+0000","last_peered":"2024-09-21T03:15:00.258634+0000","last_clean":"2024-09-21T03:15:00.258634+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:00.258634+0000","last_undegraded":"2024-09-21T03:15:00.258634+0000","last_fullsized":"2024-09-21T03:15:00.258634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259779+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:00.259779+0000","last_peered":"2024-09-21T03:15:00.259779+0000","last_clean":"2024-09-21T03:15:00.259779+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:00.259779+0000","last_undegraded":"2024-09-21T03:15:00.259779+0000","last_fullsized":"2024-09-21T03:15:00.259779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.267843+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:00.267843+0000","last_peered":"2024-09-21T03:15:00.267843+0000","last_clean":"2024-09-21T03:15:00.267843+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:00.267843+0000","last_undegraded":"2024-09-21T03:15:00.267843+0000","last_fullsized":"2024-09-21T03:15:00.267843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259025+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259025+0000","last_peered":"2024-09-21T03:15:00.259025+0000","last_clean":"2024-09-21T03:15:00.259025+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259025+0000","last_undegraded":"2024-09-21T03:15:00.259025+0000","last_fullsized":"2024-09-21T03:15:00.259025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258780+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:00.258780+0000","last_peered":"2024-09-21T03:15:00.258780+0000","last_clean":"2024-09-21T03:15:00.258780+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:00.258780+0000","last_undegraded":"2024-09-21T03:15:00.258780+0000","last_fullsized":"2024-09-21T03:15:00.258780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:05.788814+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:05.788814+0000","last_peered":"2024-09-21T03:15:05.788814+0000","last_clean":"2024-09-21T03:15:05.788814+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:05.788814+0000","last_undegraded":"2024-09-21T03:15:05.788814+0000","last_fullsized":"2024-09-21T03:15:05.788814+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69582,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":229,"num_read_kb":189,"num_write":241,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":208746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1070095,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070493,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266721,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626476,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822706,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":480169,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":479626,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:10 smithi193 ceph-mon[35224]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:10 smithi083 ceph-mon[31219]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:10 smithi142 ceph-mon[34512]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:11.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:11 smithi193 ceph-mon[35224]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:11 smithi083 ceph-mon[31219]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:11 smithi142 ceph-mon[34512]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:13.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:13.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 -- 172.21.15.83:0/1779814105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 msgr2=0x7f99dc1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 --2- 172.21.15.83:0/1779814105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc1035d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009900 tx=0x7f99cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 -- 172.21.15.83:0/1779814105 shutdown_connections 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 --2- 172.21.15.83:0/1779814105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108f50 0x7f99dc10f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 --2- 172.21.15.83:0/1779814105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108600 0x7f99dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 --2- 172.21.15.83:0/1779814105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc1035d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 -- 172.21.15.83:0/1779814105 >> 172.21.15.83:0/1779814105 conn(0x7f99dc079e30 msgr2=0x7f99dc07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 -- 172.21.15.83:0/1779814105 shutdown_connections 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.986+0000 7f99e21b3640 1 -- 172.21.15.83:0/1779814105 wait complete. 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 Processor -- start 2024-09-21T03:15:13.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 -- start start 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108600 0x7f99dc199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 0x7f99dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112c70 con 0x7f99dc1031c0 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112af0 con 0x7f99dc108f50 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99e21b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112df0 con 0x7f99dc108600 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43752/0 (socket says 172.21.15.83:43752) 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.987+0000 7f99db7fe640 1 -- 172.21.15.83:0/4271288572 learned_addr learned my addr 172.21.15.83:0/4271288572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99d3fff640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108600 0x7f99dc199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99dbfff640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 0x7f99dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 -- 172.21.15.83:0/4271288572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108600 msgr2=0x7f99dc199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108600 0x7f99dc199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 -- 172.21.15.83:0/4271288572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 msgr2=0x7f99dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 0x7f99dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 -- 172.21.15.83:0/4271288572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc19f670 con 0x7f99dc1031c0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99dbfff640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 0x7f99dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99db7fe640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005940 tx=0x7f99cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc03f030 con 0x7f99dc1031c0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc003cb0 con 0x7f99dc1031c0 2024-09-21T03:15:13.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc19f900 con 0x7f99dc1031c0 2024-09-21T03:15:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc035070 con 0x7f99dc1031c0 2024-09-21T03:15:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.988+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a60c0 con 0x7f99dc1031c0 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.989+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc1035d0 con 0x7f99dc1031c0 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.991+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f99cc042030 con 0x7f99dc1031c0 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.991+0000 7f99d97fa640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 0x7f99c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.992+0000 7f99d3fff640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 0x7f99c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.992+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f99cc107e00 con 0x7f99dc1031c0 2024-09-21T03:15:13.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.992+0000 7f99d3fff640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 0x7f99c80b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99dc06b160 tx=0x7f99c400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:13.996+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99cc0ce490 con 0x7f99dc1031c0 2024-09-21T03:15:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.172+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99dc063800 con 0x7f99c80b6670 2024-09-21T03:15:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.176+0000 7f99d97fa640 1 -- 172.21.15.83:0/4271288572 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259016 (secure 0 0 0) 0x7f99dc063800 con 0x7f99c80b6670 2024-09-21T03:15:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 msgr2=0x7f99c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 0x7f99c80b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99dc06b160 tx=0x7f99c400b390 comp rx=0 tx=0).stop 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 msgr2=0x7f99dc199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005940 tx=0x7f99cc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 shutdown_connections 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6670 0x7f99c80b8b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99dc108f50 0x7f99dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99dc108600 0x7f99dc199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 --2- 172.21.15.83:0/4271288572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99dc1031c0 0x7f99dc199700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 >> 172.21.15.83:0/4271288572 conn(0x7f99dc079e30 msgr2=0x7f99dc106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 shutdown_connections 2024-09-21T03:15:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:14.179+0000 7f99e21b3640 1 -- 172.21.15.83:0/4271288572 wait complete. 2024-09-21T03:15:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:14 smithi193 ceph-mon[35224]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:14.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:14 smithi083 ceph-mon[31219]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:14 smithi142 ceph-mon[34512]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T03:15:14.997 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-21T03:15:13.177326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":356,"num_read_kb":285,"num_write":368,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351440,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590384,"statfs":{"total":1151940427776,"available":1151580553216,"internally_reserved":0,"allocated":10727424,"data_stored":6967776,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19554,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":95,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":185,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003341"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259613+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:00.259613+0000","last_peered":"2024-09-21T03:15:00.259613+0000","last_clean":"2024-09-21T03:15:00.259613+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:00.259613+0000","last_undegraded":"2024-09-21T03:15:00.259613+0000","last_fullsized":"2024-09-21T03:15:00.259613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258368+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:00.258368+0000","last_peered":"2024-09-21T03:15:00.258368+0000","last_clean":"2024-09-21T03:15:00.258368+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:00.258368+0000","last_undegraded":"2024-09-21T03:15:00.258368+0000","last_fullsized":"2024-09-21T03:15:00.258368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260247+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:00.260247+0000","last_peered":"2024-09-21T03:15:00.260247+0000","last_clean":"2024-09-21T03:15:00.260247+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:00.260247+0000","last_undegraded":"2024-09-21T03:15:00.260247+0000","last_fullsized":"2024-09-21T03:15:00.260247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258439+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:00.258439+0000","last_peered":"2024-09-21T03:15:00.258439+0000","last_clean":"2024-09-21T03:15:00.258439+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:00.258439+0000","last_undegraded":"2024-09-21T03:15:00.258439+0000","last_fullsized":"2024-09-21T03:15:00.258439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263181+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:00.263181+0000","last_peered":"2024-09-21T03:15:00.263181+0000","last_clean":"2024-09-21T03:15:00.263181+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:00.263181+0000","last_undegraded":"2024-09-21T03:15:00.263181+0000","last_fullsized":"2024-09-21T03:15:00.263181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258788+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:00.258788+0000","last_peered":"2024-09-21T03:15:00.258788+0000","last_clean":"2024-09-21T03:15:00.258788+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:00.258788+0000","last_undegraded":"2024-09-21T03:15:00.258788+0000","last_fullsized":"2024-09-21T03:15:00.258788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:07.793019+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:15:07.793019+0000","last_peered":"2024-09-21T03:15:07.793019+0000","last_clean":"2024-09-21T03:15:07.793019+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:15:07.793019+0000","last_undegraded":"2024-09-21T03:15:07.793019+0000","last_fullsized":"2024-09-21T03:15:07.793019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263742+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:00.263742+0000","last_peered":"2024-09-21T03:15:00.263742+0000","last_clean":"2024-09-21T03:15:00.263742+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:00.263742+0000","last_undegraded":"2024-09-21T03:15:00.263742+0000","last_fullsized":"2024-09-21T03:15:00.263742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258447+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:00.258447+0000","last_peered":"2024-09-21T03:15:00.258447+0000","last_clean":"2024-09-21T03:15:00.258447+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:00.258447+0000","last_undegraded":"2024-09-21T03:15:00.258447+0000","last_fullsized":"2024-09-21T03:15:00.258447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:11.797719+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:11.797719+0000","last_peered":"2024-09-21T03:15:11.797719+0000","last_clean":"2024-09-21T03:15:11.797719+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:11.797719+0000","last_undegraded":"2024-09-21T03:15:11.797719+0000","last_fullsized":"2024-09-21T03:15:11.797719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'30","reported_seq":110,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:11.800819+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:11.800819+0000","last_peered":"2024-09-21T03:15:11.800819+0000","last_clean":"2024-09-21T03:15:11.800819+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:11.800819+0000","last_undegraded":"2024-09-21T03:15:11.800819+0000","last_fullsized":"2024-09-21T03:15:11.800819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64757,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":90,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260024+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:00.260024+0000","last_peered":"2024-09-21T03:15:00.260024+0000","last_clean":"2024-09-21T03:15:00.260024+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:00.260024+0000","last_undegraded":"2024-09-21T03:15:00.260024+0000","last_fullsized":"2024-09-21T03:15:00.260024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:11.798073+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:11.798073+0000","last_peered":"2024-09-21T03:15:11.798073+0000","last_clean":"2024-09-21T03:15:11.798073+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:11.798073+0000","last_undegraded":"2024-09-21T03:15:11.798073+0000","last_fullsized":"2024-09-21T03:15:11.798073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259547+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:00.259547+0000","last_peered":"2024-09-21T03:15:00.259547+0000","last_clean":"2024-09-21T03:15:00.259547+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:00.259547+0000","last_undegraded":"2024-09-21T03:15:00.259547+0000","last_fullsized":"2024-09-21T03:15:00.259547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263368+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:00.263368+0000","last_peered":"2024-09-21T03:15:00.263368+0000","last_clean":"2024-09-21T03:15:00.263368+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:00.263368+0000","last_undegraded":"2024-09-21T03:15:00.263368+0000","last_fullsized":"2024-09-21T03:15:00.263368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260269+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:00.260269+0000","last_peered":"2024-09-21T03:15:00.260269+0000","last_clean":"2024-09-21T03:15:00.260269+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:00.260269+0000","last_undegraded":"2024-09-21T03:15:00.260269+0000","last_fullsized":"2024-09-21T03:15:00.260269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.271029+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:00.271029+0000","last_peered":"2024-09-21T03:15:00.271029+0000","last_clean":"2024-09-21T03:15:00.271029+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:00.271029+0000","last_undegraded":"2024-09-21T03:15:00.271029+0000","last_fullsized":"2024-09-21T03:15:00.271029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:10.796851+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:10.796851+0000","last_peered":"2024-09-21T03:15:10.796851+0000","last_clean":"2024-09-21T03:15:10.796851+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:10.796851+0000","last_undegraded":"2024-09-21T03:15:10.796851+0000","last_fullsized":"2024-09-21T03:15:10.796851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260427+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:00.260427+0000","last_peered":"2024-09-21T03:15:00.260427+0000","last_clean":"2024-09-21T03:15:00.260427+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:00.260427+0000","last_undegraded":"2024-09-21T03:15:00.260427+0000","last_fullsized":"2024-09-21T03:15:00.260427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259765+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259765+0000","last_peered":"2024-09-21T03:15:00.259765+0000","last_clean":"2024-09-21T03:15:00.259765+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259765+0000","last_undegraded":"2024-09-21T03:15:00.259765+0000","last_fullsized":"2024-09-21T03:15:00.259765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258614+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:00.258614+0000","last_peered":"2024-09-21T03:15:00.258614+0000","last_clean":"2024-09-21T03:15:00.258614+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:00.258614+0000","last_undegraded":"2024-09-21T03:15:00.258614+0000","last_fullsized":"2024-09-21T03:15:00.258614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259707+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259707+0000","last_peered":"2024-09-21T03:15:00.259707+0000","last_clean":"2024-09-21T03:15:00.259707+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259707+0000","last_undegraded":"2024-09-21T03:15:00.259707+0000","last_fullsized":"2024-09-21T03:15:00.259707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:09.794582+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:09.794582+0000","last_peered":"2024-09-21T03:15:09.794582+0000","last_clean":"2024-09-21T03:15:09.794582+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:09.794582+0000","last_undegraded":"2024-09-21T03:15:09.794582+0000","last_fullsized":"2024-09-21T03:15:09.794582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264186+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:00.264186+0000","last_peered":"2024-09-21T03:15:00.264186+0000","last_clean":"2024-09-21T03:15:00.264186+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:00.264186+0000","last_undegraded":"2024-09-21T03:15:00.264186+0000","last_fullsized":"2024-09-21T03:15:00.264186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:08.793232+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:15:08.793232+0000","last_peered":"2024-09-21T03:15:08.793232+0000","last_clean":"2024-09-21T03:15:08.793232+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:15:08.793232+0000","last_undegraded":"2024-09-21T03:15:08.793232+0000","last_fullsized":"2024-09-21T03:15:08.793232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:02.607478+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:02.607478+0000","last_peered":"2024-09-21T03:15:02.607478+0000","last_clean":"2024-09-21T03:15:02.607478+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:02.607478+0000","last_undegraded":"2024-09-21T03:15:02.607478+0000","last_fullsized":"2024-09-21T03:15:02.607478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259126+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:00.259126+0000","last_peered":"2024-09-21T03:15:00.259126+0000","last_clean":"2024-09-21T03:15:00.259126+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:00.259126+0000","last_undegraded":"2024-09-21T03:15:00.259126+0000","last_fullsized":"2024-09-21T03:15:00.259126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258634+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:00.258634+0000","last_peered":"2024-09-21T03:15:00.258634+0000","last_clean":"2024-09-21T03:15:00.258634+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:00.258634+0000","last_undegraded":"2024-09-21T03:15:00.258634+0000","last_fullsized":"2024-09-21T03:15:00.258634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259779+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:00.259779+0000","last_peered":"2024-09-21T03:15:00.259779+0000","last_clean":"2024-09-21T03:15:00.259779+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:00.259779+0000","last_undegraded":"2024-09-21T03:15:00.259779+0000","last_fullsized":"2024-09-21T03:15:00.259779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.267843+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:00.267843+0000","last_peered":"2024-09-21T03:15:00.267843+0000","last_clean":"2024-09-21T03:15:00.267843+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:00.267843+0000","last_undegraded":"2024-09-21T03:15:00.267843+0000","last_fullsized":"2024-09-21T03:15:00.267843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259025+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259025+0000","last_peered":"2024-09-21T03:15:00.259025+0000","last_clean":"2024-09-21T03:15:00.259025+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259025+0000","last_undegraded":"2024-09-21T03:15:00.259025+0000","last_fullsized":"2024-09-21T03:15:00.259025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258780+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:00.258780+0000","last_peered":"2024-09-21T03:15:00.258780+0000","last_clean":"2024-09-21T03:15:00.258780+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:00.258780+0000","last_undegraded":"2024-09-21T03:15:00.258780+0000","last_fullsized":"2024-09-21T03:15:00.258780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:10.795487+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:10.795487+0000","last_peered":"2024-09-21T03:15:10.795487+0000","last_clean":"2024-09-21T03:15:10.795487+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:10.795487+0000","last_undegraded":"2024-09-21T03:15:10.795487+0000","last_fullsized":"2024-09-21T03:15:10.795487+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":275,"num_read_kb":216,"num_write":243,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743443,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939673,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072083,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626477,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822706,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":482157,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":481614,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:15.002 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:15:15.002 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:15:15.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:15:15.251 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:15.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:15 smithi083 ceph-mon[31219]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:15 smithi193 ceph-mon[35224]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:15 smithi142 ceph-mon[34512]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 -- 172.21.15.83:0/1355266174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c102a80 msgr2=0x7f914c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 --2- 172.21.15.83:0/1355266174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c102a80 0x7f914c102e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f913c009900 tx=0x7f913c02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 -- 172.21.15.83:0/1355266174 shutdown_connections 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 --2- 172.21.15.83:0/1355266174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c108d10 0x7f914c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 --2- 172.21.15.83:0/1355266174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c1083c0 0x7f914c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 --2- 172.21.15.83:0/1355266174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c102a80 0x7f914c102e90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 -- 172.21.15.83:0/1355266174 >> 172.21.15.83:0/1355266174 conn(0x7f914c0798c0 msgr2=0x7f914c079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 -- 172.21.15.83:0/1355266174 shutdown_connections 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.763+0000 7f9152878640 1 -- 172.21.15.83:0/1355266174 wait complete. 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 Processor -- start 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 -- start start 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 0x7f914c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 0x7f914c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f914c112a10 con 0x7f914c1083c0 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f914c112890 con 0x7f914c102a80 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f9152878640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f914c112b90 con 0x7f914c108d10 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43778/0 (socket says 172.21.15.83:43778) 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.764+0000 7f914b7fe640 1 -- 172.21.15.83:0/520616134 learned_addr learned my addr 172.21.15.83:0/520616134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f9150e15640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 0x7f914c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914bfff640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 0x7f914c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914b7fe640 1 -- 172.21.15.83:0/520616134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 msgr2=0x7f914c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914b7fe640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 0x7f914c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914b7fe640 1 -- 172.21.15.83:0/520616134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 msgr2=0x7f914c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914b7fe640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 0x7f914c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914b7fe640 1 -- 172.21.15.83:0/520616134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c1a6500 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f9150e15640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 0x7f914c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.765+0000 7f914bfff640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 0x7f914c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f914b7fe640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9140006fd0 tx=0x7f914000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140011830 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9140016070 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140010e00 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c1a67f0 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.766+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1a6db0 con 0x7f914c1083c0 2024-09-21T03:15:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.767+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c103d00 con 0x7f914c1083c0 2024-09-21T03:15:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.769+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f91400119d0 con 0x7f914c1083c0 2024-09-21T03:15:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.770+0000 7f91497fa640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 0x7f91340b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.770+0000 7f914bfff640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 0x7f91340b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:15.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.770+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f91400e71a0 con 0x7f914c1083c0 2024-09-21T03:15:15.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.771+0000 7f914bfff640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 0x7f91340b8bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f913c0096c0 tx=0x7f913c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:15.773+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91400ad830 con 0x7f914c1083c0 2024-09-21T03:15:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.021+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f914c06bf10 con 0x7f914c1083c0 2024-09-21T03:15:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.024+0000 7f91497fa640 1 -- 172.21.15.83:0/520616134 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f91400b1c10 con 0x7f914c1083c0 2024-09-21T03:15:16.025 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-09-21T03:15:16.025 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBEOu5mxm1WARAAL8An+qOkW+g4rQv8YDw7lA== 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 msgr2=0x7f91340b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 0x7f91340b8bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f913c0096c0 tx=0x7f913c031400 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 msgr2=0x7f914c19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9140006fd0 tx=0x7f914000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 shutdown_connections 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f91340b66a0 0x7f91340b8bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914c108d10 0x7f914c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c1083c0 0x7f914c19b840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 --2- 172.21.15.83:0/520616134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c102a80 0x7f914c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 >> 172.21.15.83:0/520616134 conn(0x7f914c0798c0 msgr2=0x7f914c106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:16.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 shutdown_connections 2024-09-21T03:15:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:16.026+0000 7f9152878640 1 -- 172.21.15.83:0/520616134 wait complete. 2024-09-21T03:15:16.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:16 smithi083 ceph-mon[31219]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:16.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:15:16.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:15:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:16 smithi193 ceph-mon[35224]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:15:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:15:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:16 smithi142 ceph-mon[34512]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:15:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/520616134' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:15:16.792 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:15:16.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:15:16.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:15:16.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:15:17.112 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 -- 172.21.15.83:0/3078014050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8102a60 msgr2=0x7f06d8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/3078014050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8102a60 0x7f06d8102e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009900 tx=0x7f06c802a410 comp rx=0 tx=0).stop 2024-09-21T03:15:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 -- 172.21.15.83:0/3078014050 shutdown_connections 2024-09-21T03:15:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/3078014050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8108cf0 0x7f06d810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/3078014050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 0x7f06d81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/3078014050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8102a60 0x7f06d8102e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 -- 172.21.15.83:0/3078014050 >> 172.21.15.83:0/3078014050 conn(0x7f06d8079730 msgr2=0x7f06d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 -- 172.21.15.83:0/3078014050 shutdown_connections 2024-09-21T03:15:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.601+0000 7f06ddb0c640 1 -- 172.21.15.83:0/3078014050 wait complete. 2024-09-21T03:15:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 Processor -- start 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 -- start start 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8102a60 0x7f06d819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 0x7f06d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112a40 con 0x7f06d8108cf0 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81128c0 con 0x7f06d81083a0 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06ddb0c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112bc0 con 0x7f06d8102a60 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:17.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43798/0 (socket says 172.21.15.83:43798) 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06d7fff640 1 -- 172.21.15.83:0/1892800589 learned_addr learned my addr 172.21.15.83:0/1892800589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06d77fe640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8102a60 0x7f06d819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.602+0000 7f06d6ffd640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 0x7f06d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 -- 172.21.15.83:0/1892800589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8102a60 msgr2=0x7f06d819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8102a60 0x7f06d819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 -- 172.21.15.83:0/1892800589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 msgr2=0x7f06d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 0x7f06d819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 -- 172.21.15.83:0/1892800589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81a04f0 con 0x7f06d8108cf0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d7fff640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f06d000b970 tx=0x7f06d000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0019070 con 0x7f06d8108cf0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81a07e0 con 0x7f06d8108cf0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d0014440 con 0x7f06d8108cf0 2024-09-21T03:15:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00075d0 con 0x7f06d8108cf0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.603+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81a6f40 con 0x7f06d8108cf0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.604+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d8102ef0 con 0x7f06d8108cf0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.606+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f06d001c070 con 0x7f06d8108cf0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.606+0000 7f06d4ff9640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 0x7f06c00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.606+0000 7f06d77fe640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 0x7f06c00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.606+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f06d00e6e60 con 0x7f06d8108cf0 2024-09-21T03:15:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.607+0000 7f06d77fe640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 0x7f06c00b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06c8031440 tx=0x7f06c8005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.609+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06d00b18d0 con 0x7f06d8108cf0 2024-09-21T03:15:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.787+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f06d819b6b0 con 0x7f06d8108cf0 2024-09-21T03:15:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.787+0000 7f06d4ff9640 1 -- 172.21.15.83:0/1892800589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v54) ==== 86+0+133 (secure 0 0 0) 0x7f06d819b6b0 con 0x7f06d8108cf0 2024-09-21T03:15:17.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 msgr2=0x7f06c00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 0x7f06c00b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06c8031440 tx=0x7f06c8005e80 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 msgr2=0x7f06d819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f06d000b970 tx=0x7f06d000bd70 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 shutdown_connections 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f06c00b65f0 0x7f06c00b8b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06d8108cf0 0x7f06d819fdc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d81083a0 0x7f06d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 --2- 172.21.15.83:0/1892800589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06d8102a60 0x7f06d819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 >> 172.21.15.83:0/1892800589 conn(0x7f06d8079730 msgr2=0x7f06d8101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 shutdown_connections 2024-09-21T03:15:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:17.790+0000 7f06ddb0c640 1 -- 172.21.15.83:0/1892800589 wait complete. 2024-09-21T03:15:18.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:18 smithi083 ceph-mon[31219]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:18.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:18 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1892800589' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:15:18.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:18 smithi193 ceph-mon[35224]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:18.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:18 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1892800589' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:15:18.583 INFO:teuthology.orchestra.run.smithi083.stdout:[{"version":54,"timestamp":"2024-09-21T03:14:28.355211+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:15:18.583 INFO:tasks.ceph_manager:config epoch is 54 2024-09-21T03:15:18.584 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:15:18.584 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:15:18.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:15:18.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:18 smithi142 ceph-mon[34512]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T03:15:18.685 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:18 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1892800589' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:15:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 -- 172.21.15.83:0/488132729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0328109e00 msgr2=0x7f0328110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 --2- 172.21.15.83:0/488132729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0328109e00 0x7f0328110630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0318009ca0 tx=0x7f031802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 -- 172.21.15.83:0/488132729 shutdown_connections 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 --2- 172.21.15.83:0/488132729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0328109e00 0x7f0328110630 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 --2- 172.21.15.83:0/488132729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03281094b0 0x7f03281098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 --2- 172.21.15.83:0/488132729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328103b70 0x7f0328103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.330+0000 7f033003e640 1 -- 172.21.15.83:0/488132729 >> 172.21.15.83:0/488132729 conn(0x7f03280ff5e0 msgr2=0x7f0328101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.331+0000 7f033003e640 1 -- 172.21.15.83:0/488132729 shutdown_connections 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.331+0000 7f033003e640 1 -- 172.21.15.83:0/488132729 wait complete. 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.331+0000 7f033003e640 1 Processor -- start 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.331+0000 7f033003e640 1 -- start start 2024-09-21T03:15:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.331+0000 7f033003e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 0x7f032819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328109e00 0x7f032819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0328113b10 con 0x7f03281094b0 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0328113990 con 0x7f0328103b70 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0328113c90 con 0x7f0328109e00 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43324/0 (socket says 172.21.15.83:43324) 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 -- 172.21.15.83:0/3657529460 learned_addr learned my addr 172.21.15.83:0/3657529460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032e5db640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328109e00 0x7f032819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032ddda640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 0x7f032819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 -- 172.21.15.83:0/3657529460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328109e00 msgr2=0x7f032819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328109e00 0x7f032819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 -- 172.21.15.83:0/3657529460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 msgr2=0x7f032819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 0x7f032819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 -- 172.21.15.83:0/3657529460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281a03b0 con 0x7f03281094b0 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032ddda640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 0x7f032819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f032d5d9640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c970 tx=0x7f031c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c007bb0 con 0x7f03281094b0 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c007d50 con 0x7f03281094b0 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00f450 con 0x7f03281094b0 2024-09-21T03:15:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.332+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281a06a0 con 0x7f03281094b0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.333+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a6e00 con 0x7f03281094b0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.334+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328103f80 con 0x7f03281094b0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.335+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f031c004eb0 con 0x7f03281094b0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.336+0000 7f0326ffd640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 0x7f03100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.336+0000 7f032ddda640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 0x7f03100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.336+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f031c0e7670 con 0x7f03281094b0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.336+0000 7f032ddda640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 0x7f03100b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f03200058c0 tx=0x7f0320005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.339+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f031c014030 con 0x7f03281094b0 2024-09-21T03:15:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.567+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f032802e640 con 0x7f03281094b0 2024-09-21T03:15:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.572+0000 7f0326ffd640 1 -- 172.21.15.83:0/3657529460 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296453 (secure 0 0 0) 0x7f031c0add00 con 0x7f03281094b0 2024-09-21T03:15:19.575 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 msgr2=0x7f03100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 0x7f03100b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f03200058c0 tx=0x7f0320005850 comp rx=0 tx=0).stop 2024-09-21T03:15:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 msgr2=0x7f032819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c970 tx=0x7f031c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 shutdown_connections 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f03100b65f0 0x7f03100b8b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0328109e00 0x7f032819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281094b0 0x7f032819a9e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 --2- 172.21.15.83:0/3657529460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0328103b70 0x7f032819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 >> 172.21.15.83:0/3657529460 conn(0x7f03280ff5e0 msgr2=0x7f0328106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 shutdown_connections 2024-09-21T03:15:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:19.575+0000 7f033003e640 1 -- 172.21.15.83:0/3657529460 wait complete. 2024-09-21T03:15:20.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:20 smithi083 ceph-mon[31219]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T03:15:20.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3657529460' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:15:20.509 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"flags":0,"active_gid":14217,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1298689131},{"type":"v1","addr":"172.21.15.83:6801","nonce":1298689131}]},"active_addr":"172.21.15.83:6801/1298689131","active_change":"2024-09-21T03:07:59.080322+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24130,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24161,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3367477869}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":488129215}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1478872012}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2132424413}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3430050212}]}]} 2024-09-21T03:15:20.515 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:15:20.515 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:15:20.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:15:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:20 smithi193 ceph-mon[35224]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T03:15:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3657529460' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:15:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:20 smithi142 ceph-mon[34512]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T03:15:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3657529460' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:15:20.763 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 -- 172.21.15.83:0/2708934811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c102a60 msgr2=0x7ff37c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 --2- 172.21.15.83:0/2708934811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c102a60 0x7ff37c102e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff378009900 tx=0x7ff37802a410 comp rx=0 tx=0).stop 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 -- 172.21.15.83:0/2708934811 shutdown_connections 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 --2- 172.21.15.83:0/2708934811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c108cf0 0x7ff37c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 --2- 172.21.15.83:0/2708934811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c1083a0 0x7ff37c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 --2- 172.21.15.83:0/2708934811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c102a60 0x7ff37c102e70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 -- 172.21.15.83:0/2708934811 >> 172.21.15.83:0/2708934811 conn(0x7ff37c079730 msgr2=0x7ff37c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 -- 172.21.15.83:0/2708934811 shutdown_connections 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.273+0000 7ff383891640 1 -- 172.21.15.83:0/2708934811 wait complete. 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 Processor -- start 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 -- start start 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c102a60 0x7ff37c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c1083a0 0x7ff37c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c112960 con 0x7ff37c108cf0 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1127e0 con 0x7ff37c102a60 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff383891640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c112ae0 con 0x7ff37c1083a0 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff381e2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff381e2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43344/0 (socket says 172.21.15.83:43344) 2024-09-21T03:15:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff381e2e640 1 -- 172.21.15.83:0/2871006558 learned_addr learned my addr 172.21.15.83:0/2871006558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.274+0000 7ff38162d640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c102a60 0x7ff37c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 -- 172.21.15.83:0/2871006558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c1083a0 msgr2=0x7ff37c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff380e2c640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c1083a0 0x7ff37c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c1083a0 0x7ff37c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 -- 172.21.15.83:0/2871006558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c102a60 msgr2=0x7ff37c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c102a60 0x7ff37c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 -- 172.21.15.83:0/2871006558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c1a8710 con 0x7ff37c108cf0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff381e2e640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff37400cad0 tx=0x7ff37400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37400ae20 con 0x7ff37c108cf0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374005020 con 0x7ff37c108cf0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c1a8a00 con 0x7ff37c108cf0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37400f6b0 con 0x7ff37c108cf0 2024-09-21T03:15:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.275+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c1a8f40 con 0x7ff37c108cf0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.276+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c102e70 con 0x7ff37c108cf0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.278+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff3740051d0 con 0x7ff37c108cf0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.278+0000 7ff36e7fc640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 0x7ff3680b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.279+0000 7ff38162d640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 0x7ff3680b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.279+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7ff3740e7930 con 0x7ff37c108cf0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.279+0000 7ff38162d640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 0x7ff3680b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff378000c00 tx=0x7ff378031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.281+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff374014030 con 0x7ff37c108cf0 2024-09-21T03:15:21.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.459+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff37c02e640 con 0x7ff37c108cf0 2024-09-21T03:15:21.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.460+0000 7ff36e7fc640 1 -- 172.21.15.83:0/2871006558 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7ff3740ae040 con 0x7ff37c108cf0 2024-09-21T03:15:21.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:21.463 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:15:21.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 msgr2=0x7ff3680b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:21.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 0x7ff3680b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff378000c00 tx=0x7ff378031400 comp rx=0 tx=0).stop 2024-09-21T03:15:21.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 msgr2=0x7ff37c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff37400cad0 tx=0x7ff37400cf90 comp rx=0 tx=0).stop 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 shutdown_connections 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff3680b6670 0x7ff3680b8b80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37c108cf0 0x7ff37c199ae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff37c1083a0 0x7ff37c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 --2- 172.21.15.83:0/2871006558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff37c102a60 0x7ff37c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 >> 172.21.15.83:0/2871006558 conn(0x7ff37c079730 msgr2=0x7ff37c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 shutdown_connections 2024-09-21T03:15:21.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:21.463+0000 7ff383891640 1 -- 172.21.15.83:0/2871006558 wait complete. 2024-09-21T03:15:22.274 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:15:22.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:15:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:22.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:22 smithi193 ceph-mon[35224]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-21T03:15:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:22 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2871006558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:22 smithi083 ceph-mon[31219]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-21T03:15:22.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:22 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2871006558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:22 smithi142 ceph-mon[34512]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-21T03:15:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:22 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2871006558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2054170032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108420 msgr2=0x7efdb4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2054170032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108420 0x7efdb4108830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efda0009900 tx=0x7efda002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2054170032 shutdown_connections 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2054170032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4108d70 0x7efdb410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2054170032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108420 0x7efdb4108830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2054170032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4102ae0 0x7efdb4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2054170032 >> 172.21.15.83:0/2054170032 conn(0x7efdb4079770 msgr2=0x7efdb4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2054170032 shutdown_connections 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.041+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2054170032 wait complete. 2024-09-21T03:15:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 Processor -- start 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 -- start start 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 0x7efdb419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4108420 0x7efdb419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4112a30 con 0x7efdb4108d70 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41128b0 con 0x7efdb4108420 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdbaf9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4112bb0 con 0x7efdb4102ae0 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.042+0000 7efdb953c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43372/0 (socket says 172.21.15.83:43372) 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 -- 172.21.15.83:0/2923865723 learned_addr learned my addr 172.21.15.83:0/2923865723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb8d3b640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 0x7efdb419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb3fff640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4108420 0x7efdb419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 -- 172.21.15.83:0/2923865723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 msgr2=0x7efdb419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 0x7efdb419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 -- 172.21.15.83:0/2923865723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4108420 msgr2=0x7efdb419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4108420 0x7efdb419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 -- 172.21.15.83:0/2923865723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41a6650 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb8d3b640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 0x7efdb419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb953c640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdac00eaa0 tx=0x7efdac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00cdb0 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41a6940 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdac005020 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0116d0 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.043+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41a6e50 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.045+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb4103d60 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.047+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7efdac0051d0 con 0x7efdb4108d70 2024-09-21T03:15:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.048+0000 7efdb1ffb640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 0x7efda40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.048+0000 7efdb8d3b640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 0x7efda40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.048+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7efdac0e7ba0 con 0x7efdb4108d70 2024-09-21T03:15:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.048+0000 7efdb8d3b640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 0x7efda40b8b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda8005ec0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.051+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efdac016030 con 0x7efdb4108d70 2024-09-21T03:15:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.233+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdb4102ef0 con 0x7efdb4108d70 2024-09-21T03:15:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.235+0000 7efdb1ffb640 1 -- 172.21.15.83:0/2923865723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7efdac0ae1d0 con 0x7efdb4108d70 2024-09-21T03:15:23.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:23.238 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 msgr2=0x7efda40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 0x7efda40b8b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda8005ec0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 msgr2=0x7efdb419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdac00eaa0 tx=0x7efdac00ef60 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 shutdown_connections 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efda40b6620 0x7efda40b8b30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb4108d70 0x7efdb419be40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4108420 0x7efdb419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 --2- 172.21.15.83:0/2923865723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb4102ae0 0x7efdb419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 >> 172.21.15.83:0/2923865723 conn(0x7efdb4079770 msgr2=0x7efdb41065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 shutdown_connections 2024-09-21T03:15:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:23.237+0000 7efdbaf9f640 1 -- 172.21.15.83:0/2923865723 wait complete. 2024-09-21T03:15:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:23 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2923865723' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:23.591 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:23 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2923865723' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:23 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2923865723' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:15:23.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:15:23.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:15:23.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:15:23.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:15:23.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:15:23.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:15:23.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:15:23.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:15:23.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:15:23.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:15:23.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:15:23.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:15:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:24 smithi193 ceph-mon[35224]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:15:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:24 smithi083 ceph-mon[31219]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:15:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:24 smithi142 ceph-mon[34512]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T03:15:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.765 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.770 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.770 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.770 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 -- 172.21.15.83:0/1153019205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52181094a0 msgr2=0x7f52181098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 --2- 172.21.15.83:0/1153019205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52181094a0 0x7f52181098b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f520c009900 tx=0x7f520c02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 -- 172.21.15.83:0/1153019205 shutdown_connections 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 --2- 172.21.15.83:0/1153019205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5218109df0 0x7f5218110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 --2- 172.21.15.83:0/1153019205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52181094a0 0x7f52181098b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 --2- 172.21.15.83:0/1153019205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5218103b60 0x7f5218103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 -- 172.21.15.83:0/1153019205 >> 172.21.15.83:0/1153019205 conn(0x7f521807b850 msgr2=0x7f521807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 -- 172.21.15.83:0/1153019205 shutdown_connections 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.522+0000 7f521d20d640 1 -- 172.21.15.83:0/1153019205 wait complete. 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 Processor -- start 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 -- start start 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5218103b60 0x7f521819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52181094a0 0x7f521819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 0x7f521819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113ab0 con 0x7f5218109df0 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113930 con 0x7f5218103b60 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.523+0000 7f521d20d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113c30 con 0x7f52181094a0 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 0x7f521819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 0x7f521819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43382/0 (socket says 172.21.15.83:43382) 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 -- 172.21.15.83:0/1244311131 learned_addr learned my addr 172.21.15.83:0/1244311131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521659c640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52181094a0 0x7f521819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f5216d9d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5218103b60 0x7f521819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 -- 172.21.15.83:0/1244311131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52181094a0 msgr2=0x7f521819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52181094a0 0x7f521819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 -- 172.21.15.83:0/1244311131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5218103b60 msgr2=0x7f521819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5218103b60 0x7f521819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52181a03f0 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521659c640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52181094a0 0x7f521819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521759e640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 0x7f521819fcc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f521000d750 tx=0x7f521000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210017070 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52181a06e0 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52100054d0 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210004c60 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.524+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a6e40 con 0x7f5218109df0 2024-09-21T03:15:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.525+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5218063b50 con 0x7f5218109df0 2024-09-21T03:15:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.527+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5210009bd0 con 0x7f5218109df0 2024-09-21T03:15:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f51fffff640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52000b65f0 0x7f52000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f5216d9d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52000b65f0 0x7f52000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f52100e75f0 con 0x7f5218109df0 2024-09-21T03:15:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f51fffff640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] conn(0x7f52000c02e0 0x7f52000c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5210021e50 con 0x7f52000c02e0 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f52100e7a40 con 0x7f5218109df0 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f521659c640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] conn(0x7f52000c02e0 0x7f52000c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f5216d9d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52000b65f0 0x7f52000b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5208005fd0 tx=0x7f5208009870 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.528+0000 7f521659c640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] conn(0x7f52000c02e0 0x7f52000c27d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.529+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== osd.0 v2:172.21.15.83:6802/4290610745 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5210021e50 con 0x7f52000c02e0 2024-09-21T03:15:25.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.542+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 --> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5218063f50 con 0x7f52000c02e0 2024-09-21T03:15:25.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f51fffff640 1 -- 172.21.15.83:0/1244311131 <== osd.0 v2:172.21.15.83:6802/4290610745 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5218063f50 con 0x7f52000c02e0 2024-09-21T03:15:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] conn(0x7f52000c02e0 msgr2=0x7f52000c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] conn(0x7f52000c02e0 0x7f52000c27d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52000b65f0 msgr2=0x7f52000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52000b65f0 0x7f52000b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5208005fd0 tx=0x7f5208009870 comp rx=0 tx=0).stop 2024-09-21T03:15:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 msgr2=0x7f521819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 --2- 172.21.15.83:0/1244311131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5218109df0 0x7f521819fcc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f521000d750 tx=0x7f521000dc10 comp rx=0 tx=0).stop 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521759e640 1 -- 172.21.15.83:0/1244311131 reap_dead start 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 shutdown_connections 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 >> 172.21.15.83:0/1244311131 conn(0x7f521807b850 msgr2=0x7f5218105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 shutdown_connections 2024-09-21T03:15:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:25.543+0000 7f521d20d640 1 -- 172.21.15.83:0/1244311131 wait complete. 2024-09-21T03:15:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:26.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:26 smithi083 ceph-mon[31219]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:26.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:26 smithi193 ceph-mon[35224]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:26 smithi142 ceph-mon[34512]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 -- 172.21.15.83:0/422199337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c108cf0 msgr2=0x7f3d0c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 --2- 172.21.15.83:0/422199337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c10f520 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ca0 tx=0x7f3d0402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 -- 172.21.15.83:0/422199337 shutdown_connections 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 --2- 172.21.15.83:0/422199337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c10f520 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 --2- 172.21.15.83:0/422199337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c1083a0 0x7f3d0c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 --2- 172.21.15.83:0/422199337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c102a60 0x7f3d0c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 -- 172.21.15.83:0/422199337 >> 172.21.15.83:0/422199337 conn(0x7f3d0c079730 msgr2=0x7f3d0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 -- 172.21.15.83:0/422199337 shutdown_connections 2024-09-21T03:15:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.742+0000 7f3d133eb640 1 -- 172.21.15.83:0/422199337 wait complete. 2024-09-21T03:15:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 Processor -- start 2024-09-21T03:15:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 -- start start 2024-09-21T03:15:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 0x7f3d0c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c1083a0 0x7f3d0c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1129b0 con 0x7f3d0c102a60 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c112830 con 0x7f3d0c1083a0 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.743+0000 7f3d133eb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c112b30 con 0x7f3d0c108cf0 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 0x7f3d0c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 0x7f3d0c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43444/0 (socket says 172.21.15.83:43444) 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 -- 172.21.15.83:0/3239181006 learned_addr learned my addr 172.21.15.83:0/3239181006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d10986640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c1083a0 0x7f3d0c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11988640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 -- 172.21.15.83:0/3239181006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c108cf0 msgr2=0x7f3d0c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 -- 172.21.15.83:0/3239181006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c1083a0 msgr2=0x7f3d0c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c1083a0 0x7f3d0c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c19f270 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11988640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c108cf0 0x7f3d0c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d10986640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d0c1083a0 0x7f3d0c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d11187640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 0x7f3d0c199300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c890 tx=0x7f3d0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004220 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c19f560 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d080043c0 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004a00 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.744+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c06ae60 con 0x7f3d0c102a60 2024-09-21T03:15:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.745+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d0c06b430 con 0x7f3d0c102a60 2024-09-21T03:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.747+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f3d080079b0 con 0x7f3d0c102a60 2024-09-21T03:15:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.748+0000 7f3d027fc640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf80b65f0 0x7f3cf80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.748+0000 7f3d10986640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf80b65f0 0x7f3cf80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.748+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3d080e7830 con 0x7f3d0c102a60 2024-09-21T03:15:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d027fc640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] conn(0x7f3cf80c02e0 0x7f3cf80c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cf80c2ea0 con 0x7f3cf80c02e0 2024-09-21T03:15:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3d080e7c80 con 0x7f3d0c102a60 2024-09-21T03:15:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d11988640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] conn(0x7f3cf80c02e0 0x7f3cf80c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d10986640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf80b65f0 0x7f3cf80b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c105f70 tx=0x7f3cfc005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.749+0000 7f3d11988640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] conn(0x7f3cf80c02e0 0x7f3cf80c27d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.750+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== osd.9 v2:172.21.15.83:6826/400576662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cf80c2ea0 con 0x7f3cf80c02e0 2024-09-21T03:15:26.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.764+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 --> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d0c06b830 con 0x7f3cf80c02e0 2024-09-21T03:15:26.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.764+0000 7f3d027fc640 1 -- 172.21.15.83:0/3239181006 <== osd.9 v2:172.21.15.83:6826/400576662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d0c06b830 con 0x7f3cf80c02e0 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] conn(0x7f3cf80c02e0 msgr2=0x7f3cf80c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] conn(0x7f3cf80c02e0 0x7f3cf80c27d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf80b65f0 msgr2=0x7f3cf80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf80b65f0 0x7f3cf80b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c105f70 tx=0x7f3cfc005f90 comp rx=0 tx=0).stop 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 msgr2=0x7f3d0c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 --2- 172.21.15.83:0/3239181006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0c102a60 0x7f3d0c199300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c890 tx=0x7f3d0800cd50 comp rx=0 tx=0).stop 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d11988640 1 -- 172.21.15.83:0/3239181006 reap_dead start 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 shutdown_connections 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 >> 172.21.15.83:0/3239181006 conn(0x7f3d0c079730 msgr2=0x7f3d0c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 shutdown_connections 2024-09-21T03:15:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.765+0000 7f3d133eb640 1 -- 172.21.15.83:0/3239181006 wait complete. 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- 172.21.15.83:0/1126911035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc108d70 msgr2=0x7f2bdc10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1126911035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc108d70 0x7f2bdc10f5a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009ca0 tx=0x7f2bd402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- 172.21.15.83:0/1126911035 shutdown_connections 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1126911035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc108d70 0x7f2bdc10f5a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1126911035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 0x7f2bdc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1126911035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- 172.21.15.83:0/1126911035 >> 172.21.15.83:0/1126911035 conn(0x7f2bdc079770 msgr2=0x7f2bdc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- 172.21.15.83:0/1126911035 shutdown_connections 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- 172.21.15.83:0/1126911035 wait complete. 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 Processor -- start 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.527+0000 7f2be38ac640 1 -- start start 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 0x7f2bdc199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc108d70 0x7f2bdc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc112a90 con 0x7f2bdc102ae0 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc112910 con 0x7f2bdc108420 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be38ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc112c10 con 0x7f2bdc108d70 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43408/0 (socket says 172.21.15.83:43408) 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 -- 172.21.15.83:0/1854232261 learned_addr learned my addr 172.21.15.83:0/1854232261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1e49640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc108d70 0x7f2bdc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be0e47640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 0x7f2bdc199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 -- 172.21.15.83:0/1854232261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc108d70 msgr2=0x7f2bdc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc108d70 0x7f2bdc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 -- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 msgr2=0x7f2bdc199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 0x7f2bdc199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc19f410 con 0x7f2bdc102ae0 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1e49640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bdc108d70 0x7f2bdc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be0e47640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bdc108420 0x7f2bdc199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2be1648640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc1994a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d770 tx=0x7f2bd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8017070 con 0x7f2bdc102ae0 2024-09-21T03:15:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.528+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd80054d0 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.529+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc19f700 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.529+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004c30 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.529+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a5e60 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.529+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2bdc063ad0 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.531+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2bd8004dd0 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2bd27fc640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bc80b6670 0x7f2bc80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2be0e47640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bc80b6670 0x7f2bc80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2bd80e74d0 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2bd27fc640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] conn(0x7f2bc80c0360 0x7f2bc80c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8021e50 con 0x7f2bc80c0360 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f2bd80e7920 con 0x7f2bdc102ae0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2be0e47640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bc80b6670 0x7f2bc80b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006fd0 tx=0x7f2bcc008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.532+0000 7f2be1e49640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] conn(0x7f2bc80c0360 0x7f2bc80c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.533+0000 7f2be1e49640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] conn(0x7f2bc80c0360 0x7f2bc80c2850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.534+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== osd.10 v2:172.21.15.142:6824/2781310607 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8021e50 con 0x7f2bc80c0360 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.548+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 --> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2bdc063f50 con 0x7f2bc80c0360 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.548+0000 7f2bd27fc640 1 -- 172.21.15.83:0/1854232261 <== osd.10 v2:172.21.15.142:6824/2781310607 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2bdc063f50 con 0x7f2bc80c0360 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] conn(0x7f2bc80c0360 msgr2=0x7f2bc80c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] conn(0x7f2bc80c0360 0x7f2bc80c2850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bc80b6670 msgr2=0x7f2bc80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bc80b6670 0x7f2bc80b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006fd0 tx=0x7f2bcc008040 comp rx=0 tx=0).stop 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 msgr2=0x7f2bdc1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 --2- 172.21.15.83:0/1854232261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc1994a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d770 tx=0x7f2bd800dc30 comp rx=0 tx=0).stop 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be1e49640 1 -- 172.21.15.83:0/1854232261 reap_dead start 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 shutdown_connections 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 >> 172.21.15.83:0/1854232261 conn(0x7f2bdc079770 msgr2=0x7f2bdc104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 shutdown_connections 2024-09-21T03:15:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.549+0000 7f2be38ac640 1 -- 172.21.15.83:0/1854232261 wait complete. 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 -- 172.21.15.83:0/726885353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75741083e0 msgr2=0x7f75741087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 --2- 172.21.15.83:0/726885353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75741083e0 0x7f75741087f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7568009900 tx=0x7f756802a410 comp rx=0 tx=0).stop 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 -- 172.21.15.83:0/726885353 shutdown_connections 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 --2- 172.21.15.83:0/726885353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7574108d30 0x7f757410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 --2- 172.21.15.83:0/726885353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75741083e0 0x7f75741087f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 --2- 172.21.15.83:0/726885353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7574102aa0 0x7f7574102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 -- 172.21.15.83:0/726885353 >> 172.21.15.83:0/726885353 conn(0x7f75740797e0 msgr2=0x7f7574079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 -- 172.21.15.83:0/726885353 shutdown_connections 2024-09-21T03:15:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.630+0000 7f7578b3a640 1 -- 172.21.15.83:0/726885353 wait complete. 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 Processor -- start 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 -- start start 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 0x7f757419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75741083e0 0x7f757419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7574108d30 0x7f757419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75741129a0 con 0x7f7574102aa0 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7574112820 con 0x7f7574108d30 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7578b3a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7574112b20 con 0x7f75741083e0 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f757259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 0x7f757419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f757259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 0x7f757419b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43422/0 (socket says 172.21.15.83:43422) 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f757259c640 1 -- 172.21.15.83:0/2186859114 learned_addr learned my addr 172.21.15.83:0/2186859114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:27.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7571d9b640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75741083e0 0x7f757419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.632+0000 7f7572d9d640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7574108d30 0x7f757419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 -- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75741083e0 msgr2=0x7f757419b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75741083e0 0x7f757419b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 -- 172.21.15.83:0/2186859114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7574108d30 msgr2=0x7f757419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7574108d30 0x7f757419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75741a6340 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f7571d9b640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75741083e0 0x7f757419b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f757259c640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 0x7f757419b0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400cd80 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75640108e0 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75741a6630 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564018980 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.633+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7574198ea0 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.634+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7574104c90 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.636+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7564010430 con 0x7f7574102aa0 2024-09-21T03:15:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f75637fe640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f755c0baa30 0x7f755c0bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f7571d9b640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f755c0baa30 0x7f755c0bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f75640e84f0 con 0x7f7574102aa0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f75637fe640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] conn(0x7f755c0c4720 0x7f755c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f755c0c72e0 con 0x7f755c0c4720 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f75640e8940 con 0x7f7574102aa0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f7571d9b640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f755c0baa30 0x7f755c0bcf40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f756800b230 tx=0x7f756800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.637+0000 7f7572d9d640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] conn(0x7f755c0c4720 0x7f755c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.638+0000 7f7572d9d640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] conn(0x7f755c0c4720 0x7f755c0c6c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.639+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== osd.2 v2:172.21.15.193:6800/905402231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f755c0c72e0 con 0x7f755c0c4720 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.667+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 --> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7574103d20 con 0x7f755c0c4720 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f75637fe640 1 -- 172.21.15.83:0/2186859114 <== osd.2 v2:172.21.15.193:6800/905402231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7574103d20 con 0x7f755c0c4720 2024-09-21T03:15:27.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] conn(0x7f755c0c4720 msgr2=0x7f755c0c6c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] conn(0x7f755c0c4720 0x7f755c0c6c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f755c0baa30 msgr2=0x7f755c0bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f755c0baa30 0x7f755c0bcf40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f756800b230 tx=0x7f756800b040 comp rx=0 tx=0).stop 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 msgr2=0x7f757419b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 --2- 172.21.15.83:0/2186859114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7574102aa0 0x7f757419b0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7572d9d640 1 -- 172.21.15.83:0/2186859114 reap_dead start 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 shutdown_connections 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.668+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 >> 172.21.15.83:0/2186859114 conn(0x7f75740797e0 msgr2=0x7f7574104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.669+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 shutdown_connections 2024-09-21T03:15:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:26.669+0000 7f7578b3a640 1 -- 172.21.15.83:0/2186859114 wait complete. 2024-09-21T03:15:27.985 INFO:teuthology.orchestra.run.smithi083.stdout:81604378693 2024-09-21T03:15:27.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:15:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 -- 172.21.15.83:0/3896151322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898108d30 msgr2=0x7f389810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 --2- 172.21.15.83:0/3896151322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898108d30 0x7f389810f560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3890009ca0 tx=0x7f389002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 -- 172.21.15.83:0/3896151322 shutdown_connections 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 --2- 172.21.15.83:0/3896151322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898108d30 0x7f389810f560 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 --2- 172.21.15.83:0/3896151322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38981083e0 0x7f38981087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 --2- 172.21.15.83:0/3896151322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3898102aa0 0x7f3898102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 -- 172.21.15.83:0/3896151322 >> 172.21.15.83:0/3896151322 conn(0x7f38980797e0 msgr2=0x7f3898079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 -- 172.21.15.83:0/3896151322 shutdown_connections 2024-09-21T03:15:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 -- 172.21.15.83:0/3896151322 wait complete. 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.153+0000 7f389d82d640 1 Processor -- start 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 -- start start 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 0x7f389819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38981083e0 0x7f389819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3898108d30 0x7f389819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3898113b20 con 0x7f3898102aa0 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38981139a0 con 0x7f3898108d30 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f389d82d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3898113ca0 con 0x7f38981083e0 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 0x7f389819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 0x7f389819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43468/0 (socket says 172.21.15.83:43468) 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 -- 172.21.15.83:0/3223955701 learned_addr learned my addr 172.21.15.83:0/3223955701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f38967fc640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38981083e0 0x7f389819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f38977fe640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3898108d30 0x7f389819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 -- 172.21.15.83:0/3223955701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38981083e0 msgr2=0x7f389819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38981083e0 0x7f389819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 -- 172.21.15.83:0/3223955701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3898108d30 msgr2=0x7f389819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3898108d30 0x7f389819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f3896ffd640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981a0400 con 0x7f3898102aa0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.154+0000 7f38977fe640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3898108d30 0x7f389819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f3896ffd640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 0x7f389819a490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f388800d770 tx=0x7f388800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888017070 con 0x7f3898102aa0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981a06f0 con 0x7f3898102aa0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a6e50 con 0x7f3898102aa0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3888009820 con 0x7f3898102aa0 2024-09-21T03:15:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.155+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888004bc0 con 0x7f3898102aa0 2024-09-21T03:15:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.156+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3898063ad0 con 0x7f3898102aa0 2024-09-21T03:15:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.158+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f38880099d0 con 0x7f3898102aa0 2024-09-21T03:15:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.158+0000 7f389c82b640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38840b65f0 0x7f38840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.158+0000 7f38967fc640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38840b65f0 0x7f38840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f38880e74f0 con 0x7f3898102aa0 2024-09-21T03:15:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f389c82b640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] conn(0x7f38840c02e0 0x7f38840c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3888021e50 con 0x7f38840c02e0 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f38880e7940 con 0x7f3898102aa0 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f38977fe640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] conn(0x7f38840c02e0 0x7f38840c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f38967fc640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38840b65f0 0x7f38840b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3898107aa0 tx=0x7f3880009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.159+0000 7f38977fe640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] conn(0x7f38840c02e0 0x7f38840c27d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.160+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== osd.3 v2:172.21.15.83:6810/4077666910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3888021e50 con 0x7f38840c02e0 2024-09-21T03:15:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.174+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 --> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3898063f50 con 0x7f38840c02e0 2024-09-21T03:15:28.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.174+0000 7f389c82b640 1 -- 172.21.15.83:0/3223955701 <== osd.3 v2:172.21.15.83:6810/4077666910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3898063f50 con 0x7f38840c02e0 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] conn(0x7f38840c02e0 msgr2=0x7f38840c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] conn(0x7f38840c02e0 0x7f38840c27d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38840b65f0 msgr2=0x7f38840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f38840b65f0 0x7f38840b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3898107aa0 tx=0x7f3880009110 comp rx=0 tx=0).stop 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 msgr2=0x7f389819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 --2- 172.21.15.83:0/3223955701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3898102aa0 0x7f389819a490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f388800d770 tx=0x7f388800db70 comp rx=0 tx=0).stop 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f38977fe640 1 -- 172.21.15.83:0/3223955701 reap_dead start 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 shutdown_connections 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.175+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 >> 172.21.15.83:0/3223955701 conn(0x7f38980797e0 msgr2=0x7f3898105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.176+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 shutdown_connections 2024-09-21T03:15:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:28.176+0000 7f389d82d640 1 -- 172.21.15.83:0/3223955701 wait complete. 2024-09-21T03:15:28.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:28 smithi193 ceph-mon[35224]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T03:15:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:28 smithi083 ceph-mon[31219]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T03:15:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:28 smithi142 ceph-mon[34512]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T03:15:29.026 INFO:teuthology.orchestra.run.smithi083.stdout:38654705742 2024-09-21T03:15:29.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:15:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.947+0000 7f245dd32640 1 -- 172.21.15.83:0/2705409175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24581094c0 msgr2=0x7f24581098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.947+0000 7f245dd32640 1 --2- 172.21.15.83:0/2705409175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24581094c0 0x7f24581098d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f244c009900 tx=0x7f244c02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 -- 172.21.15.83:0/2705409175 shutdown_connections 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 --2- 172.21.15.83:0/2705409175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458109e10 0x7f2458110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 --2- 172.21.15.83:0/2705409175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24581094c0 0x7f24581098d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 --2- 172.21.15.83:0/2705409175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2458103b80 0x7f2458103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 -- 172.21.15.83:0/2705409175 >> 172.21.15.83:0/2705409175 conn(0x7f24580ff5d0 msgr2=0x7f2458101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 -- 172.21.15.83:0/2705409175 shutdown_connections 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 -- 172.21.15.83:0/2705409175 wait complete. 2024-09-21T03:15:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 Processor -- start 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.948+0000 7f245dd32640 1 -- start start 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 0x7f245819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24581094c0 0x7f245819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458109e10 0x7f245819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2458113af0 con 0x7f2458103b80 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2458113970 con 0x7f2458109e10 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f245dd32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2458113c70 con 0x7f24581094c0 2024-09-21T03:15:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 0x7f245819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 0x7f245819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34302/0 (socket says 172.21.15.83:34302) 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 -- 172.21.15.83:0/2475718086 learned_addr learned my addr 172.21.15.83:0/2475718086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f2457fff640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458109e10 0x7f245819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f2456ffd640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24581094c0 0x7f245819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 -- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24581094c0 msgr2=0x7f245819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24581094c0 0x7f245819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 -- 172.21.15.83:0/2475718086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458109e10 msgr2=0x7f245819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458109e10 0x7f245819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24581a0470 con 0x7f2458103b80 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f24577fe640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 0x7f245819a500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f244800b4a0 tx=0x7f244800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.949+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448014070 con 0x7f2458103b80 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.950+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581a0760 con 0x7f2458103b80 2024-09-21T03:15:29.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.950+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2448010ac0 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.950+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448017c30 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.950+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a6ec0 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.951+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2458063b50 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.952+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2448017070 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2454ff9640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24400b65f0 0x7f24400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2456ffd640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24400b65f0 0x7f24400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f24480e6bb0 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2454ff9640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] conn(0x7f24400c02e0 0x7f24400c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24400c2ea0 con 0x7f24400c02e0 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f24480e7000 con 0x7f2458103b80 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.953+0000 7f2456ffd640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24400b65f0 0x7f24400b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f244c009230 tx=0x7f244c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.954+0000 7f2457fff640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] conn(0x7f24400c02e0 0x7f24400c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.954+0000 7f2457fff640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] conn(0x7f24400c02e0 0x7f24400c27d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.955+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== osd.5 v2:172.21.15.193:6808/2703219317 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24400c2ea0 con 0x7f24400c02e0 2024-09-21T03:15:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.969+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 --> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2458063f80 con 0x7f24400c02e0 2024-09-21T03:15:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.969+0000 7f2454ff9640 1 -- 172.21.15.83:0/2475718086 <== osd.5 v2:172.21.15.193:6808/2703219317 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2458063f80 con 0x7f24400c02e0 2024-09-21T03:15:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] conn(0x7f24400c02e0 msgr2=0x7f24400c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] conn(0x7f24400c02e0 0x7f24400c27d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24400b65f0 msgr2=0x7f24400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f24400b65f0 0x7f24400b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f244c009230 tx=0x7f244c031400 comp rx=0 tx=0).stop 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 msgr2=0x7f245819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 --2- 172.21.15.83:0/2475718086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2458103b80 0x7f245819a500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f244800b4a0 tx=0x7f244800b960 comp rx=0 tx=0).stop 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f2457fff640 1 -- 172.21.15.83:0/2475718086 reap_dead start 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 shutdown_connections 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 >> 172.21.15.83:0/2475718086 conn(0x7f24580ff5d0 msgr2=0x7f24581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 shutdown_connections 2024-09-21T03:15:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:29.970+0000 7f245dd32640 1 -- 172.21.15.83:0/2475718086 wait complete. 2024-09-21T03:15:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:30 smithi193 ceph-mon[35224]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:30 smithi083 ceph-mon[31219]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:30.661 INFO:teuthology.orchestra.run.smithi083.stdout:154618822711 2024-09-21T03:15:30.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:15:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:30 smithi142 ceph-mon[34512]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 -- 172.21.15.83:0/240109138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec54102a60 msgr2=0x7fec54102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 --2- 172.21.15.83:0/240109138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec54102a60 0x7fec54102e70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fec44009900 tx=0x7fec4402a410 comp rx=0 tx=0).stop 2024-09-21T03:15:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 -- 172.21.15.83:0/240109138 shutdown_connections 2024-09-21T03:15:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 --2- 172.21.15.83:0/240109138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec54108cf0 0x7fec5410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 --2- 172.21.15.83:0/240109138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec541083a0 0x7fec541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 --2- 172.21.15.83:0/240109138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec54102a60 0x7fec54102e70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 -- 172.21.15.83:0/240109138 >> 172.21.15.83:0/240109138 conn(0x7fec54079730 msgr2=0x7fec54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 -- 172.21.15.83:0/240109138 shutdown_connections 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 -- 172.21.15.83:0/240109138 wait complete. 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.817+0000 7fec5a66e640 1 Processor -- start 2024-09-21T03:15:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 -- start start 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54102a60 0x7fec5419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 0x7fec5419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec54108cf0 0x7fec5419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec54112960 con 0x7fec541083a0 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec541127e0 con 0x7fec54102a60 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec5a66e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec54112ae0 con 0x7fec54108cf0 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 0x7fec5419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 0x7fec5419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34324/0 (socket says 172.21.15.83:34324) 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec537fe640 1 -- 172.21.15.83:0/2248049812 learned_addr learned my addr 172.21.15.83:0/2248049812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec53fff640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54102a60 0x7fec5419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec58c0b640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec54108cf0 0x7fec5419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec537fe640 1 -- 172.21.15.83:0/2248049812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec54108cf0 msgr2=0x7fec5419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.818+0000 7fec537fe640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec54108cf0 0x7fec5419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec537fe640 1 -- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54102a60 msgr2=0x7fec5419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec537fe640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54102a60 0x7fec5419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec537fe640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec541a03b0 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec537fe640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 0x7fec5419a9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fec4800e9c0 tx=0x7fec4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4800cd80 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec480054d0 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec541a06a0 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48010430 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.819+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541a6e00 con 0x7fec541083a0 2024-09-21T03:15:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.820+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec5406b1e0 con 0x7fec541083a0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.822+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fec48004fb0 con 0x7fec541083a0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec517fa640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec3c0b65f0 0x7fec3c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec53fff640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec3c0b65f0 0x7fec3c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec480e8070 con 0x7fec541083a0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec517fa640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] conn(0x7fec3c0c02e0 0x7fec3c0c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec48015e80 con 0x7fec3c0c02e0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fec480e8550 con 0x7fec541083a0 2024-09-21T03:15:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.823+0000 7fec58c0b640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] conn(0x7fec3c0c02e0 0x7fec3c0c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.824+0000 7fec53fff640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec3c0b65f0 0x7fec3c0b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec44031440 tx=0x7fec44005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.825+0000 7fec58c0b640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] conn(0x7fec3c0c02e0 0x7fec3c0c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.825+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== osd.4 v2:172.21.15.142:6808/4220832012 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec48015e80 con 0x7fec3c0c02e0 2024-09-21T03:15:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.840+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 --> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec54103ce0 con 0x7fec3c0c02e0 2024-09-21T03:15:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.841+0000 7fec517fa640 1 -- 172.21.15.83:0/2248049812 <== osd.4 v2:172.21.15.142:6808/4220832012 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec54103ce0 con 0x7fec3c0c02e0 2024-09-21T03:15:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] conn(0x7fec3c0c02e0 msgr2=0x7fec3c0c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] conn(0x7fec3c0c02e0 0x7fec3c0c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec3c0b65f0 msgr2=0x7fec3c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec3c0b65f0 0x7fec3c0b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec44031440 tx=0x7fec44005d50 comp rx=0 tx=0).stop 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 msgr2=0x7fec5419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 --2- 172.21.15.83:0/2248049812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec541083a0 0x7fec5419a9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fec4800e9c0 tx=0x7fec4800ee80 comp rx=0 tx=0).stop 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec58c0b640 1 -- 172.21.15.83:0/2248049812 reap_dead start 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 shutdown_connections 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 >> 172.21.15.83:0/2248049812 conn(0x7fec54079730 msgr2=0x7fec54101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 shutdown_connections 2024-09-21T03:15:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:30.842+0000 7fec5a66e640 1 -- 172.21.15.83:0/2248049812 wait complete. 2024-09-21T03:15:31.224 INFO:teuthology.orchestra.run.smithi083.stdout:270582939676 2024-09-21T03:15:31.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:15:31.656 INFO:teuthology.orchestra.run.smithi083.stdout:253403070497 2024-09-21T03:15:31.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:15:32.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:32 smithi083 ceph-mon[31219]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:15:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:32 smithi193 ceph-mon[35224]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:15:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:32 smithi142 ceph-mon[34512]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:15:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 -- 172.21.15.83:0/3306706839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc102a90 msgr2=0x7f27bc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 --2- 172.21.15.83:0/3306706839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc102a90 0x7f27bc102ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009900 tx=0x7f27b002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 -- 172.21.15.83:0/3306706839 shutdown_connections 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 --2- 172.21.15.83:0/3306706839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27bc108d20 0x7f27bc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 --2- 172.21.15.83:0/3306706839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc1083d0 0x7f27bc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 --2- 172.21.15.83:0/3306706839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc102a90 0x7f27bc102ea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 -- 172.21.15.83:0/3306706839 >> 172.21.15.83:0/3306706839 conn(0x7f27bc079750 msgr2=0x7f27bc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 -- 172.21.15.83:0/3306706839 shutdown_connections 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.876+0000 7f27c43e3640 1 -- 172.21.15.83:0/3306706839 wait complete. 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 Processor -- start 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 -- start start 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27bc102a90 0x7f27bc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc1083d0 0x7f27bc199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 0x7f27bc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.877+0000 7f27c43e3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc112a70 con 0x7f27bc108d20 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c43e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1128f0 con 0x7f27bc102a90 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c43e3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc112bf0 con 0x7f27bc1083d0 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 0x7f27bc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 0x7f27bc19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34356/0 (socket says 172.21.15.83:34356) 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 -- 172.21.15.83:0/473088281 learned_addr learned my addr 172.21.15.83:0/473088281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c197e640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc1083d0 0x7f27bc199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c217f640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27bc102a90 0x7f27bc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 -- 172.21.15.83:0/473088281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc1083d0 msgr2=0x7f27bc199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc1083d0 0x7f27bc199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 -- 172.21.15.83:0/473088281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27bc102a90 msgr2=0x7f27bc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27bc102a90 0x7f27bc1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19f450 con 0x7f27bc108d20 2024-09-21T03:15:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c2980640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 0x7f27bc19ed20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b970 tx=0x7f27b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4019070 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b4014440 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc19f740 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40075d0 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.878+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a5ea0 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.879+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27bc063b10 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.881+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f27b401c070 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27af7fe640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27a80b65f0 0x7f27a80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27c217f640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27a80b65f0 0x7f27a80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f27b40e7330 con 0x7f27bc108d20 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27af7fe640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] conn(0x7f27a80c02e0 0x7f27a80c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27b4018ec0 con 0x7f27a80c02e0 2024-09-21T03:15:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f27b40e7780 con 0x7f27bc108d20 2024-09-21T03:15:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27c197e640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] conn(0x7f27a80c02e0 0x7f27a80c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27c217f640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27a80b65f0 0x7f27a80b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27b0031440 tx=0x7f27b0005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.882+0000 7f27c197e640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] conn(0x7f27a80c02e0 0x7f27a80c27d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.883+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== osd.6 v2:172.21.15.83:6818/2541470109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27b4018ec0 con 0x7f27a80c02e0 2024-09-21T03:15:32.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 --> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27bc063f90 con 0x7f27a80c02e0 2024-09-21T03:15:32.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27af7fe640 1 -- 172.21.15.83:0/473088281 <== osd.6 v2:172.21.15.83:6818/2541470109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f27bc063f90 con 0x7f27a80c02e0 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] conn(0x7f27a80c02e0 msgr2=0x7f27a80c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] conn(0x7f27a80c02e0 0x7f27a80c27d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27a80b65f0 msgr2=0x7f27a80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f27a80b65f0 0x7f27a80b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27b0031440 tx=0x7f27b0005d50 comp rx=0 tx=0).stop 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 msgr2=0x7f27bc19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 --2- 172.21.15.83:0/473088281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27bc108d20 0x7f27bc19ed20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b970 tx=0x7f27b400be30 comp rx=0 tx=0).stop 2024-09-21T03:15:32.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c2980640 1 -- 172.21.15.83:0/473088281 reap_dead start 2024-09-21T03:15:32.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 shutdown_connections 2024-09-21T03:15:32.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 >> 172.21.15.83:0/473088281 conn(0x7f27bc079750 msgr2=0x7f27bc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:32.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 shutdown_connections 2024-09-21T03:15:32.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.897+0000 7f27c43e3640 1 -- 172.21.15.83:0/473088281 wait complete. 2024-09-21T03:15:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:33 smithi083 ceph-mon[31219]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 -- 172.21.15.83:0/3205095512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 msgr2=0x7fd1e010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 --2- 172.21.15.83:0/3205095512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e010ecb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009ca0 tx=0x7fd1d802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 -- 172.21.15.83:0/3205095512 shutdown_connections 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 --2- 172.21.15.83:0/3205095512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e010ecb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 --2- 172.21.15.83:0/3205095512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0107b30 0x7fd1e0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 --2- 172.21.15.83:0/3205095512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1e01021f0 0x7fd1e0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 -- 172.21.15.83:0/3205095512 >> 172.21.15.83:0/3205095512 conn(0x7fd1e0077c60 msgr2=0x7fd1e00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 -- 172.21.15.83:0/3205095512 shutdown_connections 2024-09-21T03:15:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.000+0000 7fd1e71be640 1 -- 172.21.15.83:0/3205095512 wait complete. 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 Processor -- start 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 -- start start 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1e01021f0 0x7fd1e0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0107b30 0x7fd1e0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0112200 con 0x7fd1e0108480 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0112080 con 0x7fd1e0107b30 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e71be640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0112380 con 0x7fd1e01021f0 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34340/0 (socket says 172.21.15.83:34340) 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 -- 172.21.15.83:0/1597296240 learned_addr learned my addr 172.21.15.83:0/1597296240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 -- 172.21.15.83:0/1597296240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1e01021f0 msgr2=0x7fd1e0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1e01021f0 0x7fd1e0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 -- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0107b30 msgr2=0x7fd1e0199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1e0107b30 0x7fd1e0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.001+0000 7fd1e575b640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e019f470 con 0x7fd1e0108480 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1e575b640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e019ed40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8005cc0 tx=0x7fd1d8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004040 con 0x7fd1e0108480 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e019f700 con 0x7fd1e0108480 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1d8004690 con 0x7fd1e0108480 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8030e20 con 0x7fd1e0108480 2024-09-21T03:15:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.002+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e01a5ec0 con 0x7fd1e0108480 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.003+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1e01016b0 con 0x7fd1e0108480 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.004+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd1d80305e0 con 0x7fd1e0108480 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1ddffb640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd1d00b65f0 0x7fd1d00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1e4f5a640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd1d00b65f0 0x7fd1d00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd1d8108ce0 con 0x7fd1e0108480 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1ddffb640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] conn(0x7fd1d00c02e0 0x7fd1d00c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1d8044e50 con 0x7fd1d00c02e0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.005+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd1d8109130 con 0x7fd1e0108480 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.006+0000 7fd1e4f5a640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd1d00b65f0 0x7fd1d00b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009710 tx=0x7fd1d4006c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.006+0000 7fd1dffff640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] conn(0x7fd1d00c02e0 0x7fd1d00c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.006+0000 7fd1dffff640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] conn(0x7fd1d00c02e0 0x7fd1d00c27d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.007+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== osd.7 v2:172.21.15.142:6816/3641783594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1d8044e50 con 0x7fd1d00c02e0 2024-09-21T03:15:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.020+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 --> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1e0103470 con 0x7fd1d00c02e0 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1ddffb640 1 -- 172.21.15.83:0/1597296240 <== osd.7 v2:172.21.15.142:6816/3641783594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1e0103470 con 0x7fd1d00c02e0 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] conn(0x7fd1d00c02e0 msgr2=0x7fd1d00c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] conn(0x7fd1d00c02e0 0x7fd1d00c27d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd1d00b65f0 msgr2=0x7fd1d00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd1d00b65f0 0x7fd1d00b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009710 tx=0x7fd1d4006c90 comp rx=0 tx=0).stop 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 msgr2=0x7fd1e019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 --2- 172.21.15.83:0/1597296240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1e0108480 0x7fd1e019ed40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8005cc0 tx=0x7fd1d8003ba0 comp rx=0 tx=0).stop 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e575b640 1 -- 172.21.15.83:0/1597296240 reap_dead start 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 shutdown_connections 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.021+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 >> 172.21.15.83:0/1597296240 conn(0x7fd1e0077c60 msgr2=0x7fd1e0100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.022+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 shutdown_connections 2024-09-21T03:15:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:32.022+0000 7fd1e71be640 1 -- 172.21.15.83:0/1597296240 wait complete. 2024-09-21T03:15:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:33 smithi142 ceph-mon[34512]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-21T03:15:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:33 smithi193 ceph-mon[35224]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-21T03:15:34.246 INFO:teuthology.orchestra.run.smithi083.stdout:201863462956 2024-09-21T03:15:34.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:15:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 -- 172.21.15.83:0/1041196374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 msgr2=0x7f833c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 --2- 172.21.15.83:0/1041196374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c102e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f832c009900 tx=0x7f832c02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 -- 172.21.15.83:0/1041196374 shutdown_connections 2024-09-21T03:15:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 --2- 172.21.15.83:0/1041196374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c108cf0 0x7f833c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 --2- 172.21.15.83:0/1041196374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833c1083a0 0x7f833c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 --2- 172.21.15.83:0/1041196374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c102e70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 -- 172.21.15.83:0/1041196374 >> 172.21.15.83:0/1041196374 conn(0x7f833c079730 msgr2=0x7f833c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.409+0000 7f83409e4640 1 -- 172.21.15.83:0/1041196374 shutdown_connections 2024-09-21T03:15:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.410+0000 7f83409e4640 1 -- 172.21.15.83:0/1041196374 wait complete. 2024-09-21T03:15:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.410+0000 7f83409e4640 1 Processor -- start 2024-09-21T03:15:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.410+0000 7f83409e4640 1 -- start start 2024-09-21T03:15:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c1083a0 0x7f833c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833c108cf0 0x7f833c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112960 con 0x7f833c102a60 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1127e0 con 0x7f833c108cf0 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f83409e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112ae0 con 0x7f833c1083a0 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f833a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f833a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34370/0 (socket says 172.21.15.83:34370) 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f833a59c640 1 -- 172.21.15.83:0/811221292 learned_addr learned my addr 172.21.15.83:0/811221292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c1083a0 0x7f833c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f833ad9d640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833c108cf0 0x7f833c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 -- 172.21.15.83:0/811221292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833c108cf0 msgr2=0x7f833c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f833c108cf0 0x7f833c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 -- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 msgr2=0x7f833c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f8339d9b640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c19f2a0 con 0x7f833c1083a0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.411+0000 7f833a59c640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f833c102a60 0x7f833c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f8339d9b640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c1083a0 0x7f833c1998d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9c0 tx=0x7f833000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833000ccc0 con 0x7f833c1083a0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c19f590 con 0x7f833c1083a0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83300108e0 con 0x7f833c1083a0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300187c0 con 0x7f833c1083a0 2024-09-21T03:15:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.412+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c06ae60 con 0x7f833c1083a0 2024-09-21T03:15:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.413+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f833c06b370 con 0x7f833c1083a0 2024-09-21T03:15:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.414+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8330010430 con 0x7f833c1083a0 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f832b7fe640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f83240b6670 0x7f83240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f833a59c640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f83240b6670 0x7f83240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f83300e8400 con 0x7f833c1083a0 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f832b7fe640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] conn(0x7f83240c0360 0x7f83240c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8330015e80 con 0x7f83240c0360 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.415+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f83300e8840 con 0x7f833c1083a0 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.416+0000 7f833ad9d640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] conn(0x7f83240c0360 0x7f83240c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.416+0000 7f833a59c640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f83240b6670 0x7f83240b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f832c031440 tx=0x7f832c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.416+0000 7f833ad9d640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] conn(0x7f83240c0360 0x7f83240c2850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.417+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== osd.8 v2:172.21.15.193:6816/1886733655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8330015e80 con 0x7f83240c0360 2024-09-21T03:15:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.430+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 --> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f833c06b770 con 0x7f83240c0360 2024-09-21T03:15:34.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f832b7fe640 1 -- 172.21.15.83:0/811221292 <== osd.8 v2:172.21.15.193:6816/1886733655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f833c06b770 con 0x7f83240c0360 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] conn(0x7f83240c0360 msgr2=0x7f83240c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] conn(0x7f83240c0360 0x7f83240c2850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f83240b6670 msgr2=0x7f83240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f83240b6670 0x7f83240b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f832c031440 tx=0x7f832c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c1083a0 msgr2=0x7f833c1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f83409e4640 1 --2- 172.21.15.83:0/811221292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f833c1083a0 0x7f833c1998d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9c0 tx=0x7f833000ee80 comp rx=0 tx=0).stop 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.431+0000 7f833ad9d640 1 -- 172.21.15.83:0/811221292 reap_dead start 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.432+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 shutdown_connections 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.432+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 >> 172.21.15.83:0/811221292 conn(0x7f833c079730 msgr2=0x7f833c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.432+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 shutdown_connections 2024-09-21T03:15:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:34.432+0000 7f83409e4640 1 -- 172.21.15.83:0/811221292 wait complete. 2024-09-21T03:15:34.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:34 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:15:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:34 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:15:34.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:34 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:15:35.194 INFO:teuthology.orchestra.run.smithi083.stdout:227633266727 2024-09-21T03:15:35.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:15:35.306 INFO:teuthology.orchestra.run.smithi083.stdout:111669149760 2024-09-21T03:15:35.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:15:35.779 INFO:teuthology.orchestra.run.smithi083.stdout:133143986236 2024-09-21T03:15:35.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:15:35.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:35.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:35.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:35.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:35 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:35.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:35 smithi083 ceph-mon[31219]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:15:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1266135462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 msgr2=0x7ff5ac103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1266135462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac103f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009900 tx=0x7ff59c02a410 comp rx=0 tx=0).stop 2024-09-21T03:15:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1266135462 shutdown_connections 2024-09-21T03:15:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1266135462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac109e10 0x7ff5ac110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1266135462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1094c0 0x7ff5ac1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1266135462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac103f90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1266135462 >> 172.21.15.83:0/1266135462 conn(0x7ff5ac0ff5b0 msgr2=0x7ff5ac101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1266135462 shutdown_connections 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.968+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1266135462 wait complete. 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.969+0000 7ff5b13fb640 1 Processor -- start 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.969+0000 7ff5b13fb640 1 -- start start 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac1094c0 0x7ff5ac19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac109e10 0x7ff5ac19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac113ad0 con 0x7ff5ac103b80 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac113950 con 0x7ff5ac1094c0 2024-09-21T03:15:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac113c50 con 0x7ff5ac109e10 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34392/0 (socket says 172.21.15.83:34392) 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 -- 172.21.15.83:0/1181008424 learned_addr learned my addr 172.21.15.83:0/1181008424 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5ab7fe640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac109e10 0x7ff5ac19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aa7fc640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac1094c0 0x7ff5ac19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 -- 172.21.15.83:0/1181008424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac109e10 msgr2=0x7ff5ac19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac109e10 0x7ff5ac19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 -- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac1094c0 msgr2=0x7ff5ac19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac1094c0 0x7ff5ac19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac1a0430 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aa7fc640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac1094c0 0x7ff5ac19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5aaffd640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac19a4c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff59c005940 tx=0x7ff59c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c03f030 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.970+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac1a06c0 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.971+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a6e80 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.971+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c003cb0 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.971+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c049060 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.971+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5ac063b10 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.973+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff59c042030 con 0x7ff5ac103b80 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff593fff640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff5940b65f0 0x7ff5940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff5aa7fc640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff5940b65f0 0x7ff5940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7ff59c107fd0 con 0x7ff5ac103b80 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff593fff640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] conn(0x7ff5940c02e0 0x7ff5940c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5940c2ea0 con 0x7ff5940c02e0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff59c108420 con 0x7ff5ac103b80 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff5aa7fc640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff5940b65f0 0x7ff5940b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00058c0 tx=0x7ff5a0005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.974+0000 7ff5ab7fe640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] conn(0x7ff5940c02e0 0x7ff5940c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.975+0000 7ff5ab7fe640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] conn(0x7ff5940c02e0 0x7ff5940c27d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.976+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== osd.1 v2:172.21.15.142:6800/3543209885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5940c2ea0 con 0x7ff5940c02e0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.989+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 --> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5ac063d60 con 0x7ff5940c02e0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.989+0000 7ff593fff640 1 -- 172.21.15.83:0/1181008424 <== osd.1 v2:172.21.15.142:6800/3543209885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff5ac063d60 con 0x7ff5940c02e0 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] conn(0x7ff5940c02e0 msgr2=0x7ff5940c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] conn(0x7ff5940c02e0 0x7ff5940c27d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff5940b65f0 msgr2=0x7ff5940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff5940b65f0 0x7ff5940b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00058c0 tx=0x7ff5a0005850 comp rx=0 tx=0).stop 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 msgr2=0x7ff5ac19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 --2- 172.21.15.83:0/1181008424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103b80 0x7ff5ac19a4c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff59c005940 tx=0x7ff59c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5ab7fe640 1 -- 172.21.15.83:0/1181008424 reap_dead start 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 shutdown_connections 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 >> 172.21.15.83:0/1181008424 conn(0x7ff5ac0ff5b0 msgr2=0x7ff5ac108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 shutdown_connections 2024-09-21T03:15:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:35.990+0000 7ff5b13fb640 1 -- 172.21.15.83:0/1181008424 wait complete. 2024-09-21T03:15:36.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:35 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:35 smithi193 ceph-mon[35224]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:15:36.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:35 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:36.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:35 smithi142 ceph-mon[34512]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 -- 172.21.15.83:0/3955990274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e0104880 msgr2=0x7fe7e0104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3955990274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e0104880 0x7fe7e0104c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009900 tx=0x7fe7d002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 -- 172.21.15.83:0/3955990274 shutdown_connections 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3955990274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e010de30 0x7fe7e0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3955990274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e01052b0 0x7fe7e010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3955990274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e0104880 0x7fe7e0104c90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 -- 172.21.15.83:0/3955990274 >> 172.21.15.83:0/3955990274 conn(0x7fe7e0079740 msgr2=0x7fe7e0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 -- 172.21.15.83:0/3955990274 shutdown_connections 2024-09-21T03:15:36.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.399+0000 7fe7e6101640 1 -- 172.21.15.83:0/3955990274 wait complete. 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.400+0000 7fe7e6101640 1 Processor -- start 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.400+0000 7fe7e6101640 1 -- start start 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e0104880 0x7fe7e019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 0x7fe7e019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e010de30 0x7fe7e019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0113b60 con 0x7fe7e01052b0 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01139e0 con 0x7fe7e010de30 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0113ce0 con 0x7fe7e0104880 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 0x7fe7e019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 0x7fe7e019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34416/0 (socket says 172.21.15.83:34416) 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 -- 172.21.15.83:0/3261864268 learned_addr learned my addr 172.21.15.83:0/3261864268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7df7fe640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e0104880 0x7fe7e019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7dffff640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e010de30 0x7fe7e019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 -- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e0104880 msgr2=0x7fe7e019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7e0104880 0x7fe7e019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 -- 172.21.15.83:0/3261864268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e010de30 msgr2=0x7fe7e019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e010de30 0x7fe7e019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01a0520 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7deffd640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 0x7fe7e019ab50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4011830 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d4016070 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e01a0810 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4010e00 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.401+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e01a6ff0 con 0x7fe7e01052b0 2024-09-21T03:15:36.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.402+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7e0063b50 con 0x7fe7e01052b0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.404+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe7d401b460 con 0x7fe7e01052b0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.404+0000 7fe7dcff9640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe7c80b6670 0x7fe7c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7df7fe640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe7c80b6670 0x7fe7c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe7d40e7510 con 0x7fe7e01052b0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dcff9640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] conn(0x7fe7c80c0360 0x7fe7c80c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7d4022e50 con 0x7fe7c80c0360 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe7d40e7960 con 0x7fe7e01052b0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7df7fe640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe7c80b6670 0x7fe7c80b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0031440 tx=0x7fe7d0005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dffff640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] conn(0x7fe7c80c0360 0x7fe7c80c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.405+0000 7fe7dffff640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] conn(0x7fe7c80c0360 0x7fe7c80c2850 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:36.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.406+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== osd.11 v2:172.21.15.193:6824/1672057754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7d4022e50 con 0x7fe7c80c0360 2024-09-21T03:15:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.420+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 --> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7e0063f80 con 0x7fe7c80c0360 2024-09-21T03:15:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7dcff9640 1 -- 172.21.15.83:0/3261864268 <== osd.11 v2:172.21.15.193:6824/1672057754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe7e0063f80 con 0x7fe7c80c0360 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] conn(0x7fe7c80c0360 msgr2=0x7fe7c80c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] conn(0x7fe7c80c0360 0x7fe7c80c2850 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe7c80b6670 msgr2=0x7fe7c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe7c80b6670 0x7fe7c80b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0031440 tx=0x7fe7d0005e80 comp rx=0 tx=0).stop 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 msgr2=0x7fe7e019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7e6101640 1 --2- 172.21.15.83:0/3261864268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7e01052b0 0x7fe7e019ab50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:15:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.421+0000 7fe7dffff640 1 -- 172.21.15.83:0/3261864268 reap_dead start 2024-09-21T03:15:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.422+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 shutdown_connections 2024-09-21T03:15:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.422+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 >> 172.21.15.83:0/3261864268 conn(0x7fe7e0079740 msgr2=0x7fe7e0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.422+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 shutdown_connections 2024-09-21T03:15:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:36.422+0000 7fe7e6101640 1 -- 172.21.15.83:0/3261864268 wait complete. 2024-09-21T03:15:37.111 INFO:teuthology.orchestra.run.smithi083.stdout:296352743448 2024-09-21T03:15:37.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:15:37.537 INFO:teuthology.orchestra.run.smithi083.stdout:180388626482 2024-09-21T03:15:37.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:15:37.559 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:37.566 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:38.024 INFO:teuthology.orchestra.run.smithi083.stdout:60129542219 2024-09-21T03:15:38.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:15:38.046 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:38.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:38 smithi083 ceph-mon[31219]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:38 smithi193 ceph-mon[35224]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:38 smithi142 ceph-mon[34512]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- 172.21.15.83:0/180280297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 msgr2=0x7fcc24110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/180280297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc24110300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ca0 tx=0x7fcc1c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- 172.21.15.83:0/180280297 shutdown_connections 2024-09-21T03:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/180280297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc24110300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/180280297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 0x7fcc2410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/180280297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc24104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- 172.21.15.83:0/180280297 >> 172.21.15.83:0/180280297 conn(0x7fcc24079730 msgr2=0x7fcc24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- 172.21.15.83:0/180280297 shutdown_connections 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- 172.21.15.83:0/180280297 wait complete. 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 Processor -- start 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.149+0000 7fcc2a6da640 1 -- start start 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc2419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 0x7fcc2419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24113a90 con 0x7fcc2410de10 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24113910 con 0x7fcc24105300 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc2a6da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24113c10 con 0x7fcc241049b0 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34434/0 (socket says 172.21.15.83:34434) 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 -- 172.21.15.83:0/214004011 learned_addr learned my addr 172.21.15.83:0/214004011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc1bfff640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 0x7fcc2419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc23fff640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc2419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 -- 172.21.15.83:0/214004011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 msgr2=0x7fcc2419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc2419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 -- 172.21.15.83:0/214004011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 msgr2=0x7fcc2419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 0x7fcc2419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 -- 172.21.15.83:0/214004011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc241a0390 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc23fff640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc2419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc28c77640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005cc0 tx=0x7fcc1c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.150+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003bf0 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.151+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc241a0620 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.151+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c003d90 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.151+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c02fbc0 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.151+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc241a6de0 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.151+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc24104dc0 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.153+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fcc1c02fd60 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.154+0000 7fcc21ffb640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 0x7fcc100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.154+0000 7fcc23fff640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 0x7fcc100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.154+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fcc1c108c30 con 0x7fcc2410de10 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.154+0000 7fcc23fff640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 0x7fcc100b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc140058c0 tx=0x7fcc14005ec0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.157+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc1c0cf2c0 con 0x7fcc2410de10 2024-09-21T03:15:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.344+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcc241a7010 con 0x7fcc2410de10 2024-09-21T03:15:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.344+0000 7fcc21ffb640 1 -- 172.21.15.83:0/214004011 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcc1c0d36a0 con 0x7fcc2410de10 2024-09-21T03:15:39.346 INFO:teuthology.orchestra.run.smithi083.stdout:296352743449 2024-09-21T03:15:39.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 msgr2=0x7fcc100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:39.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 0x7fcc100b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc140058c0 tx=0x7fcc14005ec0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 msgr2=0x7fcc2419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005cc0 tx=0x7fcc1c02f380 comp rx=0 tx=0).stop 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 shutdown_connections 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc100b65f0 0x7fcc100b8b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc2410de10 0x7fcc2419fc60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.346+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc24105300 0x7fcc2419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.347+0000 7fcc2a6da640 1 --2- 172.21.15.83:0/214004011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241049b0 0x7fcc2419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.347+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 >> 172.21.15.83:0/214004011 conn(0x7fcc24079730 msgr2=0x7fcc24101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.347+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 shutdown_connections 2024-09-21T03:15:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:39.347+0000 7fcc2a6da640 1 -- 172.21.15.83:0/214004011 wait complete. 2024-09-21T03:15:40.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743448 got 296352743449 for osd.11 2024-09-21T03:15:40.242 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 -- 172.21.15.83:0/3092498648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc741083a0 msgr2=0x7fdc741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/3092498648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc741083a0 0x7fdc741087b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009ca0 tx=0x7fdc6802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 -- 172.21.15.83:0/3092498648 shutdown_connections 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/3092498648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc74108cf0 0x7fdc7410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/3092498648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc741083a0 0x7fdc741087b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/3092498648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74102a60 0x7fdc74102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 -- 172.21.15.83:0/3092498648 >> 172.21.15.83:0/3092498648 conn(0x7fdc74079730 msgr2=0x7fdc74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 -- 172.21.15.83:0/3092498648 shutdown_connections 2024-09-21T03:15:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.404+0000 7fdc78dd7640 1 -- 172.21.15.83:0/3092498648 wait complete. 2024-09-21T03:15:40.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 Processor -- start 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 -- start start 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 0x7fdc74199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74108cf0 0x7fdc7419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74112960 con 0x7fdc74102a60 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741127e0 con 0x7fdc74108cf0 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc78dd7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74112ae0 con 0x7fdc741083a0 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc7259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc7259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47222/0 (socket says 172.21.15.83:47222) 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc7259c640 1 -- 172.21.15.83:0/1570390662 learned_addr learned my addr 172.21.15.83:0/1570390662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc72d9d640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74108cf0 0x7fdc7419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.405+0000 7fdc71d9b640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 0x7fdc74199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 -- 172.21.15.83:0/1570390662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 msgr2=0x7fdc74199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 0x7fdc74199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 -- 172.21.15.83:0/1570390662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74108cf0 msgr2=0x7fdc7419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74108cf0 0x7fdc7419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 -- 172.21.15.83:0/1570390662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7419f340 con 0x7fdc74102a60 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc71d9b640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 0x7fdc74199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc7259c640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d950 tx=0x7fdc6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc74102a60 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7419f630 con 0x7fdc74102a60 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c009c30 con 0x7fdc74102a60 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00f620 con 0x7fdc74102a60 2024-09-21T03:15:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.406+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7406ae60 con 0x7fdc74102a60 2024-09-21T03:15:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.407+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc74103ce0 con 0x7fdc74102a60 2024-09-21T03:15:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.409+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fdc6c005030 con 0x7fdc74102a60 2024-09-21T03:15:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.409+0000 7fdc637fe640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 0x7fdc5c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.409+0000 7fdc71d9b640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 0x7fdc5c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.409+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdc6c0e7190 con 0x7fdc74102a60 2024-09-21T03:15:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.410+0000 7fdc71d9b640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 0x7fdc5c0b8b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdc68006010 tx=0x7fdc68000f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:40.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.412+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc6c0ad820 con 0x7fdc74102a60 2024-09-21T03:15:40.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:40 smithi083 ceph-mon[31219]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:15:40.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:40 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/214004011' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:15:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:40 smithi193 ceph-mon[35224]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:15:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:40 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/214004011' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.598+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdc7406b880 con 0x7fdc74102a60 2024-09-21T03:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.600+0000 7fdc637fe640 1 -- 172.21.15.83:0/1570390662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdc6c0b1c00 con 0x7fdc74102a60 2024-09-21T03:15:40.601 INFO:teuthology.orchestra.run.smithi083.stdout:133143986238 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 msgr2=0x7fdc5c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 0x7fdc5c0b8b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdc68006010 tx=0x7fdc68000f40 comp rx=0 tx=0).stop 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 msgr2=0x7fdc74199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d950 tx=0x7fdc6c00de10 comp rx=0 tx=0).stop 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 shutdown_connections 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdc5c0b6620 0x7fdc5c0b8b30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc74108cf0 0x7fdc7419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc741083a0 0x7fdc74199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 --2- 172.21.15.83:0/1570390662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74102a60 0x7fdc74199490 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 >> 172.21.15.83:0/1570390662 conn(0x7fdc74079730 msgr2=0x7fdc741044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 shutdown_connections 2024-09-21T03:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.603+0000 7fdc78dd7640 1 -- 172.21.15.83:0/1570390662 wait complete. 2024-09-21T03:15:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:40 smithi142 ceph-mon[34512]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T03:15:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:40 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/214004011' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:15:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 -- 172.21.15.83:0/251359118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 msgr2=0x7f33f81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/251359118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f81087b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009900 tx=0x7f33e002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 -- 172.21.15.83:0/251359118 shutdown_connections 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/251359118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8108cf0 0x7f33f810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/251359118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f81087b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/251359118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8102a60 0x7f33f8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 -- 172.21.15.83:0/251359118 >> 172.21.15.83:0/251359118 conn(0x7f33f8079730 msgr2=0x7f33f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 -- 172.21.15.83:0/251359118 shutdown_connections 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.696+0000 7f33fdaf5640 1 -- 172.21.15.83:0/251359118 wait complete. 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.697+0000 7f33fdaf5640 1 Processor -- start 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 -- start start 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8102a60 0x7f33f819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8108cf0 0x7f33f819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8113a80 con 0x7f33f81083a0 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8113900 con 0x7f33f8102a60 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33fdaf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8113c00 con 0x7f33f8108cf0 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47256/0 (socket says 172.21.15.83:47256) 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 -- 172.21.15.83:0/1213377149 learned_addr learned my addr 172.21.15.83:0/1213377149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f7fff640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8108cf0 0x7f33f819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f77fe640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8102a60 0x7f33f819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 -- 172.21.15.83:0/1213377149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8108cf0 msgr2=0x7f33f819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8108cf0 0x7f33f819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 -- 172.21.15.83:0/1213377149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8102a60 msgr2=0x7f33f819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8102a60 0x7f33f819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.698+0000 7f33f6ffd640 1 -- 172.21.15.83:0/1213377149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f81a0420 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33f6ffd640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33e0006010 tx=0x7f33e0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e003f030 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e0003aa0 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0035070 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f81a06b0 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.699+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f810fe90 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.700+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f8103ce0 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.703+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f33e0042030 con 0x7f33f81083a0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.703+0000 7f33f4ff9640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 0x7f33e40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.703+0000 7f33f77fe640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 0x7f33e40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.703+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f33e0107dc0 con 0x7f33f81083a0 2024-09-21T03:15:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.704+0000 7f33f77fe640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 0x7f33e40b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005fd0 tx=0x7f33e8005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.707+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33e00ce450 con 0x7f33f81083a0 2024-09-21T03:15:40.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.900+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f33f810aae0 con 0x7f33f81083a0 2024-09-21T03:15:40.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.900+0000 7f33f4ff9640 1 -- 172.21.15.83:0/1213377149 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f33e00d2830 con 0x7f33f81083a0 2024-09-21T03:15:40.902 INFO:teuthology.orchestra.run.smithi083.stdout:253403070500 2024-09-21T03:15:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 msgr2=0x7f33e40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 0x7f33e40b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005fd0 tx=0x7f33e8005850 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 msgr2=0x7f33f819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33e0006010 tx=0x7f33e0004060 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 shutdown_connections 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e40b6670 0x7f33e40b8b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f8108cf0 0x7f33f819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f81083a0 0x7f33f819aa50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 --2- 172.21.15.83:0/1213377149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f8102a60 0x7f33f819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 >> 172.21.15.83:0/1213377149 conn(0x7f33f8079730 msgr2=0x7f33f8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 shutdown_connections 2024-09-21T03:15:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.902+0000 7f33fdaf5640 1 -- 172.21.15.83:0/1213377149 wait complete. 2024-09-21T03:15:41.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986236 got 133143986238 for osd.4 2024-09-21T03:15:41.427 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:41.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1570390662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:15:41.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2914126626' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:15:41.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1213377149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 -- 172.21.15.83:0/466399489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c410de30 msgr2=0x7f74c4110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/466399489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c410de30 0x7f74c4110320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009ca0 tx=0x7f74bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 -- 172.21.15.83:0/466399489 shutdown_connections 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/466399489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c410de30 0x7f74c4110320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/466399489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c4105320 0x7f74c410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/466399489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 -- 172.21.15.83:0/466399489 >> 172.21.15.83:0/466399489 conn(0x7f74c4079740 msgr2=0x7f74c4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 -- 172.21.15.83:0/466399489 shutdown_connections 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.590+0000 7f74ca9e6640 1 -- 172.21.15.83:0/466399489 wait complete. 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 Processor -- start 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 -- start start 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 0x7f74c4103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113a90 con 0x7f74c4105320 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113910 con 0x7f74c41049d0 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74ca9e6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4113c10 con 0x7f74c410de30 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:41.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47236/0 (socket says 172.21.15.83:47236) 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 -- 172.21.15.83:0/2914126626 learned_addr learned my addr 172.21.15.83:0/2914126626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c3fff640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c8f83640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 0x7f74c4103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 -- 172.21.15.83:0/2914126626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 msgr2=0x7f74c4103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 0x7f74c4103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 -- 172.21.15.83:0/2914126626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 msgr2=0x7f74c4102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.591+0000 7f74c37fe640 1 -- 172.21.15.83:0/2914126626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c41a87a0 con 0x7f74c4105320 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c3fff640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c8f83640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 0x7f74c4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c37fe640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e9f0 tx=0x7f74ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00ccc0 con 0x7f74c4105320 2024-09-21T03:15:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c41a8a90 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac0108e0 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac018860 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.592+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c41a9020 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.593+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c4104de0 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.595+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f74ac010430 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.596+0000 7f74c17fa640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 0x7f74b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.596+0000 7f74c3fff640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 0x7f74b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.596+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f74ac0e7dc0 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.597+0000 7f74c3fff640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 0x7f74b00b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74b4005d00 tx=0x7f74b4005ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.600+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74ac0ae450 con 0x7f74c4105320 2024-09-21T03:15:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.799+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f74c4103a10 con 0x7f74c4105320 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stdout:201863462957 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.800+0000 7f74c17fa640 1 -- 172.21.15.83:0/2914126626 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74c4103a10 con 0x7f74c4105320 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 msgr2=0x7f74b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 0x7f74b00b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74b4005d00 tx=0x7f74b4005ed0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 msgr2=0x7f74c4102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e9f0 tx=0x7f74ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 shutdown_connections 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74b00b65f0 0x7f74b00b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c410de30 0x7f74c4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74c4105320 0x7f74c4102c50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 --2- 172.21.15.83:0/2914126626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74c41049d0 0x7f74c4102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 >> 172.21.15.83:0/2914126626 conn(0x7f74c4079740 msgr2=0x7f74c41079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 shutdown_connections 2024-09-21T03:15:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:40.802+0000 7f74ca9e6640 1 -- 172.21.15.83:0/2914126626 wait complete. 2024-09-21T03:15:41.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1570390662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:15:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2914126626' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:15:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1213377149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:15:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1570390662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:15:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2914126626' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:15:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1213377149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:15:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1332950453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8108cf0 msgr2=0x7f5ab810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1332950453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8108cf0 0x7f5ab810f520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009900 tx=0x7f5ab002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1332950453 shutdown_connections 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1332950453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8108cf0 0x7f5ab810f520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1332950453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81083a0 0x7f5ab81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1332950453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab8102a60 0x7f5ab8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1332950453 >> 172.21.15.83:0/1332950453 conn(0x7f5ab8079730 msgr2=0x7f5ab8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1332950453 shutdown_connections 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.183+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1332950453 wait complete. 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.184+0000 7f5abdbe8640 1 Processor -- start 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.184+0000 7f5abdbe8640 1 -- start start 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab81083a0 0x7f5ab819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab8108cf0 0x7f5ab819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab806bc60 con 0x7f5ab8102a60 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab806bae0 con 0x7f5ab81083a0 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5abdbe8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81a0330 con 0x7f5ab8108cf0 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47266/0 (socket says 172.21.15.83:47266) 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 -- 172.21.15.83:0/1473627569 learned_addr learned my addr 172.21.15.83:0/1473627569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab6ffd640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab81083a0 0x7f5ab819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab7fff640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab8108cf0 0x7f5ab819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 -- 172.21.15.83:0/1473627569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab8108cf0 msgr2=0x7f5ab819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab8108cf0 0x7f5ab819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 -- 172.21.15.83:0/1473627569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab81083a0 msgr2=0x7f5ab819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab81083a0 0x7f5ab819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 -- 172.21.15.83:0/1473627569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab81a0530 con 0x7f5ab8102a60 2024-09-21T03:15:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.185+0000 7f5ab77fe640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c920 tx=0x7f5aa800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.186+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8007c70 con 0x7f5ab8102a60 2024-09-21T03:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.186+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81a0820 con 0x7f5ab8102a60 2024-09-21T03:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.186+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa8007e10 con 0x7f5ab8102a60 2024-09-21T03:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.186+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8018820 con 0x7f5ab8102a60 2024-09-21T03:15:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.186+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a7090 con 0x7f5ab8102a60 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.187+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab8103ce0 con 0x7f5ab8102a60 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.188+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5aa800f450 con 0x7f5ab8102a60 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.189+0000 7f5ab4ff9640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 0x7f5aa00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.189+0000 7f5ab6ffd640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 0x7f5aa00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.189+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f5aa80e7810 con 0x7f5ab8102a60 2024-09-21T03:15:42.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.189+0000 7f5ab6ffd640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 0x7f5aa00b8bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8079d30 tx=0x7f5aac00d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:42.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.192+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5aa8014030 con 0x7f5ab8102a60 2024-09-21T03:15:42.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:42 smithi083 ceph-mon[31219]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:42.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.380+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5ab810aae0 con 0x7f5ab8102a60 2024-09-21T03:15:42.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.381+0000 7f5ab4ff9640 1 -- 172.21.15.83:0/1473627569 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5aa80adfb0 con 0x7f5ab8102a60 2024-09-21T03:15:42.382 INFO:teuthology.orchestra.run.smithi083.stdout:81604378696 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 msgr2=0x7f5aa00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 0x7f5aa00b8bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8079d30 tx=0x7f5aac00d040 comp rx=0 tx=0).stop 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 msgr2=0x7f5ab819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c920 tx=0x7f5aa800cde0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 shutdown_connections 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5aa00b66a0 0x7f5aa00b8bb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab8108cf0 0x7f5ab819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab81083a0 0x7f5ab819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 --2- 172.21.15.83:0/1473627569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ab8102a60 0x7f5ab819a500 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 >> 172.21.15.83:0/1473627569 conn(0x7f5ab8079730 msgr2=0x7f5ab81052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 shutdown_connections 2024-09-21T03:15:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:42.383+0000 7f5abdbe8640 1 -- 172.21.15.83:0/1473627569 wait complete. 2024-09-21T03:15:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:42 smithi193 ceph-mon[35224]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:42.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462956 got 201863462957 for osd.7 2024-09-21T03:15:42.643 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:42 smithi142 ceph-mon[34512]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 -- 172.21.15.83:0/3123036150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61981083a0 msgr2=0x7f61981087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 --2- 172.21.15.83:0/3123036150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61981083a0 0x7f61981087b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f618c009ca0 tx=0x7f618c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 -- 172.21.15.83:0/3123036150 shutdown_connections 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 --2- 172.21.15.83:0/3123036150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6198108cf0 0x7f619810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 --2- 172.21.15.83:0/3123036150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61981083a0 0x7f61981087b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 --2- 172.21.15.83:0/3123036150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198102a60 0x7f6198102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 -- 172.21.15.83:0/3123036150 >> 172.21.15.83:0/3123036150 conn(0x7f6198079730 msgr2=0x7f6198079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 -- 172.21.15.83:0/3123036150 shutdown_connections 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.240+0000 7f619de76640 1 -- 172.21.15.83:0/3123036150 wait complete. 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.241+0000 7f619de76640 1 Processor -- start 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.241+0000 7f619de76640 1 -- start start 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.241+0000 7f619de76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61981083a0 0x7f619819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198108cf0 0x7f619819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6198113b20 con 0x7f6198102a60 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61981139a0 con 0x7f61981083a0 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6198113ca0 con 0x7f6198108cf0 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47276/0 (socket says 172.21.15.83:47276) 2024-09-21T03:15:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 -- 172.21.15.83:0/1950564663 learned_addr learned my addr 172.21.15.83:0/1950564663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f6197fff640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198108cf0 0x7f619819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 -- 172.21.15.83:0/1950564663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198108cf0 msgr2=0x7f619819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198108cf0 0x7f619819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 -- 172.21.15.83:0/1950564663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61981083a0 msgr2=0x7f619819ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61981083a0 0x7f619819ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 -- 172.21.15.83:0/1950564663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61981a0560 con 0x7f6198102a60 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f61977fe640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f618800b4a0 tx=0x7f618800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188014070 con 0x7f6198102a60 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188010ac0 con 0x7f6198102a60 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61981a0850 con 0x7f6198102a60 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188017c30 con 0x7f6198102a60 2024-09-21T03:15:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.242+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61981a6fb0 con 0x7f6198102a60 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.243+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6198102ef0 con 0x7f6198102a60 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.245+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6188017070 con 0x7f6198102a60 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.245+0000 7f6194ff9640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 0x7f61800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.246+0000 7f6196ffd640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 0x7f61800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.246+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f61880e6c50 con 0x7f6198102a60 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.246+0000 7f6196ffd640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 0x7f61800b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f618c0094e0 tx=0x7f618c005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.248+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61880ad2e0 con 0x7f6198102a60 2024-09-21T03:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.438+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f619819b720 con 0x7f6198102a60 2024-09-21T03:15:43.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1473627569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.438+0000 7f6194ff9640 1 -- 172.21.15.83:0/1950564663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f619819b720 con 0x7f6198102a60 2024-09-21T03:15:43.440 INFO:teuthology.orchestra.run.smithi083.stdout:154618822713 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 msgr2=0x7f61800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 0x7f61800b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f618c0094e0 tx=0x7f618c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 msgr2=0x7f619819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f618800b4a0 tx=0x7f618800b960 comp rx=0 tx=0).stop 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 shutdown_connections 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61800b65f0 0x7f61800b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6198108cf0 0x7f619819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61981083a0 0x7f619819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 --2- 172.21.15.83:0/1950564663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6198102a60 0x7f619819a5f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.440+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 >> 172.21.15.83:0/1950564663 conn(0x7f6198079730 msgr2=0x7f6198105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.441+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 shutdown_connections 2024-09-21T03:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:43.441+0000 7f619de76640 1 -- 172.21.15.83:0/1950564663 wait complete. 2024-09-21T03:15:43.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1473627569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:15:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1473627569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:15:44.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822713 for osd.5 2024-09-21T03:15:44.218 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:44.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:44 smithi083 ceph-mon[31219]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:44.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:44 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1950564663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:15:44.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:44 smithi193 ceph-mon[35224]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:44 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1950564663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:15:44.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:44 smithi142 ceph-mon[34512]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:44.579 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:44 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1950564663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:15:44.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378693 got 81604378696 for osd.2 2024-09-21T03:15:44.818 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 -- 172.21.15.83:0/4009792073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c1083d0 msgr2=0x7fd87c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 --2- 172.21.15.83:0/4009792073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c1083d0 0x7fd87c1087e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 -- 172.21.15.83:0/4009792073 shutdown_connections 2024-09-21T03:15:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 --2- 172.21.15.83:0/4009792073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd87c108d20 0x7fd87c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 --2- 172.21.15.83:0/4009792073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c1083d0 0x7fd87c1087e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 --2- 172.21.15.83:0/4009792073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd87c102a90 0x7fd87c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 -- 172.21.15.83:0/4009792073 >> 172.21.15.83:0/4009792073 conn(0x7fd87c079750 msgr2=0x7fd87c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 -- 172.21.15.83:0/4009792073 shutdown_connections 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.007+0000 7fd88271e640 1 -- 172.21.15.83:0/4009792073 wait complete. 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 Processor -- start 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 -- start start 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd87c102a90 0x7fd87c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd87c1083d0 0x7fd87c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c113ab0 con 0x7fd87c108d20 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c113930 con 0x7fd87c102a90 2024-09-21T03:15:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd88271e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c113c30 con 0x7fd87c1083d0 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd880cbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd880cbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47294/0 (socket says 172.21.15.83:47294) 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.008+0000 7fd880cbb640 1 -- 172.21.15.83:0/725670416 learned_addr learned my addr 172.21.15.83:0/725670416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 -- 172.21.15.83:0/725670416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd87c1083d0 msgr2=0x7fd87c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd87c1083d0 0x7fd87c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 -- 172.21.15.83:0/725670416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd87c102a90 msgr2=0x7fd87c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd87c102a90 0x7fd87c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 -- 172.21.15.83:0/725670416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1a03b0 con 0x7fd87c108d20 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd880cbb640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd87400cce0 tx=0x7fd87400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87400f450 con 0x7fd87c108d20 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd87400faa0 con 0x7fd87c108d20 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874018b70 con 0x7fd87c108d20 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a06a0 con 0x7fd87c108d20 2024-09-21T03:15:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.009+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1a6ec0 con 0x7fd87c108d20 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.010+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c103d10 con 0x7fd87c108d20 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.012+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd874018d10 con 0x7fd87c108d20 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.012+0000 7fd8797fa640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 0x7fd8640b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.012+0000 7fd87bfff640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 0x7fd8640b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.012+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd8740e7a10 con 0x7fd87c108d20 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.013+0000 7fd87bfff640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 0x7fd8640b8bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00a8f0 tx=0x7fd86c005bf0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.016+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd874014030 con 0x7fd87c108d20 2024-09-21T03:15:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.203+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd87c19b630 con 0x7fd87c108d20 2024-09-21T03:15:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.204+0000 7fd8797fa640 1 -- 172.21.15.83:0/725670416 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd8740a9090 con 0x7fd87c108d20 2024-09-21T03:15:45.206 INFO:teuthology.orchestra.run.smithi083.stdout:270582939680 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 msgr2=0x7fd8640b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 0x7fd8640b8bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00a8f0 tx=0x7fd86c005bf0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 msgr2=0x7fd87c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd87400cce0 tx=0x7fd87400a620 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 shutdown_connections 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd8640b66a0 0x7fd8640b8bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd87c108d20 0x7fd87c19fd40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd87c1083d0 0x7fd87c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 --2- 172.21.15.83:0/725670416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd87c102a90 0x7fd87c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 >> 172.21.15.83:0/725670416 conn(0x7fd87c079750 msgr2=0x7fd87c1044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:45.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 shutdown_connections 2024-09-21T03:15:45.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.206+0000 7fd88271e640 1 -- 172.21.15.83:0/725670416 wait complete. 2024-09-21T03:15:45.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:45 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/725670416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:15:45.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070497 got 253403070500 for osd.9 2024-09-21T03:15:45.507 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:45.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:45 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/725670416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:15:45.596 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:45 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/725670416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:15:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.964+0000 7f55c0709640 1 -- 172.21.15.83:0/1429688860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b8109e10 msgr2=0x7f55b8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.964+0000 7f55c0709640 1 --2- 172.21.15.83:0/1429688860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b8109e10 0x7f55b8110640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009ca0 tx=0x7f55b002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 -- 172.21.15.83:0/1429688860 shutdown_connections 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 --2- 172.21.15.83:0/1429688860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b8109e10 0x7f55b8110640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 --2- 172.21.15.83:0/1429688860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b81094c0 0x7f55b81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 --2- 172.21.15.83:0/1429688860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 0x7f55b8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 -- 172.21.15.83:0/1429688860 >> 172.21.15.83:0/1429688860 conn(0x7f55b80ff5d0 msgr2=0x7f55b8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 -- 172.21.15.83:0/1429688860 shutdown_connections 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 -- 172.21.15.83:0/1429688860 wait complete. 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 Processor -- start 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 -- start start 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.965+0000 7f55c0709640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 0x7f55b819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b8109e10 0x7f55b819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8113ad0 con 0x7f55b81094c0 2024-09-21T03:15:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8113950 con 0x7f55b8109e10 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8113c50 con 0x7f55b8103b80 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47308/0 (socket says 172.21.15.83:47308) 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 -- 172.21.15.83:0/306867963 learned_addr learned my addr 172.21.15.83:0/306867963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55be4a5640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 0x7f55b819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55beca6640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b8109e10 0x7f55b819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 -- 172.21.15.83:0/306867963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 msgr2=0x7f55b819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 0x7f55b819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 -- 172.21.15.83:0/306867963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b8109e10 msgr2=0x7f55b819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b8109e10 0x7f55b819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 -- 172.21.15.83:0/306867963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a03b0 con 0x7f55b81094c0 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55bdca4640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d740 tx=0x7f55a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8017070 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a06a0 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a800de50 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.966+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a800f620 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.967+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a6e00 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.967+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8103f90 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.969+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f55a801a070 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.970+0000 7f55af7fe640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 0x7f55a40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.970+0000 7f55be4a5640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 0x7f55a40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.970+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f55a80e6ec0 con 0x7f55b81094c0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.970+0000 7f55be4a5640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 0x7f55a40b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:45.973+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55a80b1930 con 0x7f55b81094c0 2024-09-21T03:15:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.172+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f55b819b570 con 0x7f55b81094c0 2024-09-21T03:15:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.172+0000 7f55af7fe640 1 -- 172.21.15.83:0/306867963 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55b819b570 con 0x7f55b81094c0 2024-09-21T03:15:46.174 INFO:teuthology.orchestra.run.smithi083.stdout:111669149764 2024-09-21T03:15:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 msgr2=0x7f55a40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 0x7f55a40b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b400b390 comp rx=0 tx=0).stop 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 msgr2=0x7f55b819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d740 tx=0x7f55a800dc00 comp rx=0 tx=0).stop 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 shutdown_connections 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f55a40b65f0 0x7f55a40b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b8109e10 0x7f55b819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55b81094c0 0x7f55b819a9e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 --2- 172.21.15.83:0/306867963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55b8103b80 0x7f55b819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.174+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 >> 172.21.15.83:0/306867963 conn(0x7f55b80ff5d0 msgr2=0x7f55b81019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.175+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 shutdown_connections 2024-09-21T03:15:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:46.175+0000 7f55c0709640 1 -- 172.21.15.83:0/306867963 wait complete. 2024-09-21T03:15:46.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:46 smithi193 ceph-mon[35224]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T03:15:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:46 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/306867963' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:15:46.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:46 smithi083 ceph-mon[31219]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T03:15:46.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:46 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/306867963' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:15:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:46 smithi142 ceph-mon[34512]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T03:15:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:46 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/306867963' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:15:46.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149760 got 111669149764 for osd.3 2024-09-21T03:15:46.905 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 -- 172.21.15.83:0/2094047607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 msgr2=0x7fc048102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/2094047607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc048102b10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc038009900 tx=0x7fc03802a480 comp rx=0 tx=0).stop 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 -- 172.21.15.83:0/2094047607 shutdown_connections 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/2094047607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 0x7fc0481126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/2094047607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 0x7fc04810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/2094047607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc048102b10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 -- 172.21.15.83:0/2094047607 >> 172.21.15.83:0/2094047607 conn(0x7fc0480796d0 msgr2=0x7fc048079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 -- 172.21.15.83:0/2094047607 shutdown_connections 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.227+0000 7fc04ddb0640 1 -- 172.21.15.83:0/2094047607 wait complete. 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.228+0000 7fc04ddb0640 1 Processor -- start 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.228+0000 7fc04ddb0640 1 -- start start 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 0x7fc04819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 0x7fc0481a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115d20 con 0x7fc048102700 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115ba0 con 0x7fc048103050 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115ea0 con 0x7fc0481102b0 2024-09-21T03:15:47.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47328/0 (socket says 172.21.15.83:47328) 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 -- 172.21.15.83:0/1099849540 learned_addr learned my addr 172.21.15.83:0/1099849540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc046ffd640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 0x7fc04819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc047fff640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 0x7fc0481a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 -- 172.21.15.83:0/1099849540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 msgr2=0x7fc0481a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 0x7fc0481a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 -- 172.21.15.83:0/1099849540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 msgr2=0x7fc04819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 0x7fc04819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 -- 172.21.15.83:0/1099849540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0481a26a0 con 0x7fc048102700 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc0477fe640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc038005940 tx=0x7fc0380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03803f030 con 0x7fc048102700 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc038003cb0 con 0x7fc048102700 2024-09-21T03:15:47.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0481a2930 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038035070 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.229+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0481a90f0 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.230+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc048102b10 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.232+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc038042030 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.233+0000 7fc044ff9640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 0x7fc0300b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.233+0000 7fc046ffd640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 0x7fc0300b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.233+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc038107f80 con 0x7fc048102700 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.233+0000 7fc046ffd640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 0x7fc0300b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0481120c0 tx=0x7fc03c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.236+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0380d29f0 con 0x7fc048102700 2024-09-21T03:15:47.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.417+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc0481a9320 con 0x7fc048102700 2024-09-21T03:15:47.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.417+0000 7fc044ff9640 1 -- 172.21.15.83:0/1099849540 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc0380d7930 con 0x7fc048102700 2024-09-21T03:15:47.419 INFO:teuthology.orchestra.run.smithi083.stdout:227633266730 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 msgr2=0x7fc0300b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 0x7fc0300b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0481120c0 tx=0x7fc03c00b410 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 msgr2=0x7fc04819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc038005940 tx=0x7fc0380042f0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 shutdown_connections 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0300b6670 0x7fc0300b8b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0481102b0 0x7fc0481a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc048103050 0x7fc04819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 --2- 172.21.15.83:0/1099849540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc048102700 0x7fc04819c730 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 >> 172.21.15.83:0/1099849540 conn(0x7fc0480796d0 msgr2=0x7fc048108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:47.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 shutdown_connections 2024-09-21T03:15:47.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:47.420+0000 7fc04ddb0640 1 -- 172.21.15.83:0/1099849540 wait complete. 2024-09-21T03:15:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:47.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939676 got 270582939680 for osd.10 2024-09-21T03:15:47.594 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:48.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266730 for osd.8 2024-09-21T03:15:48.244 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:48.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:48 smithi083 ceph-mon[31219]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:15:48.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:48 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1099849540' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:15:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:48 smithi193 ceph-mon[35224]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:15:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:48 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1099849540' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:15:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:48 smithi142 ceph-mon[34512]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-21T03:15:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:48 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1099849540' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2456826097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d01081c0 msgr2=0x7fc3d01085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2456826097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d01081c0 0x7fc3d01085d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009900 tx=0x7fc3c002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2456826097 shutdown_connections 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2456826097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d0108b10 0x7fc3d010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2456826097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d01081c0 0x7fc3d01085d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2456826097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 0x7fc3d0102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2456826097 >> 172.21.15.83:0/2456826097 conn(0x7fc3d00fe2d0 msgr2=0x7fc3d0100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2456826097 shutdown_connections 2024-09-21T03:15:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.085+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2456826097 wait complete. 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 Processor -- start 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 -- start start 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 0x7fc3d019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 0x7fc3d019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 0x7fc3d019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01138c0 con 0x7fc3d0108b10 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0113740 con 0x7fc3d01081c0 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.086+0000 7fc3cf59e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0113a40 con 0x7fc3d0102880 2024-09-21T03:15:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3cdd9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 0x7fc3d019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 0x7fc3d019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3cdd9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 0x7fc3d019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:39264/0 (socket says 172.21.15.83:39264) 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3cdd9b640 1 -- 172.21.15.83:0/2816668990 learned_addr learned my addr 172.21.15.83:0/2816668990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ce59c640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 0x7fc3d019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 -- 172.21.15.83:0/2816668990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 msgr2=0x7fc3d019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 0x7fc3d019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 -- 172.21.15.83:0/2816668990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 msgr2=0x7fc3d019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 0x7fc3d019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 -- 172.21.15.83:0/2816668990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d01a0220 con 0x7fc3d0108b10 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3ced9d640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 0x7fc3d019faf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e940 tx=0x7fc3c800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8004d20 con 0x7fc3d0108b10 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d01a0510 con 0x7fc3d0108b10 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c8005020 con 0x7fc3d0108b10 2024-09-21T03:15:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80116d0 con 0x7fc3d0108b10 2024-09-21T03:15:49.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.087+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01a6c70 con 0x7fc3d0108b10 2024-09-21T03:15:49.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.088+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d0102c90 con 0x7fc3d0108b10 2024-09-21T03:15:49.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.090+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc3c8005460 con 0x7fc3d0108b10 2024-09-21T03:15:49.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.090+0000 7fc3bf7fe640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 0x7fc3b80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.091+0000 7fc3ce59c640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 0x7fc3b80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.091+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc3c80e7a60 con 0x7fc3d0108b10 2024-09-21T03:15:49.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.091+0000 7fc3ce59c640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 0x7fc3b80b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4005fd0 tx=0x7fc3c400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.094+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3c8016030 con 0x7fc3d0108b10 2024-09-21T03:15:49.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.277+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc3d019b3e0 con 0x7fc3d0108b10 2024-09-21T03:15:49.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.278+0000 7fc3bf7fe640 1 -- 172.21.15.83:0/2816668990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3d019b3e0 con 0x7fc3d0108b10 2024-09-21T03:15:49.280 INFO:teuthology.orchestra.run.smithi083.stdout:38654705747 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 msgr2=0x7fc3b80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 0x7fc3b80b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4005fd0 tx=0x7fc3c400b390 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 msgr2=0x7fc3d019faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 0x7fc3d019faf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e940 tx=0x7fc3c800ee00 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 shutdown_connections 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc3b80b65f0 0x7fc3b80b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d0108b10 0x7fc3d019faf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3d01081c0 0x7fc3d019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 --2- 172.21.15.83:0/2816668990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d0102880 0x7fc3d019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 >> 172.21.15.83:0/2816668990 conn(0x7fc3d00fe2d0 msgr2=0x7fc3d01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 shutdown_connections 2024-09-21T03:15:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.280+0000 7fc3cf59e640 1 -- 172.21.15.83:0/2816668990 wait complete. 2024-09-21T03:15:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 -- 172.21.15.83:0/200031871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814102a80 msgr2=0x7f8814102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 --2- 172.21.15.83:0/200031871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814102a80 0x7f8814102e90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8804009900 tx=0x7f880402a410 comp rx=0 tx=0).stop 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 -- 172.21.15.83:0/200031871 shutdown_connections 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 --2- 172.21.15.83:0/200031871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814108d10 0x7f881410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 --2- 172.21.15.83:0/200031871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 0x7f88141087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 --2- 172.21.15.83:0/200031871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814102a80 0x7f8814102e90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 -- 172.21.15.83:0/200031871 >> 172.21.15.83:0/200031871 conn(0x7f8814079740 msgr2=0x7f8814079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 -- 172.21.15.83:0/200031871 shutdown_connections 2024-09-21T03:15:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.316+0000 7f8819545640 1 -- 172.21.15.83:0/200031871 wait complete. 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 Processor -- start 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 -- start start 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814102a80 0x7f881419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 0x7f881419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8814112980 con 0x7f8814108d10 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8814112800 con 0x7f88141083c0 2024-09-21T03:15:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f8819545640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8814112b00 con 0x7f8814102a80 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f88137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.317+0000 7f88137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57708/0 (socket says 172.21.15.83:57708) 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 -- 172.21.15.83:0/1723566924 learned_addr learned my addr 172.21.15.83:0/1723566924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8812ffd640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814102a80 0x7f881419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f880bfff640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 0x7f881419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 -- 172.21.15.83:0/1723566924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814102a80 msgr2=0x7f881419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814102a80 0x7f881419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 -- 172.21.15.83:0/1723566924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 msgr2=0x7f881419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 0x7f881419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 -- 172.21.15.83:0/1723566924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141a6540 con 0x7f8814108d10 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f88137fe640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f880c00eaa0 tx=0x7f880c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c00cdb0 con 0x7f8814108d10 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880c005020 con 0x7f8814108d10 2024-09-21T03:15:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0116d0 con 0x7f8814108d10 2024-09-21T03:15:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141a6830 con 0x7f8814108d10 2024-09-21T03:15:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.318+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141a6cc0 con 0x7f8814108d10 2024-09-21T03:15:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.320+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814102e90 con 0x7f8814108d10 2024-09-21T03:15:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.321+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f880c0051d0 con 0x7f8814108d10 2024-09-21T03:15:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.322+0000 7f8810ff9640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 0x7f88000b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.322+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f880c0e7990 con 0x7f8814108d10 2024-09-21T03:15:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.322+0000 7f8812ffd640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 0x7f88000b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.322+0000 7f8812ffd640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 0x7f88000b8bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8804000c00 tx=0x7f8804031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.325+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f880c016030 con 0x7f8814108d10 2024-09-21T03:15:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.515+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f881406bc30 con 0x7f8814108d10 2024-09-21T03:15:49.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.516+0000 7f8810ff9640 1 -- 172.21.15.83:0/1723566924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f881406bc30 con 0x7f8814108d10 2024-09-21T03:15:49.518 INFO:teuthology.orchestra.run.smithi083.stdout:60129542222 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 msgr2=0x7f88000b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 0x7f88000b8bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8804000c00 tx=0x7f8804031400 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 msgr2=0x7f881419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f880c00eaa0 tx=0x7f880c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 shutdown_connections 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b66a0 0x7f88000b8bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8814108d10 0x7f881419bd30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88141083c0 0x7f881419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 --2- 172.21.15.83:0/1723566924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8814102a80 0x7f881419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 >> 172.21.15.83:0/1723566924 conn(0x7f8814079740 msgr2=0x7f8814101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 shutdown_connections 2024-09-21T03:15:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.519+0000 7f8819545640 1 -- 172.21.15.83:0/1723566924 wait complete. 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.527+0000 7f11b0527640 1 -- 172.21.15.83:0/1734148488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a8105ae0 msgr2=0x7f11a810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.527+0000 7f11b0527640 1 --2- 172.21.15.83:0/1734148488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a8105ae0 0x7f11a810f530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009900 tx=0x7f11a002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 -- 172.21.15.83:0/1734148488 shutdown_connections 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 --2- 172.21.15.83:0/1734148488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a8105ae0 0x7f11a810f530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 --2- 172.21.15.83:0/1734148488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 0x7f11a81055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 --2- 172.21.15.83:0/1734148488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a81031c0 0x7f11a81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 -- 172.21.15.83:0/1734148488 >> 172.21.15.83:0/1734148488 conn(0x7f11a8077c60 msgr2=0x7f11a80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 -- 172.21.15.83:0/1734148488 shutdown_connections 2024-09-21T03:15:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.528+0000 7f11b0527640 1 -- 172.21.15.83:0/1734148488 wait complete. 2024-09-21T03:15:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 Processor -- start 2024-09-21T03:15:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 -- start start 2024-09-21T03:15:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 0x7f11a819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a8105ae0 0x7f11a819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112980 con 0x7f11a81031c0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112800 con 0x7f11a8105ae0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11b0527640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112b00 con 0x7f11a8105190 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57724/0 (socket says 172.21.15.83:57724) 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 -- 172.21.15.83:0/1449959234 learned_addr learned my addr 172.21.15.83:0/1449959234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11adac2640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 0x7f11a819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11aeac4640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a8105ae0 0x7f11a819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 -- 172.21.15.83:0/1449959234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 msgr2=0x7f11a819b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 0x7f11a819b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 -- 172.21.15.83:0/1449959234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a8105ae0 msgr2=0x7f11a819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a8105ae0 0x7f11a819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.529+0000 7f11ae2c3640 1 -- 172.21.15.83:0/1449959234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a81a6480 con 0x7f11a81031c0 2024-09-21T03:15:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f11ae2c3640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11a400ebc0 tx=0x7f11a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4010630 con 0x7f11a81031c0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81a6770 con 0x7f11a81031c0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a4010c80 con 0x7f11a81031c0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4018b60 con 0x7f11a81031c0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.530+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a6cb0 con 0x7f11a81031c0 2024-09-21T03:15:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.531+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a81035d0 con 0x7f11a81031c0 2024-09-21T03:15:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.534+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f11a4018d00 con 0x7f11a81031c0 2024-09-21T03:15:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.535+0000 7f119f7fe640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 0x7f11940b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.535+0000 7f11adac2640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 0x7f11940b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.535+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f11a40e80d0 con 0x7f11a81031c0 2024-09-21T03:15:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.535+0000 7f11adac2640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 0x7f11940b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f11a8109240 tx=0x7f1198005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:49.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.538+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11a40ae780 con 0x7f11a81031c0 2024-09-21T03:15:49.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.724+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f11a8102230 con 0x7f11a81031c0 2024-09-21T03:15:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.725+0000 7f119f7fe640 1 -- 172.21.15.83:0/1449959234 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f11a40b2b60 con 0x7f11a81031c0 2024-09-21T03:15:49.726 INFO:teuthology.orchestra.run.smithi083.stdout:180388626485 2024-09-21T03:15:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 msgr2=0x7f11940b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 0x7f11940b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f11a8109240 tx=0x7f1198005f90 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 msgr2=0x7f11a819b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11a400ebc0 tx=0x7f11a400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 shutdown_connections 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11940b6620 0x7f11940b8b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11a8105ae0 0x7f11a819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11a8105190 0x7f11a819b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 --2- 172.21.15.83:0/1449959234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a81031c0 0x7f11a819b1f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 >> 172.21.15.83:0/1449959234 conn(0x7f11a8077c60 msgr2=0x7f11a81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 shutdown_connections 2024-09-21T03:15:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:49.727+0000 7f11b0527640 1 -- 172.21.15.83:0/1449959234 wait complete. 2024-09-21T03:15:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:50 smithi083 ceph-mon[31219]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2816668990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:15:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1723566924' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:15:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1449959234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:15:50.406 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626482 got 180388626485 for osd.6 2024-09-21T03:15:50.406 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:50 smithi193 ceph-mon[35224]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2816668990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:15:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1723566924' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:15:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1449959234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:15:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:50 smithi142 ceph-mon[34512]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:15:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2816668990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:15:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1723566924' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:15:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1449959234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:15:51.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542219 got 60129542222 for osd.1 2024-09-21T03:15:51.567 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:52.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:52 smithi083 ceph-mon[31219]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:52.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705742 got 38654705747 for osd.0 2024-09-21T03:15:52.398 DEBUG:teuthology.parallel:result is None 2024-09-21T03:15:52.398 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:15:52.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:52.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:52 smithi193 ceph-mon[35224]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:52.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:52 smithi142 ceph-mon[34512]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:52.683 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 -- 172.21.15.83:0/2741030773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 msgr2=0x7f39541098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 --2- 172.21.15.83:0/2741030773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f39541098d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3950009900 tx=0x7f395002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 -- 172.21.15.83:0/2741030773 shutdown_connections 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 --2- 172.21.15.83:0/2741030773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 0x7f3954110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 --2- 172.21.15.83:0/2741030773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f39541098d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 --2- 172.21.15.83:0/2741030773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f3954103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 -- 172.21.15.83:0/2741030773 >> 172.21.15.83:0/2741030773 conn(0x7f39540ff5b0 msgr2=0x7f3954101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 -- 172.21.15.83:0/2741030773 shutdown_connections 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.196+0000 7f395b6c9640 1 -- 172.21.15.83:0/2741030773 wait complete. 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 Processor -- start 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 -- start start 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f395419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 0x7f395419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113af0 con 0x7f39541094c0 2024-09-21T03:15:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113970 con 0x7f3954103b80 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.197+0000 7f395b6c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113c70 con 0x7f3954109e10 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57728/0 (socket says 172.21.15.83:57728) 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 -- 172.21.15.83:0/4011403556 learned_addr learned my addr 172.21.15.83:0/4011403556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3959c66640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 0x7f395419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3959465640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f395419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 -- 172.21.15.83:0/4011403556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 msgr2=0x7f395419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 0x7f395419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 -- 172.21.15.83:0/4011403556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 msgr2=0x7f395419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f395419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 -- 172.21.15.83:0/4011403556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39541a04b0 con 0x7f39541094c0 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3959465640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f395419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f3958c64640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f395002a930 tx=0x7f3950004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395003f030 con 0x7f39541094c0 2024-09-21T03:15:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3950003aa0 con 0x7f39541094c0 2024-09-21T03:15:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39541a0740 con 0x7f39541094c0 2024-09-21T03:15:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950035070 con 0x7f39541094c0 2024-09-21T03:15:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.198+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a6f00 con 0x7f39541094c0 2024-09-21T03:15:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.199+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954103f90 con 0x7f39541094c0 2024-09-21T03:15:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.201+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f3950042030 con 0x7f39541094c0 2024-09-21T03:15:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.202+0000 7f394a7fc640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 0x7f39400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.202+0000 7f3959465640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 0x7f39400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.202+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3950108030 con 0x7f39541094c0 2024-09-21T03:15:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.203+0000 7f3959465640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 0x7f39400b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39440058c0 tx=0x7f3944005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.205+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39500ce6c0 con 0x7f39541094c0 2024-09-21T03:15:53.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.399+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3954063800 con 0x7f39400b65f0 2024-09-21T03:15:53.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.404+0000 7f394a7fc640 1 -- 172.21.15.83:0/4011403556 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259012 (secure 0 0 0) 0x7f3954063800 con 0x7f39400b65f0 2024-09-21T03:15:53.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:53.409 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 msgr2=0x7f39400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 0x7f39400b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39440058c0 tx=0x7f3944005850 comp rx=0 tx=0).stop 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 msgr2=0x7f395419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f395002a930 tx=0x7f3950004270 comp rx=0 tx=0).stop 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 shutdown_connections 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39400b65f0 0x7f39400b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3954109e10 0x7f395419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39541094c0 0x7f395419aae0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 --2- 172.21.15.83:0/4011403556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3954103b80 0x7f395419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:53.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 >> 172.21.15.83:0/4011403556 conn(0x7f39540ff5b0 msgr2=0x7f3954107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:53.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 shutdown_connections 2024-09-21T03:15:53.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:53.407+0000 7f395b6c9640 1 -- 172.21.15.83:0/4011403556 wait complete. 2024-09-21T03:15:54.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-21T03:15:53.188683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":756,"num_read_kb":485,"num_write":368,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351440,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590384,"statfs":{"total":1151940427776,"available":1151580553216,"internally_reserved":0,"allocated":10727424,"data_stored":6967776,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259613+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:00.259613+0000","last_peered":"2024-09-21T03:15:00.259613+0000","last_clean":"2024-09-21T03:15:00.259613+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:00.259613+0000","last_undegraded":"2024-09-21T03:15:00.259613+0000","last_fullsized":"2024-09-21T03:15:00.259613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258368+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:00.258368+0000","last_peered":"2024-09-21T03:15:00.258368+0000","last_clean":"2024-09-21T03:15:00.258368+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:00.258368+0000","last_undegraded":"2024-09-21T03:15:00.258368+0000","last_fullsized":"2024-09-21T03:15:00.258368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260247+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:00.260247+0000","last_peered":"2024-09-21T03:15:00.260247+0000","last_clean":"2024-09-21T03:15:00.260247+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:00.260247+0000","last_undegraded":"2024-09-21T03:15:00.260247+0000","last_fullsized":"2024-09-21T03:15:00.260247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258439+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:00.258439+0000","last_peered":"2024-09-21T03:15:00.258439+0000","last_clean":"2024-09-21T03:15:00.258439+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:00.258439+0000","last_undegraded":"2024-09-21T03:15:00.258439+0000","last_fullsized":"2024-09-21T03:15:00.258439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263181+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:00.263181+0000","last_peered":"2024-09-21T03:15:00.263181+0000","last_clean":"2024-09-21T03:15:00.263181+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:00.263181+0000","last_undegraded":"2024-09-21T03:15:00.263181+0000","last_fullsized":"2024-09-21T03:15:00.263181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258788+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:00.258788+0000","last_peered":"2024-09-21T03:15:00.258788+0000","last_clean":"2024-09-21T03:15:00.258788+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:00.258788+0000","last_undegraded":"2024-09-21T03:15:00.258788+0000","last_fullsized":"2024-09-21T03:15:00.258788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:47.848583+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:15:47.848583+0000","last_peered":"2024-09-21T03:15:47.848583+0000","last_clean":"2024-09-21T03:15:47.848583+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:15:47.848583+0000","last_undegraded":"2024-09-21T03:15:47.848583+0000","last_fullsized":"2024-09-21T03:15:47.848583+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263742+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:00.263742+0000","last_peered":"2024-09-21T03:15:00.263742+0000","last_clean":"2024-09-21T03:15:00.263742+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:00.263742+0000","last_undegraded":"2024-09-21T03:15:00.263742+0000","last_fullsized":"2024-09-21T03:15:00.263742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258447+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:00.258447+0000","last_peered":"2024-09-21T03:15:00.258447+0000","last_clean":"2024-09-21T03:15:00.258447+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:00.258447+0000","last_undegraded":"2024-09-21T03:15:00.258447+0000","last_fullsized":"2024-09-21T03:15:00.258447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.853561+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:51.853561+0000","last_peered":"2024-09-21T03:15:51.853561+0000","last_clean":"2024-09-21T03:15:51.853561+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:51.853561+0000","last_undegraded":"2024-09-21T03:15:51.853561+0000","last_fullsized":"2024-09-21T03:15:51.853561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'30","reported_seq":190,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.853531+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:51.853531+0000","last_peered":"2024-09-21T03:15:51.853531+0000","last_clean":"2024-09-21T03:15:51.853531+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:51.853531+0000","last_undegraded":"2024-09-21T03:15:51.853531+0000","last_fullsized":"2024-09-21T03:15:51.853531+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64757,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":130,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260024+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:00.260024+0000","last_peered":"2024-09-21T03:15:00.260024+0000","last_clean":"2024-09-21T03:15:00.260024+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:00.260024+0000","last_undegraded":"2024-09-21T03:15:00.260024+0000","last_fullsized":"2024-09-21T03:15:00.260024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.854019+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:51.854019+0000","last_peered":"2024-09-21T03:15:51.854019+0000","last_clean":"2024-09-21T03:15:51.854019+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:51.854019+0000","last_undegraded":"2024-09-21T03:15:51.854019+0000","last_fullsized":"2024-09-21T03:15:51.854019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259547+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:00.259547+0000","last_peered":"2024-09-21T03:15:00.259547+0000","last_clean":"2024-09-21T03:15:00.259547+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:00.259547+0000","last_undegraded":"2024-09-21T03:15:00.259547+0000","last_fullsized":"2024-09-21T03:15:00.259547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263368+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:00.263368+0000","last_peered":"2024-09-21T03:15:00.263368+0000","last_clean":"2024-09-21T03:15:00.263368+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:00.263368+0000","last_undegraded":"2024-09-21T03:15:00.263368+0000","last_fullsized":"2024-09-21T03:15:00.263368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260269+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:00.260269+0000","last_peered":"2024-09-21T03:15:00.260269+0000","last_clean":"2024-09-21T03:15:00.260269+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:00.260269+0000","last_undegraded":"2024-09-21T03:15:00.260269+0000","last_fullsized":"2024-09-21T03:15:00.260269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.271029+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:00.271029+0000","last_peered":"2024-09-21T03:15:00.271029+0000","last_clean":"2024-09-21T03:15:00.271029+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:00.271029+0000","last_undegraded":"2024-09-21T03:15:00.271029+0000","last_fullsized":"2024-09-21T03:15:00.271029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:50.852818+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:50.852818+0000","last_peered":"2024-09-21T03:15:50.852818+0000","last_clean":"2024-09-21T03:15:50.852818+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:50.852818+0000","last_undegraded":"2024-09-21T03:15:50.852818+0000","last_fullsized":"2024-09-21T03:15:50.852818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260427+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:00.260427+0000","last_peered":"2024-09-21T03:15:00.260427+0000","last_clean":"2024-09-21T03:15:00.260427+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:00.260427+0000","last_undegraded":"2024-09-21T03:15:00.260427+0000","last_fullsized":"2024-09-21T03:15:00.260427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259765+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259765+0000","last_peered":"2024-09-21T03:15:00.259765+0000","last_clean":"2024-09-21T03:15:00.259765+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259765+0000","last_undegraded":"2024-09-21T03:15:00.259765+0000","last_fullsized":"2024-09-21T03:15:00.259765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258614+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:00.258614+0000","last_peered":"2024-09-21T03:15:00.258614+0000","last_clean":"2024-09-21T03:15:00.258614+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:00.258614+0000","last_undegraded":"2024-09-21T03:15:00.258614+0000","last_fullsized":"2024-09-21T03:15:00.258614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259707+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259707+0000","last_peered":"2024-09-21T03:15:00.259707+0000","last_clean":"2024-09-21T03:15:00.259707+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259707+0000","last_undegraded":"2024-09-21T03:15:00.259707+0000","last_fullsized":"2024-09-21T03:15:00.259707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:49.850481+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:49.850481+0000","last_peered":"2024-09-21T03:15:49.850481+0000","last_clean":"2024-09-21T03:15:49.850481+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:49.850481+0000","last_undegraded":"2024-09-21T03:15:49.850481+0000","last_fullsized":"2024-09-21T03:15:49.850481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264186+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:00.264186+0000","last_peered":"2024-09-21T03:15:00.264186+0000","last_clean":"2024-09-21T03:15:00.264186+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:00.264186+0000","last_undegraded":"2024-09-21T03:15:00.264186+0000","last_fullsized":"2024-09-21T03:15:00.264186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:48.849047+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:15:48.849047+0000","last_peered":"2024-09-21T03:15:48.849047+0000","last_clean":"2024-09-21T03:15:48.849047+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:15:48.849047+0000","last_undegraded":"2024-09-21T03:15:48.849047+0000","last_fullsized":"2024-09-21T03:15:48.849047+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:02.607478+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:02.607478+0000","last_peered":"2024-09-21T03:15:02.607478+0000","last_clean":"2024-09-21T03:15:02.607478+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:02.607478+0000","last_undegraded":"2024-09-21T03:15:02.607478+0000","last_fullsized":"2024-09-21T03:15:02.607478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259126+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:00.259126+0000","last_peered":"2024-09-21T03:15:00.259126+0000","last_clean":"2024-09-21T03:15:00.259126+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:00.259126+0000","last_undegraded":"2024-09-21T03:15:00.259126+0000","last_fullsized":"2024-09-21T03:15:00.259126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258634+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:00.258634+0000","last_peered":"2024-09-21T03:15:00.258634+0000","last_clean":"2024-09-21T03:15:00.258634+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:00.258634+0000","last_undegraded":"2024-09-21T03:15:00.258634+0000","last_fullsized":"2024-09-21T03:15:00.258634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259779+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:00.259779+0000","last_peered":"2024-09-21T03:15:00.259779+0000","last_clean":"2024-09-21T03:15:00.259779+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:00.259779+0000","last_undegraded":"2024-09-21T03:15:00.259779+0000","last_fullsized":"2024-09-21T03:15:00.259779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.267843+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:00.267843+0000","last_peered":"2024-09-21T03:15:00.267843+0000","last_clean":"2024-09-21T03:15:00.267843+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:00.267843+0000","last_undegraded":"2024-09-21T03:15:00.267843+0000","last_fullsized":"2024-09-21T03:15:00.267843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259025+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259025+0000","last_peered":"2024-09-21T03:15:00.259025+0000","last_clean":"2024-09-21T03:15:00.259025+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259025+0000","last_undegraded":"2024-09-21T03:15:00.259025+0000","last_fullsized":"2024-09-21T03:15:00.259025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258780+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:00.258780+0000","last_peered":"2024-09-21T03:15:00.258780+0000","last_clean":"2024-09-21T03:15:00.258780+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:00.258780+0000","last_undegraded":"2024-09-21T03:15:00.258780+0000","last_fullsized":"2024-09-21T03:15:00.258780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:50.851442+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:50.851442+0000","last_peered":"2024-09-21T03:15:50.851442+0000","last_clean":"2024-09-21T03:15:50.851442+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:50.851442+0000","last_undegraded":"2024-09-21T03:15:50.851442+0000","last_fullsized":"2024-09-21T03:15:50.851442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":416,"num_write":243,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743452,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939682,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072083,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":482157,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":481614,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:54.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:15:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.316 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:15:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:15:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:15:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 -- 172.21.15.83:0/1986046445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99141094a0 msgr2=0x7f99141098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 --2- 172.21.15.83:0/1986046445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99141094a0 0x7f99141098b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9904009900 tx=0x7f990402a410 comp rx=0 tx=0).stop 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 -- 172.21.15.83:0/1986046445 shutdown_connections 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 --2- 172.21.15.83:0/1986046445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9914109df0 0x7f9914110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 --2- 172.21.15.83:0/1986046445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99141094a0 0x7f99141098b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 --2- 172.21.15.83:0/1986046445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 0x7f9914103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 -- 172.21.15.83:0/1986046445 >> 172.21.15.83:0/1986046445 conn(0x7f991407b850 msgr2=0x7f991407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 -- 172.21.15.83:0/1986046445 shutdown_connections 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.806+0000 7f991be79640 1 -- 172.21.15.83:0/1986046445 wait complete. 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 Processor -- start 2024-09-21T03:15:54.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 -- start start 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 0x7f9914075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141094a0 0x7f9914075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9914113b50 con 0x7f9914109df0 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99141139d0 con 0x7f99141094a0 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.807+0000 7f991be79640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9914113cd0 con 0x7f9914103b60 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57738/0 (socket says 172.21.15.83:57738) 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 -- 172.21.15.83:0/1506909401 learned_addr learned my addr 172.21.15.83:0/1506909401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f9919c15640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 0x7f9914075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 -- 172.21.15.83:0/1506909401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 msgr2=0x7f9914075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 0x7f9914075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 -- 172.21.15.83:0/1506909401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141094a0 msgr2=0x7f9914075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f9919414640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141094a0 0x7f9914075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141094a0 0x7f9914075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 -- 172.21.15.83:0/1506909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99141a8980 con 0x7f9914109df0 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991a416640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b970 tx=0x7f990c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c019070 con 0x7f9914109df0 2024-09-21T03:15:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141a8c70 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c014440 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0075d0 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.808+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a91b0 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.809+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9914103ff0 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.811+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f990c01c070 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.811+0000 7f990affd640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 0x7f99000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.812+0000 7f9919c15640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 0x7f99000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.812+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f990c0e6f20 con 0x7f9914109df0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.812+0000 7f9919c15640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 0x7f99000b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9914077160 tx=0x7f9910009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.815+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f990c0ad5b0 con 0x7f9914109df0 2024-09-21T03:15:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.991+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9914063800 con 0x7f99000b65f0 2024-09-21T03:15:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.995+0000 7f990affd640 1 -- 172.21.15.83:0/1506909401 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259012 (secure 0 0 0) 0x7f9914063800 con 0x7f99000b65f0 2024-09-21T03:15:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 msgr2=0x7f99000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 0x7f99000b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9914077160 tx=0x7f9910009110 comp rx=0 tx=0).stop 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 msgr2=0x7f9914076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b970 tx=0x7f990c00bd70 comp rx=0 tx=0).stop 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 shutdown_connections 2024-09-21T03:15:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99000b65f0 0x7f99000b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914109df0 0x7f9914076190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99141094a0 0x7f9914075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 --2- 172.21.15.83:0/1506909401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9914103b60 0x7f9914075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 >> 172.21.15.83:0/1506909401 conn(0x7f991407b850 msgr2=0x7f99141076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 shutdown_connections 2024-09-21T03:15:55.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:54.998+0000 7f991be79640 1 -- 172.21.15.83:0/1506909401 wait complete. 2024-09-21T03:15:55.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:55 smithi193 ceph-mon[35224]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:55 smithi083 ceph-mon[31219]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:55 smithi142 ceph-mon[34512]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:55.723 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-21T03:15:53.188683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":756,"num_read_kb":485,"num_write":368,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351440,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590384,"statfs":{"total":1151940427776,"available":1151580553216,"internally_reserved":0,"allocated":10727424,"data_stored":6967776,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262056+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:00.262056+0000","last_peered":"2024-09-21T03:15:00.262056+0000","last_clean":"2024-09-21T03:15:00.262056+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:00.262056+0000","last_undegraded":"2024-09-21T03:15:00.262056+0000","last_fullsized":"2024-09-21T03:15:00.262056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263298+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:00.263298+0000","last_peered":"2024-09-21T03:15:00.263298+0000","last_clean":"2024-09-21T03:15:00.263298+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:00.263298+0000","last_undegraded":"2024-09-21T03:15:00.263298+0000","last_fullsized":"2024-09-21T03:15:00.263298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263300+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:00.263300+0000","last_peered":"2024-09-21T03:15:00.263300+0000","last_clean":"2024-09-21T03:15:00.263300+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:00.263300+0000","last_undegraded":"2024-09-21T03:15:00.263300+0000","last_fullsized":"2024-09-21T03:15:00.263300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263348+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:00.263348+0000","last_peered":"2024-09-21T03:15:00.263348+0000","last_clean":"2024-09-21T03:15:00.263348+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:00.263348+0000","last_undegraded":"2024-09-21T03:15:00.263348+0000","last_fullsized":"2024-09-21T03:15:00.263348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260593+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:00.260593+0000","last_peered":"2024-09-21T03:15:00.260593+0000","last_clean":"2024-09-21T03:15:00.260593+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:00.260593+0000","last_undegraded":"2024-09-21T03:15:00.260593+0000","last_fullsized":"2024-09-21T03:15:00.260593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263504+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:00.263504+0000","last_peered":"2024-09-21T03:15:00.263504+0000","last_clean":"2024-09-21T03:15:00.263504+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:00.263504+0000","last_undegraded":"2024-09-21T03:15:00.263504+0000","last_fullsized":"2024-09-21T03:15:00.263504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259613+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:00.259613+0000","last_peered":"2024-09-21T03:15:00.259613+0000","last_clean":"2024-09-21T03:15:00.259613+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:00.259613+0000","last_undegraded":"2024-09-21T03:15:00.259613+0000","last_fullsized":"2024-09-21T03:15:00.259613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258368+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:00.258368+0000","last_peered":"2024-09-21T03:15:00.258368+0000","last_clean":"2024-09-21T03:15:00.258368+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:00.258368+0000","last_undegraded":"2024-09-21T03:15:00.258368+0000","last_fullsized":"2024-09-21T03:15:00.258368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265818+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:00.265818+0000","last_peered":"2024-09-21T03:15:00.265818+0000","last_clean":"2024-09-21T03:15:00.265818+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:00.265818+0000","last_undegraded":"2024-09-21T03:15:00.265818+0000","last_fullsized":"2024-09-21T03:15:00.265818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263570+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:00.263570+0000","last_peered":"2024-09-21T03:15:00.263570+0000","last_clean":"2024-09-21T03:15:00.263570+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:00.263570+0000","last_undegraded":"2024-09-21T03:15:00.263570+0000","last_fullsized":"2024-09-21T03:15:00.263570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260247+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:00.260247+0000","last_peered":"2024-09-21T03:15:00.260247+0000","last_clean":"2024-09-21T03:15:00.260247+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:00.260247+0000","last_undegraded":"2024-09-21T03:15:00.260247+0000","last_fullsized":"2024-09-21T03:15:00.260247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263417+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:00.263417+0000","last_peered":"2024-09-21T03:15:00.263417+0000","last_clean":"2024-09-21T03:15:00.263417+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:00.263417+0000","last_undegraded":"2024-09-21T03:15:00.263417+0000","last_fullsized":"2024-09-21T03:15:00.263417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265843+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:00.265843+0000","last_peered":"2024-09-21T03:15:00.265843+0000","last_clean":"2024-09-21T03:15:00.265843+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:00.265843+0000","last_undegraded":"2024-09-21T03:15:00.265843+0000","last_fullsized":"2024-09-21T03:15:00.265843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263272+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:00.263272+0000","last_peered":"2024-09-21T03:15:00.263272+0000","last_clean":"2024-09-21T03:15:00.263272+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:00.263272+0000","last_undegraded":"2024-09-21T03:15:00.263272+0000","last_fullsized":"2024-09-21T03:15:00.263272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263162+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:00.263162+0000","last_peered":"2024-09-21T03:15:00.263162+0000","last_clean":"2024-09-21T03:15:00.263162+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:00.263162+0000","last_undegraded":"2024-09-21T03:15:00.263162+0000","last_fullsized":"2024-09-21T03:15:00.263162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264711+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:00.264711+0000","last_peered":"2024-09-21T03:15:00.264711+0000","last_clean":"2024-09-21T03:15:00.264711+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:00.264711+0000","last_undegraded":"2024-09-21T03:15:00.264711+0000","last_fullsized":"2024-09-21T03:15:00.264711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262210+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:00.262210+0000","last_peered":"2024-09-21T03:15:00.262210+0000","last_clean":"2024-09-21T03:15:00.262210+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:00.262210+0000","last_undegraded":"2024-09-21T03:15:00.262210+0000","last_fullsized":"2024-09-21T03:15:00.262210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261697+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:00.261697+0000","last_peered":"2024-09-21T03:15:00.261697+0000","last_clean":"2024-09-21T03:15:00.261697+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:00.261697+0000","last_undegraded":"2024-09-21T03:15:00.261697+0000","last_fullsized":"2024-09-21T03:15:00.261697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265800+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:00.265800+0000","last_peered":"2024-09-21T03:15:00.265800+0000","last_clean":"2024-09-21T03:15:00.265800+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:00.265800+0000","last_undegraded":"2024-09-21T03:15:00.265800+0000","last_fullsized":"2024-09-21T03:15:00.265800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261706+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:00.261706+0000","last_peered":"2024-09-21T03:15:00.261706+0000","last_clean":"2024-09-21T03:15:00.261706+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:00.261706+0000","last_undegraded":"2024-09-21T03:15:00.261706+0000","last_fullsized":"2024-09-21T03:15:00.261706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265852+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:00.265852+0000","last_peered":"2024-09-21T03:15:00.265852+0000","last_clean":"2024-09-21T03:15:00.265852+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:00.265852+0000","last_undegraded":"2024-09-21T03:15:00.265852+0000","last_fullsized":"2024-09-21T03:15:00.265852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264797+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:00.264797+0000","last_peered":"2024-09-21T03:15:00.264797+0000","last_clean":"2024-09-21T03:15:00.264797+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:00.264797+0000","last_undegraded":"2024-09-21T03:15:00.264797+0000","last_fullsized":"2024-09-21T03:15:00.264797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258439+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:00.258439+0000","last_peered":"2024-09-21T03:15:00.258439+0000","last_clean":"2024-09-21T03:15:00.258439+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:00.258439+0000","last_undegraded":"2024-09-21T03:15:00.258439+0000","last_fullsized":"2024-09-21T03:15:00.258439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263181+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:00.263181+0000","last_peered":"2024-09-21T03:15:00.263181+0000","last_clean":"2024-09-21T03:15:00.263181+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:00.263181+0000","last_undegraded":"2024-09-21T03:15:00.263181+0000","last_fullsized":"2024-09-21T03:15:00.263181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258788+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:00.258788+0000","last_peered":"2024-09-21T03:15:00.258788+0000","last_clean":"2024-09-21T03:15:00.258788+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:00.258788+0000","last_undegraded":"2024-09-21T03:15:00.258788+0000","last_fullsized":"2024-09-21T03:15:00.258788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266136+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:00.266136+0000","last_peered":"2024-09-21T03:15:00.266136+0000","last_clean":"2024-09-21T03:15:00.266136+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:00.266136+0000","last_undegraded":"2024-09-21T03:15:00.266136+0000","last_fullsized":"2024-09-21T03:15:00.266136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261638+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:00.261638+0000","last_peered":"2024-09-21T03:15:00.261638+0000","last_clean":"2024-09-21T03:15:00.261638+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:00.261638+0000","last_undegraded":"2024-09-21T03:15:00.261638+0000","last_fullsized":"2024-09-21T03:15:00.261638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265654+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:00.265654+0000","last_peered":"2024-09-21T03:15:00.265654+0000","last_clean":"2024-09-21T03:15:00.265654+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:00.265654+0000","last_undegraded":"2024-09-21T03:15:00.265654+0000","last_fullsized":"2024-09-21T03:15:00.265654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263480+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:00.263480+0000","last_peered":"2024-09-21T03:15:00.263480+0000","last_clean":"2024-09-21T03:15:00.263480+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:00.263480+0000","last_undegraded":"2024-09-21T03:15:00.263480+0000","last_fullsized":"2024-09-21T03:15:00.263480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265788+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:00.265788+0000","last_peered":"2024-09-21T03:15:00.265788+0000","last_clean":"2024-09-21T03:15:00.265788+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:00.265788+0000","last_undegraded":"2024-09-21T03:15:00.265788+0000","last_fullsized":"2024-09-21T03:15:00.265788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.269390+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:00.269390+0000","last_peered":"2024-09-21T03:15:00.269390+0000","last_clean":"2024-09-21T03:15:00.269390+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:00.269390+0000","last_undegraded":"2024-09-21T03:15:00.269390+0000","last_fullsized":"2024-09-21T03:15:00.269390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261580+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:00.261580+0000","last_peered":"2024-09-21T03:15:00.261580+0000","last_clean":"2024-09-21T03:15:00.261580+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:00.261580+0000","last_undegraded":"2024-09-21T03:15:00.261580+0000","last_fullsized":"2024-09-21T03:15:00.261580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261023+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:00.261023+0000","last_peered":"2024-09-21T03:15:00.261023+0000","last_clean":"2024-09-21T03:15:00.261023+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:00.261023+0000","last_undegraded":"2024-09-21T03:15:00.261023+0000","last_fullsized":"2024-09-21T03:15:00.261023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:47.848583+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:15:47.848583+0000","last_peered":"2024-09-21T03:15:47.848583+0000","last_clean":"2024-09-21T03:15:47.848583+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:15:47.848583+0000","last_undegraded":"2024-09-21T03:15:47.848583+0000","last_fullsized":"2024-09-21T03:15:47.848583+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263742+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:00.263742+0000","last_peered":"2024-09-21T03:15:00.263742+0000","last_clean":"2024-09-21T03:15:00.263742+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:00.263742+0000","last_undegraded":"2024-09-21T03:15:00.263742+0000","last_fullsized":"2024-09-21T03:15:00.263742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258447+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:00.258447+0000","last_peered":"2024-09-21T03:15:00.258447+0000","last_clean":"2024-09-21T03:15:00.258447+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:00.258447+0000","last_undegraded":"2024-09-21T03:15:00.258447+0000","last_fullsized":"2024-09-21T03:15:00.258447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261453+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:00.261453+0000","last_peered":"2024-09-21T03:15:00.261453+0000","last_clean":"2024-09-21T03:15:00.261453+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:00.261453+0000","last_undegraded":"2024-09-21T03:15:00.261453+0000","last_fullsized":"2024-09-21T03:15:00.261453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.853561+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:15:51.853561+0000","last_peered":"2024-09-21T03:15:51.853561+0000","last_clean":"2024-09-21T03:15:51.853561+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:15:51.853561+0000","last_undegraded":"2024-09-21T03:15:51.853561+0000","last_fullsized":"2024-09-21T03:15:51.853561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265378+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:00.265378+0000","last_peered":"2024-09-21T03:15:00.265378+0000","last_clean":"2024-09-21T03:15:00.265378+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:00.265378+0000","last_undegraded":"2024-09-21T03:15:00.265378+0000","last_fullsized":"2024-09-21T03:15:00.265378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263573+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:00.263573+0000","last_peered":"2024-09-21T03:15:00.263573+0000","last_clean":"2024-09-21T03:15:00.263573+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:00.263573+0000","last_undegraded":"2024-09-21T03:15:00.263573+0000","last_fullsized":"2024-09-21T03:15:00.263573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265202+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:00.265202+0000","last_peered":"2024-09-21T03:15:00.265202+0000","last_clean":"2024-09-21T03:15:00.265202+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:00.265202+0000","last_undegraded":"2024-09-21T03:15:00.265202+0000","last_fullsized":"2024-09-21T03:15:00.265202+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'30","reported_seq":190,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.853531+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:15:51.853531+0000","last_peered":"2024-09-21T03:15:51.853531+0000","last_clean":"2024-09-21T03:15:51.853531+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:15:51.853531+0000","last_undegraded":"2024-09-21T03:15:51.853531+0000","last_fullsized":"2024-09-21T03:15:51.853531+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64757,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":130,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261370+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:00.261370+0000","last_peered":"2024-09-21T03:15:00.261370+0000","last_clean":"2024-09-21T03:15:00.261370+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:00.261370+0000","last_undegraded":"2024-09-21T03:15:00.261370+0000","last_fullsized":"2024-09-21T03:15:00.261370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265455+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:00.265455+0000","last_peered":"2024-09-21T03:15:00.265455+0000","last_clean":"2024-09-21T03:15:00.265455+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:00.265455+0000","last_undegraded":"2024-09-21T03:15:00.265455+0000","last_fullsized":"2024-09-21T03:15:00.265455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260024+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:00.260024+0000","last_peered":"2024-09-21T03:15:00.260024+0000","last_clean":"2024-09-21T03:15:00.260024+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:00.260024+0000","last_undegraded":"2024-09-21T03:15:00.260024+0000","last_fullsized":"2024-09-21T03:15:00.260024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:51.854019+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:15:51.854019+0000","last_peered":"2024-09-21T03:15:51.854019+0000","last_clean":"2024-09-21T03:15:51.854019+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:15:51.854019+0000","last_undegraded":"2024-09-21T03:15:51.854019+0000","last_fullsized":"2024-09-21T03:15:51.854019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259547+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:00.259547+0000","last_peered":"2024-09-21T03:15:00.259547+0000","last_clean":"2024-09-21T03:15:00.259547+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:00.259547+0000","last_undegraded":"2024-09-21T03:15:00.259547+0000","last_fullsized":"2024-09-21T03:15:00.259547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263368+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:00.263368+0000","last_peered":"2024-09-21T03:15:00.263368+0000","last_clean":"2024-09-21T03:15:00.263368+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:00.263368+0000","last_undegraded":"2024-09-21T03:15:00.263368+0000","last_fullsized":"2024-09-21T03:15:00.263368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262061+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:00.262061+0000","last_peered":"2024-09-21T03:15:00.262061+0000","last_clean":"2024-09-21T03:15:00.262061+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:00.262061+0000","last_undegraded":"2024-09-21T03:15:00.262061+0000","last_fullsized":"2024-09-21T03:15:00.262061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260269+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:00.260269+0000","last_peered":"2024-09-21T03:15:00.260269+0000","last_clean":"2024-09-21T03:15:00.260269+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:00.260269+0000","last_undegraded":"2024-09-21T03:15:00.260269+0000","last_fullsized":"2024-09-21T03:15:00.260269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.271029+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:00.271029+0000","last_peered":"2024-09-21T03:15:00.271029+0000","last_clean":"2024-09-21T03:15:00.271029+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:00.271029+0000","last_undegraded":"2024-09-21T03:15:00.271029+0000","last_fullsized":"2024-09-21T03:15:00.271029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:50.852818+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:15:50.852818+0000","last_peered":"2024-09-21T03:15:50.852818+0000","last_clean":"2024-09-21T03:15:50.852818+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:15:50.852818+0000","last_undegraded":"2024-09-21T03:15:50.852818+0000","last_fullsized":"2024-09-21T03:15:50.852818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260427+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:00.260427+0000","last_peered":"2024-09-21T03:15:00.260427+0000","last_clean":"2024-09-21T03:15:00.260427+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:00.260427+0000","last_undegraded":"2024-09-21T03:15:00.260427+0000","last_fullsized":"2024-09-21T03:15:00.260427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259765+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259765+0000","last_peered":"2024-09-21T03:15:00.259765+0000","last_clean":"2024-09-21T03:15:00.259765+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259765+0000","last_undegraded":"2024-09-21T03:15:00.259765+0000","last_fullsized":"2024-09-21T03:15:00.259765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264004+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264004+0000","last_peered":"2024-09-21T03:15:00.264004+0000","last_clean":"2024-09-21T03:15:00.264004+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264004+0000","last_undegraded":"2024-09-21T03:15:00.264004+0000","last_fullsized":"2024-09-21T03:15:00.264004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258614+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:00.258614+0000","last_peered":"2024-09-21T03:15:00.258614+0000","last_clean":"2024-09-21T03:15:00.258614+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:00.258614+0000","last_undegraded":"2024-09-21T03:15:00.258614+0000","last_fullsized":"2024-09-21T03:15:00.258614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259707+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259707+0000","last_peered":"2024-09-21T03:15:00.259707+0000","last_clean":"2024-09-21T03:15:00.259707+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259707+0000","last_undegraded":"2024-09-21T03:15:00.259707+0000","last_fullsized":"2024-09-21T03:15:00.259707+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265219+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:00.265219+0000","last_peered":"2024-09-21T03:15:00.265219+0000","last_clean":"2024-09-21T03:15:00.265219+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:00.265219+0000","last_undegraded":"2024-09-21T03:15:00.265219+0000","last_fullsized":"2024-09-21T03:15:00.265219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:49.850481+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:15:49.850481+0000","last_peered":"2024-09-21T03:15:49.850481+0000","last_clean":"2024-09-21T03:15:49.850481+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:15:49.850481+0000","last_undegraded":"2024-09-21T03:15:49.850481+0000","last_fullsized":"2024-09-21T03:15:49.850481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264186+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:00.264186+0000","last_peered":"2024-09-21T03:15:00.264186+0000","last_clean":"2024-09-21T03:15:00.264186+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:00.264186+0000","last_undegraded":"2024-09-21T03:15:00.264186+0000","last_fullsized":"2024-09-21T03:15:00.264186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:48.849047+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:15:48.849047+0000","last_peered":"2024-09-21T03:15:48.849047+0000","last_clean":"2024-09-21T03:15:48.849047+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:15:48.849047+0000","last_undegraded":"2024-09-21T03:15:48.849047+0000","last_fullsized":"2024-09-21T03:15:48.849047+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.266475+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:00.266475+0000","last_peered":"2024-09-21T03:15:00.266475+0000","last_clean":"2024-09-21T03:15:00.266475+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:00.266475+0000","last_undegraded":"2024-09-21T03:15:00.266475+0000","last_fullsized":"2024-09-21T03:15:00.266475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265087+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265087+0000","last_peered":"2024-09-21T03:15:00.265087+0000","last_clean":"2024-09-21T03:15:00.265087+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265087+0000","last_undegraded":"2024-09-21T03:15:00.265087+0000","last_fullsized":"2024-09-21T03:15:00.265087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.262888+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:00.262888+0000","last_peered":"2024-09-21T03:15:00.262888+0000","last_clean":"2024-09-21T03:15:00.262888+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:00.262888+0000","last_undegraded":"2024-09-21T03:15:00.262888+0000","last_fullsized":"2024-09-21T03:15:00.262888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261615+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261615+0000","last_peered":"2024-09-21T03:15:00.261615+0000","last_clean":"2024-09-21T03:15:00.261615+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261615+0000","last_undegraded":"2024-09-21T03:15:00.261615+0000","last_fullsized":"2024-09-21T03:15:00.261615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265685+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:00.265685+0000","last_peered":"2024-09-21T03:15:00.265685+0000","last_clean":"2024-09-21T03:15:00.265685+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:00.265685+0000","last_undegraded":"2024-09-21T03:15:00.265685+0000","last_fullsized":"2024-09-21T03:15:00.265685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:02.607478+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:02.607478+0000","last_peered":"2024-09-21T03:15:02.607478+0000","last_clean":"2024-09-21T03:15:02.607478+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:02.607478+0000","last_undegraded":"2024-09-21T03:15:02.607478+0000","last_fullsized":"2024-09-21T03:15:02.607478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259126+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:00.259126+0000","last_peered":"2024-09-21T03:15:00.259126+0000","last_clean":"2024-09-21T03:15:00.259126+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:00.259126+0000","last_undegraded":"2024-09-21T03:15:00.259126+0000","last_fullsized":"2024-09-21T03:15:00.259126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261859+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:00.261859+0000","last_peered":"2024-09-21T03:15:00.261859+0000","last_clean":"2024-09-21T03:15:00.261859+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:00.261859+0000","last_undegraded":"2024-09-21T03:15:00.261859+0000","last_fullsized":"2024-09-21T03:15:00.261859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265921+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265921+0000","last_peered":"2024-09-21T03:15:00.265921+0000","last_clean":"2024-09-21T03:15:00.265921+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265921+0000","last_undegraded":"2024-09-21T03:15:00.265921+0000","last_fullsized":"2024-09-21T03:15:00.265921+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258634+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:00.258634+0000","last_peered":"2024-09-21T03:15:00.258634+0000","last_clean":"2024-09-21T03:15:00.258634+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:00.258634+0000","last_undegraded":"2024-09-21T03:15:00.258634+0000","last_fullsized":"2024-09-21T03:15:00.258634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259779+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:00.259779+0000","last_peered":"2024-09-21T03:15:00.259779+0000","last_clean":"2024-09-21T03:15:00.259779+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:00.259779+0000","last_undegraded":"2024-09-21T03:15:00.259779+0000","last_fullsized":"2024-09-21T03:15:00.259779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.267843+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:00.267843+0000","last_peered":"2024-09-21T03:15:00.267843+0000","last_clean":"2024-09-21T03:15:00.267843+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:00.267843+0000","last_undegraded":"2024-09-21T03:15:00.267843+0000","last_fullsized":"2024-09-21T03:15:00.267843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265078+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:00.265078+0000","last_peered":"2024-09-21T03:15:00.265078+0000","last_clean":"2024-09-21T03:15:00.265078+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:00.265078+0000","last_undegraded":"2024-09-21T03:15:00.265078+0000","last_fullsized":"2024-09-21T03:15:00.265078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263536+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:00.263536+0000","last_peered":"2024-09-21T03:15:00.263536+0000","last_clean":"2024-09-21T03:15:00.263536+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:00.263536+0000","last_undegraded":"2024-09-21T03:15:00.263536+0000","last_fullsized":"2024-09-21T03:15:00.263536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259025+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259025+0000","last_peered":"2024-09-21T03:15:00.259025+0000","last_clean":"2024-09-21T03:15:00.259025+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259025+0000","last_undegraded":"2024-09-21T03:15:00.259025+0000","last_fullsized":"2024-09-21T03:15:00.259025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258780+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:00.258780+0000","last_peered":"2024-09-21T03:15:00.258780+0000","last_clean":"2024-09-21T03:15:00.258780+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:00.258780+0000","last_undegraded":"2024-09-21T03:15:00.258780+0000","last_fullsized":"2024-09-21T03:15:00.258780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:50.851442+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:15:50.851442+0000","last_peered":"2024-09-21T03:15:50.851442+0000","last_clean":"2024-09-21T03:15:50.851442+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:15:50.851442+0000","last_undegraded":"2024-09-21T03:15:50.851442+0000","last_fullsized":"2024-09-21T03:15:50.851442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":416,"num_write":243,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743452,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":416756,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939682,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072083,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":482157,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":481614,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":872448,"data_stored":417674,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:15:55.727 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:15:55.728 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:15:55.728 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:15:55.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:15:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:56.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:56 smithi083 ceph-mon[31219]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:56.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:56 smithi083 ceph-mon[31219]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:56.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:56 smithi193 ceph-mon[35224]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:56 smithi193 ceph-mon[35224]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:56.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:56 smithi142 ceph-mon[34512]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:15:56.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:56 smithi142 ceph-mon[34512]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:15:56.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 -- 172.21.15.83:0/1888093189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 msgr2=0x7f74181087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 --2- 172.21.15.83:0/1888093189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f74181087b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7414009900 tx=0x7f741402a410 comp rx=0 tx=0).stop 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 -- 172.21.15.83:0/1888093189 shutdown_connections 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 --2- 172.21.15.83:0/1888093189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 --2- 172.21.15.83:0/1888093189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f74181087b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 --2- 172.21.15.83:0/1888093189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f7418102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 -- 172.21.15.83:0/1888093189 >> 172.21.15.83:0/1888093189 conn(0x7f7418079730 msgr2=0x7f7418079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 -- 172.21.15.83:0/1888093189 shutdown_connections 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.508+0000 7f741f921640 1 -- 172.21.15.83:0/1888093189 wait complete. 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 Processor -- start 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 -- start start 2024-09-21T03:15:56.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f741819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74181129a0 con 0x7f74181083a0 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7418112820 con 0x7f7418108cf0 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741f921640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7418112b20 con 0x7f7418102a60 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57770/0 (socket says 172.21.15.83:57770) 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 -- 172.21.15.83:0/461125347 learned_addr learned my addr 172.21.15.83:0/461125347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741d6bd640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f741819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741debe640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 -- 172.21.15.83:0/461125347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 msgr2=0x7f741819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f741819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 -- 172.21.15.83:0/461125347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 msgr2=0x7f741819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:56.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741cebc640 1 -- 172.21.15.83:0/461125347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181a65a0 con 0x7f74181083a0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741d6bd640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f741819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.509+0000 7f741debe640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f741cebc640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7414031a90 tx=0x7f7414004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741403f030 con 0x7f74181083a0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7414003cb0 con 0x7f74181083a0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181a6830 con 0x7f74181083a0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414035070 con 0x7f74181083a0 2024-09-21T03:15:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.510+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181a6d90 con 0x7f74181083a0 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.511+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7418102e70 con 0x7f74181083a0 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.512+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7414042030 con 0x7f74181083a0 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.513+0000 7f740e7fc640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 0x7f74040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.513+0000 7f741d6bd640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 0x7f74040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.513+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f7414107c40 con 0x7f74181083a0 2024-09-21T03:15:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.514+0000 7f741d6bd640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 0x7f74040b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f741806ba20 tx=0x7f7408008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.517+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f741410e080 con 0x7f74181083a0 2024-09-21T03:15:56.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.737+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f741802e640 con 0x7f74181083a0 2024-09-21T03:15:56.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.738+0000 7f740e7fc640 1 -- 172.21.15.83:0/461125347 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f74140ce2d0 con 0x7f74181083a0 2024-09-21T03:15:56.739 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:56.739 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:15:56.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 msgr2=0x7f74040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 0x7f74040b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f741806ba20 tx=0x7f7408008040 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 msgr2=0x7f741819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7414031a90 tx=0x7f7414004270 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 shutdown_connections 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74040b65f0 0x7f74040b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7418108cf0 0x7f741819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74181083a0 0x7f741819b7d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 --2- 172.21.15.83:0/461125347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7418102a60 0x7f741819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 >> 172.21.15.83:0/461125347 conn(0x7f7418079730 msgr2=0x7f74181052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 shutdown_connections 2024-09-21T03:15:56.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:56.740+0000 7f741f921640 1 -- 172.21.15.83:0/461125347 wait complete. 2024-09-21T03:15:57.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:57 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/461125347' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:15:57.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:57 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/461125347' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:15:57.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:57 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/461125347' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:15:57.456 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:15:57.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:15:57.702 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:58.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:58 smithi083 ceph-mon[31219]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 -- 172.21.15.83:0/546134484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75941031c0 msgr2=0x7f75941035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 --2- 172.21.15.83:0/546134484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75941031c0 0x7f75941035d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7590009900 tx=0x7f759002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 -- 172.21.15.83:0/546134484 shutdown_connections 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 --2- 172.21.15.83:0/546134484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7594108f50 0x7f759410f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 --2- 172.21.15.83:0/546134484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108600 0x7f7594108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 --2- 172.21.15.83:0/546134484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75941031c0 0x7f75941035d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 -- 172.21.15.83:0/546134484 >> 172.21.15.83:0/546134484 conn(0x7f7594079e30 msgr2=0x7f759407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 -- 172.21.15.83:0/546134484 shutdown_connections 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.169+0000 7f759bbe0640 1 -- 172.21.15.83:0/546134484 wait complete. 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 Processor -- start 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 -- start start 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75941031c0 0x7f759419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108f50 0x7f759419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113d30 con 0x7f7594108600 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113bb0 con 0x7f7594108f50 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759bbe0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113eb0 con 0x7f75941031c0 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57780/0 (socket says 172.21.15.83:57780) 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 -- 172.21.15.83:0/2075782636 learned_addr learned my addr 172.21.15.83:0/2075782636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759997c640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75941031c0 0x7f759419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759a17d640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108f50 0x7f759419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 -- 172.21.15.83:0/2075782636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75941031c0 msgr2=0x7f759419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75941031c0 0x7f759419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 -- 172.21.15.83:0/2075782636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108f50 msgr2=0x7f759419ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108f50 0x7f759419ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.170+0000 7f759917b640 1 -- 172.21.15.83:0/2075782636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75941a06d0 con 0x7f7594108600 2024-09-21T03:15:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f759917b640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f758400c970 tx=0x7f758400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584007bb0 con 0x7f7594108600 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7584007d50 con 0x7f7594108600 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758400f450 con 0x7f7594108600 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a09c0 con 0x7f7594108600 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.171+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a7120 con 0x7f7594108600 2024-09-21T03:15:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.172+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75941035d0 con 0x7f7594108600 2024-09-21T03:15:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.174+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7584004eb0 con 0x7f7594108600 2024-09-21T03:15:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.175+0000 7f758affd640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 0x7f75800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.175+0000 7f759997c640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 0x7f75800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.175+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f75840e76c0 con 0x7f7594108600 2024-09-21T03:15:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.175+0000 7f759997c640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 0x7f75800b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7590031440 tx=0x7f7590009560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.178+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7584014030 con 0x7f7594108600 2024-09-21T03:15:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:58 smithi193 ceph-mon[35224]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.390+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759419b890 con 0x7f7594108600 2024-09-21T03:15:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.391+0000 7f758affd640 1 -- 172.21.15.83:0/2075782636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f759419b890 con 0x7f7594108600 2024-09-21T03:15:58.393 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:58.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:15:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 msgr2=0x7f75800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 0x7f75800b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7590031440 tx=0x7f7590009560 comp rx=0 tx=0).stop 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 msgr2=0x7f759419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f758400c970 tx=0x7f758400ce30 comp rx=0 tx=0).stop 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 shutdown_connections 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f75800b65f0 0x7f75800b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7594108f50 0x7f759419ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7594108600 0x7f759419ad00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.393+0000 7f759bbe0640 1 --2- 172.21.15.83:0/2075782636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75941031c0 0x7f759419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.394+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 >> 172.21.15.83:0/2075782636 conn(0x7f7594079e30 msgr2=0x7f75941066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.394+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 shutdown_connections 2024-09-21T03:15:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:58.394+0000 7f759bbe0640 1 -- 172.21.15.83:0/2075782636 wait complete. 2024-09-21T03:15:58.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:58 smithi142 ceph-mon[34512]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:15:59.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:15:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:15:59.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:15:59 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2075782636' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:15:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:15:59 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2075782636' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:15:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:15:59 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2075782636' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.769+0000 7fd497406640 1 -- 172.21.15.83:0/1989046211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 msgr2=0x7fd490104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.769+0000 7fd497406640 1 --2- 172.21.15.83:0/1989046211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd490104de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd480009900 tx=0x7fd48002a410 comp rx=0 tx=0).stop 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 -- 172.21.15.83:0/1989046211 shutdown_connections 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 --2- 172.21.15.83:0/1989046211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 0x7fd490110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 --2- 172.21.15.83:0/1989046211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 0x7fd49010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 --2- 172.21.15.83:0/1989046211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd490104de0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 -- 172.21.15.83:0/1989046211 >> 172.21.15.83:0/1989046211 conn(0x7fd490079740 msgr2=0x7fd490079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 -- 172.21.15.83:0/1989046211 shutdown_connections 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 -- 172.21.15.83:0/1989046211 wait complete. 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 Processor -- start 2024-09-21T03:15:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.770+0000 7fd497406640 1 -- start start 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 0x7fd49019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 0x7fd49019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd490113af0 con 0x7fd4901049d0 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd490113970 con 0x7fd49010de30 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd490113c70 con 0x7fd490105320 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48384/0 (socket says 172.21.15.83:48384) 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 -- 172.21.15.83:0/3137197255 learned_addr learned my addr 172.21.15.83:0/3137197255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4949a1640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 0x7fd49019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4959a3640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 0x7fd49019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 -- 172.21.15.83:0/3137197255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 msgr2=0x7fd49019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 0x7fd49019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 -- 172.21.15.83:0/3137197255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 msgr2=0x7fd49019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 0x7fd49019fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 -- 172.21.15.83:0/3137197255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4901a04b0 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd4951a2640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd480031a90 tx=0x7fd480004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48003f030 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd480003cb0 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4901a0740 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.771+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480035070 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.772+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901a0c20 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.772+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49006c240 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.774+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd480042030 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.775+0000 7fd48e7fc640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 0x7fd4780b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.775+0000 7fd4949a1640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 0x7fd4780b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.775+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd480108010 con 0x7fd4901049d0 2024-09-21T03:15:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.775+0000 7fd4949a1640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 0x7fd4780b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd490079fd0 tx=0x7fd484008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:15:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.778+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4800ce6a0 con 0x7fd4901049d0 2024-09-21T03:15:59.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.992+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd49019b730 con 0x7fd4901049d0 2024-09-21T03:15:59.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.993+0000 7fd48e7fc640 1 -- 172.21.15.83:0/3137197255 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7fd4800d2a80 con 0x7fd4901049d0 2024-09-21T03:15:59.995 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:15:59.995 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:15:59.995 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 msgr2=0x7fd4780b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 0x7fd4780b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd490079fd0 tx=0x7fd484008040 comp rx=0 tx=0).stop 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 msgr2=0x7fd49019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd480031a90 tx=0x7fd480004270 comp rx=0 tx=0).stop 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 shutdown_connections 2024-09-21T03:15:59.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd4780b6620 0x7fd4780b8b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd49010de30 0x7fd49019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd490105320 0x7fd49019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 --2- 172.21.15.83:0/3137197255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4901049d0 0x7fd49019a600 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 >> 172.21.15.83:0/3137197255 conn(0x7fd490079740 msgr2=0x7fd4900ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 shutdown_connections 2024-09-21T03:15:59.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:15:59.995+0000 7fd497406640 1 -- 172.21.15.83:0/3137197255 wait complete. 2024-09-21T03:16:00.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:00 smithi193 ceph-mon[35224]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:00.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:00 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3137197255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:00 smithi083 ceph-mon[31219]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:00.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:00 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3137197255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:00 smithi142 ceph-mon[34512]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:00 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3137197255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:00.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:16:00.970 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:01.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2213790245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24102a60 msgr2=0x7f4d24102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:01.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2213790245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24102a60 0x7f4d24102e70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009900 tx=0x7f4d1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:01.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2213790245 shutdown_connections 2024-09-21T03:16:01.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2213790245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d24108cf0 0x7f4d2410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2213790245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d241083a0 0x7f4d241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2213790245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24102a60 0x7f4d24102e70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2213790245 >> 172.21.15.83:0/2213790245 conn(0x7f4d24079730 msgr2=0x7f4d24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2213790245 shutdown_connections 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.413+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2213790245 wait complete. 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 Processor -- start 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 -- start start 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d24102a60 0x7f4d24104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d241083a0 0x7f4d24104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24112960 con 0x7f4d24108cf0 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241127e0 con 0x7f4d241083a0 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d2b22c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24112ae0 con 0x7f4d24102a60 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d297c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d297c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48406/0 (socket says 172.21.15.83:48406) 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d297c9640 1 -- 172.21.15.83:0/2171893492 learned_addr learned my addr 172.21.15.83:0/2171893492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.414+0000 7f4d28fc8640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d24102a60 0x7f4d24104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d23fff640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d241083a0 0x7f4d24104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 -- 172.21.15.83:0/2171893492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d24102a60 msgr2=0x7f4d24104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d24102a60 0x7f4d24104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 -- 172.21.15.83:0/2171893492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d241083a0 msgr2=0x7f4d24104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d241083a0 0x7f4d24104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 -- 172.21.15.83:0/2171893492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d241a8690 con 0x7f4d24108cf0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d297c9640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c970 tx=0x7f4d0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c007bb0 con 0x7f4d24108cf0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d241a8980 con 0x7f4d24108cf0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c007d50 con 0x7f4d24108cf0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c018770 con 0x7f4d24108cf0 2024-09-21T03:16:01.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.415+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d241a8e70 con 0x7f4d24108cf0 2024-09-21T03:16:01.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.416+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d24102e70 con 0x7f4d24108cf0 2024-09-21T03:16:01.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.418+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4d0c00f990 con 0x7f4d24108cf0 2024-09-21T03:16:01.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.418+0000 7f4d21ffb640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 0x7f4d100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:01.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.418+0000 7f4d28fc8640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 0x7f4d100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:01.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.418+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4d0c0e7510 con 0x7f4d24108cf0 2024-09-21T03:16:01.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.419+0000 7f4d28fc8640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 0x7f4d100b8a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000c00 tx=0x7f4d1c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:01.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.423+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d0c014030 con 0x7f4d24108cf0 2024-09-21T03:16:01.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.603+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d241a92a0 con 0x7f4d24108cf0 2024-09-21T03:16:01.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.604+0000 7f4d21ffb640 1 -- 172.21.15.83:0/2171893492 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7f4d0c0add00 con 0x7f4d24108cf0 2024-09-21T03:16:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:01.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:16:01.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 msgr2=0x7f4d100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 0x7f4d100b8a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000c00 tx=0x7f4d1c031400 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 msgr2=0x7f4d24104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c970 tx=0x7f4d0c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 shutdown_connections 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4d100b6560 0x7f4d100b8a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d24108cf0 0x7f4d24104cd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d241083a0 0x7f4d24104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 --2- 172.21.15.83:0/2171893492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d24102a60 0x7f4d24104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 >> 172.21.15.83:0/2171893492 conn(0x7f4d24079730 msgr2=0x7f4d24101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 shutdown_connections 2024-09-21T03:16:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:01.607+0000 7f4d2b22c640 1 -- 172.21.15.83:0/2171893492 wait complete. 2024-09-21T03:16:02.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:16:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:02.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:02 smithi193 ceph-mon[35224]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:02 smithi193 ceph-mon[35224]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:16:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:02 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2171893492' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:02 smithi083 ceph-mon[31219]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:02 smithi083 ceph-mon[31219]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:16:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:02 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2171893492' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:02 smithi142 ceph-mon[34512]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:02 smithi142 ceph-mon[34512]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:16:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:02 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2171893492' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.023+0000 7f4617c3e640 1 -- 172.21.15.83:0/2490332393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 msgr2=0x7f4610110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.023+0000 7f4617c3e640 1 --2- 172.21.15.83:0/2490332393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610110650 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4608009ca0 tx=0x7f460802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.023+0000 7f4617c3e640 1 -- 172.21.15.83:0/2490332393 shutdown_connections 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 --2- 172.21.15.83:0/2490332393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610110650 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 --2- 172.21.15.83:0/2490332393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46101094d0 0x7f46101098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 --2- 172.21.15.83:0/2490332393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4610103b90 0x7f4610103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 -- 172.21.15.83:0/2490332393 >> 172.21.15.83:0/2490332393 conn(0x7f46100ff5e0 msgr2=0x7f4610101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 -- 172.21.15.83:0/2490332393 shutdown_connections 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.024+0000 7f4617c3e640 1 -- 172.21.15.83:0/2490332393 wait complete. 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 Processor -- start 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 -- start start 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4610103b90 0x7f4610075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 0x7f4610075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113ba0 con 0x7f4610109e20 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113a20 con 0x7f46101094d0 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f4617c3e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113d20 con 0x7f4610103b90 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48424/0 (socket says 172.21.15.83:48424) 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 -- 172.21.15.83:0/3547344870 learned_addr learned my addr 172.21.15.83:0/3547344870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46159da640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4610103b90 0x7f4610075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46151d9640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 0x7f4610075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 -- 172.21.15.83:0/3547344870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4610103b90 msgr2=0x7f4610075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4610103b90 0x7f4610075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 -- 172.21.15.83:0/3547344870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 msgr2=0x7f4610075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 0x7f4610075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46161db640 1 -- 172.21.15.83:0/3547344870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101a8850 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.025+0000 7f46151d9640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 0x7f4610075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f46161db640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4608005cc0 tx=0x7f4608003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608004040 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4608004690 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101a8ae0 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608030e70 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.026+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a9070 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.027+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4610104020 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.028+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f46080305e0 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.029+0000 7f4606ffd640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 0x7f45fc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.029+0000 7f46159da640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 0x7f45fc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.029+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f46081089a0 con 0x7f4610109e20 2024-09-21T03:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.030+0000 7f46159da640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 0x7f45fc0b8af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4610077160 tx=0x7f460c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.032+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46080cf190 con 0x7f4610109e20 2024-09-21T03:16:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.246+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4610076a10 con 0x7f4610109e20 2024-09-21T03:16:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.246+0000 7f4606ffd640 1 -- 172.21.15.83:0/3547344870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f4610076a10 con 0x7f4610109e20 2024-09-21T03:16:03.249 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:03.249 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:16:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 msgr2=0x7f45fc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 0x7f45fc0b8af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4610077160 tx=0x7f460c008040 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 msgr2=0x7f4610076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4608005cc0 tx=0x7f4608003ba0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 shutdown_connections 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45fc0b65e0 0x7f45fc0b8af0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4610109e20 0x7f4610076190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46101094d0 0x7f4610075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 --2- 172.21.15.83:0/3547344870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4610103b90 0x7f4610075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 >> 172.21.15.83:0/3547344870 conn(0x7f46100ff5e0 msgr2=0x7f4610101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 shutdown_connections 2024-09-21T03:16:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:03.249+0000 7f4617c3e640 1 -- 172.21.15.83:0/3547344870 wait complete. 2024-09-21T03:16:03.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:03 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3547344870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:03 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3547344870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:03 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3547344870' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:03.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:14:37.183732+0000', 'modified': '2024-09-21T03:14:48.753854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24647, 'mds_1': 24628, 'mds_2': 24620, 'mds_3': 24613, 'mds_4': 24601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/3046510766', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3046510766}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3046510766}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/2540451962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 2540451962}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 2540451962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/1409264490', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 1409264490}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 1409264490}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24571': {'gid': 24571, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/817680520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 817680520}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 817680520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24593': {'gid': 24593, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/3055596594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 3055596594}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 3055596594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24601': {'gid': 24601, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.193:6835/1130551949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1130551949}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1130551949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.193:6837/3456789844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 3456789844}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 3456789844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.142:6837/1691832808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1691832808}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1691832808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/357057598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 357057598}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 357057598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24647': {'gid': 24647, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.142:6839/3152549242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3152549242}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3152549242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24647, 'qdb_cluster': [24601, 24613, 24620, 24628, 24647]} 2024-09-21T03:16:03.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:16:03.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T03:16:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:04.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:04 smithi083 ceph-mon[31219]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:04.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:04 smithi193 ceph-mon[35224]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 -- 172.21.15.83:0/2642073041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10102b10 msgr2=0x7f0e10102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 --2- 172.21.15.83:0/2642073041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10102b10 0x7f0e10102f20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 -- 172.21.15.83:0/2642073041 shutdown_connections 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 --2- 172.21.15.83:0/2642073041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1010f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 --2- 172.21.15.83:0/2642073041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10108450 0x7f0e10108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 --2- 172.21.15.83:0/2642073041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10102b10 0x7f0e10102f20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 -- 172.21.15.83:0/2642073041 >> 172.21.15.83:0/2642073041 conn(0x7f0e100796d0 msgr2=0x7f0e10079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 -- 172.21.15.83:0/2642073041 shutdown_connections 2024-09-21T03:16:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.600+0000 7f0e17b38640 1 -- 172.21.15.83:0/2642073041 wait complete. 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 Processor -- start 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 -- start start 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 0x7f0e1019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10112a60 con 0x7f0e10108450 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e101128e0 con 0x7f0e10108da0 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e17b38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10112be0 con 0x7f0e10102b10 2024-09-21T03:16:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48444/0 (socket says 172.21.15.83:48444) 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 -- 172.21.15.83:0/157918227 learned_addr learned my addr 172.21.15.83:0/157918227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e158d4640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 0x7f0e1019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e160d5640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 -- 172.21.15.83:0/157918227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 msgr2=0x7f0e1019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 0x7f0e1019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 -- 172.21.15.83:0/157918227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 msgr2=0x7f0e1019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e150d3640 1 -- 172.21.15.83:0/157918227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101a6680 con 0x7f0e10108450 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e160d5640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.601+0000 7f0e158d4640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 0x7f0e1019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e150d3640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d750 tx=0x7f0e0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e06ffd640 1 -- 172.21.15.83:0/157918227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00017070 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101a6970 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e06ffd640 1 -- 172.21.15.83:0/157918227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e000054d0 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e06ffd640 1 -- 172.21.15.83:0/157918227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00004c30 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.602+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101a6eb0 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.604+0000 7f0e1e0ac740 1 Processor -- start 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.604+0000 7f0e1e0ac740 1 -- start start 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e06ffd640 1 -- 172.21.15.83:0/157918227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0e00004dd0 con 0x7f0e10108450 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636ee14cc10 con 0x5636ede060a0 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636ee13c040 con 0x5636ede0e280 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636ee13fe50 con 0x5636ede2ef60 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48460/0 (socket says 172.21.15.83:48460) 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 -- 172.21.15.83:0/223986573 learned_addr learned my addr 172.21.15.83:0/223986573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e160d5640 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e150d3640 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 -- 172.21.15.83:0/223986573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 msgr2=0x5636ee1d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1d4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 -- 172.21.15.83:0/223986573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 msgr2=0x5636ee1d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.605+0000 7f0e158d4640 1 -- 172.21.15.83:0/223986573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ede9a540 con 0x5636ede060a0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0e158d4640 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f19b4b5251ee2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0e06ffd640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 0x7f0dfc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0e160d5640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 0x7f0dfc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0dfbfff640 1 -- 172.21.15.83:0/223986573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c03f070 con 0x5636ede060a0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0dfbfff640 1 -- 172.21.15.83:0/223986573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c0316a0 con 0x5636ede060a0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0dfbfff640 1 -- 172.21.15.83:0/223986573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c035070 con 0x5636ede060a0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.606+0000 7f0e06ffd640 1 -- 172.21.15.83:0/157918227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0e000e6ec0 con 0x7f0e10108450 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/223986573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 msgr2=0x5636ee1d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e160d5640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 0x7f0dfc0b8aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005fd0 tx=0x7f0e08008370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c032ce0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/223986573 shutdown_connections 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1d5440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/223986573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/223986573 >> 172.21.15.83:0/223986573 conn(0x5636ede23350 msgr2=0x5636ee1d2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/223986573 shutdown_connections 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.607+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/223986573 wait complete. 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 -- start start 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 Processor -- start 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636edf9d5f0 con 0x5636ede0e280 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636ee0d7010 con 0x5636ede060a0 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1e3680 con 0x5636ede2ef60 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e1e0ac740 10 client.0 _ll_register_callbacks cb 0x5636ee136340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.608+0000 7f0e150d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48462/0 (socket says 172.21.15.83:48462) 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 -- 172.21.15.83:0/842265344 learned_addr learned my addr 172.21.15.83:0/842265344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e160d5640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 -- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 msgr2=0x5636ee1ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 -- 172.21.15.83:0/842265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 msgr2=0x5636ee1e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1e07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 -- 172.21.15.83:0/842265344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ee104a00 con 0x5636ede0e280 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e158d4640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e150d3640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d720 tx=0x7f0e000a8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f180cd94bea62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0df97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00020030 con 0x5636ede0e280 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c0316a0 con 0x5636ede0e280 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0e000b6540 con 0x5636ede0e280 2024-09-21T03:16:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e000ac720 con 0x5636ede0e280 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e1e0ac740 10 client.15234 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.609+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636ee1ef620 con 0x5636ede0e280 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c0316a0 con 0x5636ede0e280 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0e00098190 con 0x5636ede0e280 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 10 client.15234 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 20 client.15234 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636edded8e0 0x5636ee1f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 4 client.15234 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636ee14eb60 con 0x5636edded8e0 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.610+0000 7f0e158d4640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636edded8e0 0x5636ee1f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.611+0000 7f0e158d4640 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636edded8e0 0x5636ee1f3090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2defc03af6d57c4 server_cookie=dd65d9081d83898c in_seq=0 out_seq=0 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.611+0000 7f0df97fa640 10 client.15234 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0df97fa640 1 -- 172.21.15.83:0/842265344 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636ee14eb60 con 0x5636edded8e0 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0df97fa640 10 client.15234 handle_command_reply: tid=0 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 client.15234 shutdown 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 msgr2=0x5636ee1e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d720 tx=0x7f0e000a8f10 comp rx=0 tx=0).stop 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 shutdown_connections 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636edded8e0 0x5636ee1f3090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede0e280 0x5636ee1e0db0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/842265344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0df97fa640 0 client.15234 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 >> 172.21.15.83:0/842265344 conn(0x5636ede23350 msgr2=0x5636ee1d8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 shutdown_connections 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/842265344 wait complete. 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.612+0000 7f0e1e0ac740 20 client.15234 trim_cache size 0 max 16384 2024-09-21T03:16:04.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 Processor -- start 2024-09-21T03:16:04.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 -- start start 2024-09-21T03:16:04.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f13d0 con 0x5636ede060a0 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f1aa0 con 0x5636ede0e280 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f2170 con 0x5636ede2ef60 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48474/0 (socket says 172.21.15.83:48474) 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 -- 172.21.15.83:0/1643153354 learned_addr learned my addr 172.21.15.83:0/1643153354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e150d3640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 -- 172.21.15.83:0/1643153354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 msgr2=0x5636ee1ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e160d5640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 -- 172.21.15.83:0/1643153354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 msgr2=0x5636ee1cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 -- 172.21.15.83:0/1643153354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ee1d7790 con 0x5636ede060a0 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e160d5640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e150d3640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.633+0000 7f0e158d4640 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5636ee1d3b80 tx=0x7f0e0c0052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea9c8cc7ed887f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.634+0000 7f0dfb7fe640 1 -- 172.21.15.83:0/1643153354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c03f040 con 0x5636ede060a0 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.634+0000 7f0dfb7fe640 1 -- 172.21.15.83:0/1643153354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c004030 con 0x5636ede060a0 2024-09-21T03:16:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.634+0000 7f0dfb7fe640 1 -- 172.21.15.83:0/1643153354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c035040 con 0x5636ede060a0 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1643153354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 msgr2=0x5636ee1db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5636ee1d3b80 tx=0x7f0e0c0052c0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1643153354 shutdown_connections 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede2ef60 0x5636ee1ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede0e280 0x5636ee1cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1643153354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede060a0 0x5636ee1db020 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1643153354 >> 172.21.15.83:0/1643153354 conn(0x5636ede23350 msgr2=0x5636ee1db980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1643153354 shutdown_connections 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1643153354 wait complete. 2024-09-21T03:16:04.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 -- start start 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 Processor -- start 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.635+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede0e280 0x5636ee1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f2f10 con 0x5636ede2ef60 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f2840 con 0x5636ede060a0 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636ee1f35e0 con 0x5636ede0e280 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 10 client.0 _ll_register_callbacks cb 0x7f0e0000de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e150d3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede0e280 0x5636ee1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e158d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48478/0 (socket says 172.21.15.83:48478) 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 -- 172.21.15.83:0/1427156634 learned_addr learned my addr 172.21.15.83:0/1427156634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 -- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede0e280 msgr2=0x5636ee1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede0e280 0x5636ee1f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 -- 172.21.15.83:0/1427156634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 msgr2=0x5636ee1f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 -- 172.21.15.83:0/1427156634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ee1fad40 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e160d5640 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0e08010ef0 tx=0x7f0e08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8263522003be6e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0df97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0801f640 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0e0801f7e0 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.636+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c004030 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e08004b40 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 10 client.15240 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636ee25e880 con 0x5636ede2ef60 2024-09-21T03:16:04.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c004030 con 0x5636ede2ef60 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0e0802f070 con 0x5636ede2ef60 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 10 client.15240 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 20 client.15240 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636eddc2c90 0x5636ee262260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 4 client.15240 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "status"}) -- 0x5636ee14a610 con 0x5636eddc2c90 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.637+0000 7f0e158d4640 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636eddc2c90 0x5636ee262260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0df97fa640 10 client.15240 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e158d4640 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636eddc2c90 0x5636ee262260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee567f33a81be5ff server_cookie=f91aa3c9a387880c in_seq=0 out_seq=0 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0df97fa640 1 -- 172.21.15.83:0/1427156634 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5636ee14a610 con 0x5636eddc2c90 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0df97fa640 10 client.15240 handle_command_reply: tid=0 2024-09-21T03:16:04.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 client.15240 shutdown 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 msgr2=0x5636ee1f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0e08010ef0 tx=0x7f0e08004a40 comp rx=0 tx=0).stop 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 shutdown_connections 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5636eddc2c90 0x5636ee262260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636ede2ef60 0x5636ee1f5ed0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0df97fa640 0 client.15240 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636ede0e280 0x5636ee1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 --2- 172.21.15.83:0/1427156634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636ede060a0 0x5636ee1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 >> 172.21.15.83:0/1427156634 conn(0x5636ede23350 msgr2=0x5636ee1d2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 shutdown_connections 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.638+0000 7f0e1e0ac740 1 -- 172.21.15.83:0/1427156634 wait complete. 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e1e0ac740 20 client.15240 trim_cache size 0 max 16384 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24601, 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 76.781690514000005, 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 113.392396875 2024-09-21T03:16:04.650 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 msgr2=0x7f0dfc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 0x7f0dfc0b8aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005fd0 tx=0x7f0e08008370 comp rx=0 tx=0).stop 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 msgr2=0x7f0e1019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d750 tx=0x7f0e0000dc10 comp rx=0 tx=0).stop 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 shutdown_connections 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0dfc0b6590 0x7f0dfc0b8aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e10108da0 0x7f0e1019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e10108450 0x7f0e1019b930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 --2- 172.21.15.83:0/157918227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e10102b10 0x7f0e1019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 >> 172.21.15.83:0/157918227 conn(0x7f0e100796d0 msgr2=0x7f0e101065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 shutdown_connections 2024-09-21T03:16:04.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:04.639+0000 7f0e17b38640 1 -- 172.21.15.83:0/157918227 wait complete. 2024-09-21T03:16:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:04 smithi142 ceph-mon[34512]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:05.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:16:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 -- 172.21.15.83:0/2381802883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc1083d0 msgr2=0x7fa7dc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 --2- 172.21.15.83:0/2381802883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc1083d0 0x7fa7dc1087e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009900 tx=0x7fa7d002a410 comp rx=0 tx=0).stop 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 -- 172.21.15.83:0/2381802883 shutdown_connections 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 --2- 172.21.15.83:0/2381802883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108d20 0x7fa7dc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 --2- 172.21.15.83:0/2381802883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc1083d0 0x7fa7dc1087e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 --2- 172.21.15.83:0/2381802883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a90 0x7fa7dc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 -- 172.21.15.83:0/2381802883 >> 172.21.15.83:0/2381802883 conn(0x7fa7dc079750 msgr2=0x7fa7dc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 -- 172.21.15.83:0/2381802883 shutdown_connections 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.068+0000 7fa7e111d640 1 -- 172.21.15.83:0/2381802883 wait complete. 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.069+0000 7fa7e111d640 1 Processor -- start 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 -- start start 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a90 0x7fa7dc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc1083d0 0x7fa7dc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc112990 con 0x7fa7dc108d20 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc112810 con 0x7fa7dc102a90 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7e111d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc112b10 con 0x7fa7dc1083d0 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7db59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7db59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48502/0 (socket says 172.21.15.83:48502) 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7db59e640 1 -- 172.21.15.83:0/377550175 learned_addr learned my addr 172.21.15.83:0/377550175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7db59e640 1 -- 172.21.15.83:0/377550175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc1083d0 msgr2=0x7fa7dc1998e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.070+0000 7fa7db59e640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc1083d0 0x7fa7dc1998e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7db59e640 1 -- 172.21.15.83:0/377550175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a90 msgr2=0x7fa7dc199340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7db59e640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a90 0x7fa7dc199340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7db59e640 1 -- 172.21.15.83:0/377550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc19f2b0 con 0x7fa7dc108d20 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7db59e640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400eaa0 tx=0x7fa7d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7c3fff640 1 -- 172.21.15.83:0/377550175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400cdb0 con 0x7fa7dc108d20 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7c3fff640 1 -- 172.21.15.83:0/377550175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d4005020 con 0x7fa7dc108d20 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc19f5a0 con 0x7fa7dc108d20 2024-09-21T03:16:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7c3fff640 1 -- 172.21.15.83:0/377550175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40116d0 con 0x7fa7dc108d20 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.071+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1a5d00 con 0x7fa7dc108d20 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 Processor -- start 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7c3fff640 1 -- 172.21.15.83:0/377550175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa7d4011870 con 0x7fa7dc108d20 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 -- start start 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36addaf60 0x55a36b180f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 0x55a36b181980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a36b0f8c10 con 0x55a36adb20a0 2024-09-21T03:16:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a36b0e8040 con 0x55a36adba280 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7e7691740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a36b0ebe50 con 0x55a36addaf60 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7dad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7dad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48518/0 (socket says 172.21.15.83:48518) 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7dad9d640 1 -- 172.21.15.83:0/1368343444 learned_addr learned my addr 172.21.15.83:0/1368343444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.074+0000 7fa7da59c640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36addaf60 0x55a36b180f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7c3fff640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 0x7fa7c40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7da59c640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 0x55a36b181980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 -- 172.21.15.83:0/1368343444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36addaf60 msgr2=0x55a36b180f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36addaf60 0x55a36b180f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 -- 172.21.15.83:0/1368343444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 msgr2=0x55a36b181980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 0x55a36b181980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 -- 172.21.15.83:0/1368343444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36ae46540 con 0x55a36adb20a0 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7da59c640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 0x55a36b181980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7db59e640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 0x7fa7c40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ba80 tx=0x7fa7cc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae1b8fe66ed7822 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7c3fff640 1 -- 172.21.15.83:0/377550175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7d40e7640 con 0x7fa7dc108d20 2024-09-21T03:16:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7c17fa640 1 -- 172.21.15.83:0/1368343444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc016070 con 0x55a36adb20a0 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7c17fa640 1 -- 172.21.15.83:0/1368343444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc0054d0 con 0x55a36adb20a0 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.075+0000 7fa7c17fa640 1 -- 172.21.15.83:0/1368343444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc010460 con 0x55a36adb20a0 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- 172.21.15.83:0/1368343444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 msgr2=0x55a36b181440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ba80 tx=0x7fa7cc00bf40 comp rx=0 tx=0).stop 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7db59e640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 0x7fa7c40b8a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40a9d00 tx=0x7fa7d40de1d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- 172.21.15.83:0/1368343444 shutdown_connections 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adba280 0x55a36b181980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b181440 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 --2- 172.21.15.83:0/1368343444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36addaf60 0x55a36b180f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- 172.21.15.83:0/1368343444 >> 172.21.15.83:0/1368343444 conn(0x55a36adcf350 msgr2=0x55a36b17edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- 172.21.15.83:0/1368343444 shutdown_connections 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- 172.21.15.83:0/1368343444 wait complete. 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.076+0000 7fa7e7691740 1 -- start start 2024-09-21T03:16:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 Processor -- start 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b18cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36addaf60 0x55a36b198a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a36af495f0 con 0x55a36adb20a0 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a36b083010 con 0x55a36addaf60 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a36b18f6e0 con 0x55a36adba280 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7e7691740 10 client.0 _ll_register_callbacks cb 0x55a36b0e2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48524/0 (socket says 172.21.15.83:48524) 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 -- 172.21.15.83:0/164569214 learned_addr learned my addr 172.21.15.83:0/164569214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36addaf60 0x55a36b198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b18cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 -- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 msgr2=0x55a36b18cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b18cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 -- 172.21.15.83:0/164569214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36addaf60 msgr2=0x55a36b198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36addaf60 0x55a36b198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 -- 172.21.15.83:0/164569214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36b0b0a00 con 0x55a36adb20a0 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b18cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:06.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7da59c640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00057c0 tx=0x7fa7d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7731045e89696152 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.077+0000 7fa7beff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0035070 con 0x55a36adb20a0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7d0004030 con 0x55a36adb20a0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc0054d0 con 0x55a36adb20a0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa7d0004230 con 0x55a36adb20a0 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 10 client.15252 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a36b19b610 con 0x55a36adb20a0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc0054d0 con 0x55a36adb20a0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa7d003a040 con 0x55a36adb20a0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 10 client.15252 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 20 client.15252 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36ad998e0 0x55a36b19f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 4 client.15252 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.078+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36b0fab60 con 0x55a36ad998e0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.079+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36ad998e0 0x55a36b19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.079+0000 7fa7beff5640 10 client.15252 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.079+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36ad998e0 0x55a36b19f080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34fadb0aae9aabd3 server_cookie=41687b73df611393 in_seq=0 out_seq=0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7beff5640 1 -- 172.21.15.83:0/164569214 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a36b0fab60 con 0x55a36ad998e0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7beff5640 10 client.15252 handle_command_reply: tid=0 2024-09-21T03:16:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 client.15252 shutdown 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 msgr2=0x55a36b18caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00057c0 tx=0x7fa7d002a410 comp rx=0 tx=0).stop 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 shutdown_connections 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36addaf60 0x55a36b198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36ad998e0 0x55a36b19f080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b18cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 --2- 172.21.15.83:0/164569214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adb20a0 0x55a36b18caa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7beff5640 0 client.15252 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 >> 172.21.15.83:0/164569214 conn(0x55a36adcf350 msgr2=0x55a36b184090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 shutdown_connections 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 1 -- 172.21.15.83:0/164569214 wait complete. 2024-09-21T03:16:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.080+0000 7fa7e7691740 20 client.15252 trim_cache size 0 max 16384 2024-09-21T03:16:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 Processor -- start 2024-09-21T03:16:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 -- start start 2024-09-21T03:16:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b18b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b186010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19d3c0 con 0x55a36adcf350 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19da90 con 0x55a36adb20a0 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7e7691740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19e160 con 0x55a36adba280 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48540/0 (socket says 172.21.15.83:48540) 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.098+0000 7fa7da59c640 1 -- 172.21.15.83:0/311351175 learned_addr learned my addr 172.21.15.83:0/311351175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 -- 172.21.15.83:0/311351175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 msgr2=0x55a36b186010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b186010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 -- 172.21.15.83:0/311351175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 msgr2=0x55a36b18b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b18b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 -- 172.21.15.83:0/311351175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36b179080 con 0x55a36adcf350 2024-09-21T03:16:06.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7da59c640 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00059d0 tx=0x7fa7d0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6520bc14ff6ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7c0ff9640 1 -- 172.21.15.83:0/311351175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0035040 con 0x55a36adcf350 2024-09-21T03:16:06.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7c0ff9640 1 -- 172.21.15.83:0/311351175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d0005030 con 0x55a36adcf350 2024-09-21T03:16:06.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.099+0000 7fa7c0ff9640 1 -- 172.21.15.83:0/311351175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0031970 con 0x55a36adcf350 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- 172.21.15.83:0/311351175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 msgr2=0x55a36b18b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00059d0 tx=0x7fa7d0031400 comp rx=0 tx=0).stop 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- 172.21.15.83:0/311351175 shutdown_connections 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b186010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b18b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- 172.21.15.83:0/311351175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b18b120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- 172.21.15.83:0/311351175 >> 172.21.15.83:0/311351175 conn(0x55a36b199120 msgr2=0x55a36b187a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- 172.21.15.83:0/311351175 shutdown_connections 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- 172.21.15.83:0/311351175 wait complete. 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 -- start start 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 Processor -- start 2024-09-21T03:16:06.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b1a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.100+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b1a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19ef00 con 0x55a36adcf350 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19e830 con 0x55a36adb20a0 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a36b19f5d0 con 0x55a36adba280 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 10 client.0 _ll_register_callbacks cb 0x7fa7d0031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48546/0 (socket says 172.21.15.83:48546) 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 -- 172.21.15.83:0/742221791 learned_addr learned my addr 172.21.15.83:0/742221791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b1a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b1a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 -- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 msgr2=0x55a36b1a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b1a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 -- 172.21.15.83:0/742221791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 msgr2=0x55a36b1a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b1a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 -- 172.21.15.83:0/742221791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36b1a6990 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7da59c640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0004200 tx=0x7fa7d00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fbee0ad3a5957 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7beff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0062060 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7d00046c0 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0005030 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa7d00048c0 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 10 client.15258 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.101+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a36b20a710 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d0005030 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa7d003ec80 con 0x55a36adcf350 2024-09-21T03:16:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 20 client.15258 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36addaf60 0x55a36b20dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 4 client.15258 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "status"}) -- 0x55a36b0f6610 con 0x55a36addaf60 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.102+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36addaf60 0x55a36b20dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7dad9d640 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36addaf60 0x55a36b20dc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=716be7a200ba976a server_cookie=66ee50e2b448facc in_seq=0 out_seq=0 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7beff5640 10 client.15258 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7beff5640 1 -- 172.21.15.83:0/742221791 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a36b0f6610 con 0x55a36addaf60 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7beff5640 10 client.15258 handle_command_reply: tid=0 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 client.15258 shutdown 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 msgr2=0x55a36b1a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0004200 tx=0x7fa7d00035f0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 shutdown_connections 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a36adcf350 0x55a36b1a19e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55a36addaf60 0x55a36b20dc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a36adba280 0x55a36b1a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 --2- 172.21.15.83:0/742221791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a36adb20a0 0x55a36b1a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 >> 172.21.15.83:0/742221791 conn(0x55a36b199120 msgr2=0x55a36b18b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7beff5640 0 client.15258 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 shutdown_connections 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 1 -- 172.21.15.83:0/742221791 wait complete. 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.103+0000 7fa7e7691740 20 client.15258 trim_cache size 0 max 16384 2024-09-21T03:16:06.111 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24613, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 79.259314474000007, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 112.250188057 2024-09-21T03:16:06.112 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 msgr2=0x7fa7c40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 0x7fa7c40b8a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40a9d00 tx=0x7fa7d40de1d0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 msgr2=0x7fa7dc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400eaa0 tx=0x7fa7d400ef60 comp rx=0 tx=0).stop 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 shutdown_connections 2024-09-21T03:16:06.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6560 0x7fa7c40b8a70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc108d20 0x7fa7dc19eb80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc1083d0 0x7fa7dc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 --2- 172.21.15.83:0/377550175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a90 0x7fa7dc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 >> 172.21.15.83:0/377550175 conn(0x7fa7dc079750 msgr2=0x7fa7dc104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 shutdown_connections 2024-09-21T03:16:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:06.104+0000 7fa7e111d640 1 -- 172.21.15.83:0/377550175 wait complete. 2024-09-21T03:16:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:06 smithi083 ceph-mon[31219]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:06.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:06 smithi193 ceph-mon[35224]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:06 smithi142 ceph-mon[34512]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:06.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:16:07.068 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:07.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/2947142529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4102ae0 msgr2=0x7f4cb4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/2947142529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4102ae0 0x7f4cb4102ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009900 tx=0x7f4ca402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:07.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/2947142529 shutdown_connections 2024-09-21T03:16:07.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/2947142529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108d70 0x7f4cb410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/2947142529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4108420 0x7f4cb4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/2947142529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4102ae0 0x7f4cb4102ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/2947142529 >> 172.21.15.83:0/2947142529 conn(0x7f4cb4079770 msgr2=0x7f4cb4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/2947142529 shutdown_connections 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.528+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/2947142529 wait complete. 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 Processor -- start 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 -- start start 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4102ae0 0x7f4cb419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 0x7f4cb419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112a30 con 0x7f4cb4108d70 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41128b0 con 0x7f4cb4108420 2024-09-21T03:16:07.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbc5b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112bb0 con 0x7f4cb4102ae0 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbab54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbab54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48568/0 (socket says 172.21.15.83:48568) 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.529+0000 7f4cbab54640 1 -- 172.21.15.83:0/1816020029 learned_addr learned my addr 172.21.15.83:0/1816020029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 0x7f4cb419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 -- 172.21.15.83:0/1816020029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4102ae0 msgr2=0x7f4cb419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cba353640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4102ae0 0x7f4cb419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4102ae0 0x7f4cb419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 -- 172.21.15.83:0/1816020029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 msgr2=0x7f4cb419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 0x7f4cb419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 -- 172.21.15.83:0/1816020029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb41a03c0 con 0x7f4cb4108d70 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 0x7f4cb419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbab54640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00cce0 tx=0x7f4cac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cb37fe640 1 -- 172.21.15.83:0/1816020029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00f450 con 0x7f4cb4108d70 2024-09-21T03:16:07.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41a06b0 con 0x7f4cb4108d70 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cb37fe640 1 -- 172.21.15.83:0/1816020029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cac00faa0 con 0x7f4cb4108d70 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cb37fe640 1 -- 172.21.15.83:0/1816020029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac018b70 con 0x7f4cb4108d70 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.530+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb41a6ed0 con 0x7f4cb4108d70 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.532+0000 7f4cc2b2b740 1 Processor -- start 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cb37fe640 1 -- 172.21.15.83:0/1816020029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4cac018d10 con 0x7f4cb4108d70 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 -- start start 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd67d280 0x55c3cda44980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3cd9bbc10 con 0x55c3cd69df60 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3cd9ab040 con 0x55c3cd6750a0 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3cd9aee50 con 0x55c3cd67d280 2024-09-21T03:16:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48570/0 (socket says 172.21.15.83:48570) 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cb9b52640 1 -- 172.21.15.83:0/3653264619 learned_addr learned my addr 172.21.15.83:0/3653264619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cba353640 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.533+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd67d280 0x55c3cda44980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb37fe640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 0x7f4c9c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 -- 172.21.15.83:0/3653264619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd67d280 msgr2=0x55c3cda44980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd67d280 0x55c3cda44980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 -- 172.21.15.83:0/3653264619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 msgr2=0x55c3cda44440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda44440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 -- 172.21.15.83:0/3653264619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3cd709540 con 0x55c3cd69df60 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cbab54640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 0x7f4c9c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800f360 tx=0x7f4ca800b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dbca1cb5f8ad30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb37fe640 1 -- 172.21.15.83:0/1816020029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4cac0e7910 con 0x7f4cb4108d70 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb0ff9640 1 -- 172.21.15.83:0/3653264619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8012860 con 0x55c3cd69df60 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb0ff9640 1 -- 172.21.15.83:0/3653264619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca8011440 con 0x55c3cd69df60 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.534+0000 7f4cb0ff9640 1 -- 172.21.15.83:0/3653264619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800d5d0 con 0x55c3cd69df60 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/3653264619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 msgr2=0x55c3cda43f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800f360 tx=0x7f4ca800b4e0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cbab54640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 0x7f4c9c0b8aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0e56e0 tx=0x7f4cac0de4a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/3653264619 shutdown_connections 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd67d280 0x55c3cda44980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda44440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/3653264619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd69df60 0x55c3cda43f00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/3653264619 >> 172.21.15.83:0/3653264619 conn(0x55c3cd692350 msgr2=0x55c3cda41df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/3653264619 shutdown_connections 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/3653264619 wait complete. 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.535+0000 7f4cc2b2b740 1 -- start start 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 Processor -- start 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda4ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd69df60 0x55c3cda5b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3cd80c5f0 con 0x55c3cd6750a0 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3cd946010 con 0x55c3cd67d280 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda526e0 con 0x55c3cd69df60 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cc2b2b740 10 client.0 _ll_register_callbacks cb 0x55c3cd9a5340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48584/0 (socket says 172.21.15.83:48584) 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 -- 172.21.15.83:0/984337215 learned_addr learned my addr 172.21.15.83:0/984337215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cba353640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda4ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd69df60 0x55c3cda5b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 -- 172.21.15.83:0/984337215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd69df60 msgr2=0x55c3cda5b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd69df60 0x55c3cda5b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 -- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 msgr2=0x55c3cda4ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda4ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.536+0000 7f4cb9b52640 1 -- 172.21.15.83:0/984337215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cd973a00 con 0x55c3cd6750a0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b7b0 tx=0x7f4ca8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91064b2e244ca599 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8012a70 con 0x55c3cd6750a0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4ca8012c10 con 0x55c3cd6750a0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca8011440 con 0x55c3cd6750a0 2024-09-21T03:16:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ca8032050 con 0x55c3cd6750a0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3cda5e600 con 0x55c3cd6750a0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca8011440 con 0x55c3cd6750a0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca802b060 con 0x55c3cd6750a0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 20 client.15270 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd65c8e0 0x55c3cda61d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 4 client.15270 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.537+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3cd9bdb60 con 0x55c3cd65c8e0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.538+0000 7f4cba353640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd65c8e0 0x55c3cda61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.538+0000 7f4c99ffb640 10 client.15270 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.538+0000 7f4cba353640 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd65c8e0 0x55c3cda61d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=643a095b2b91f45b server_cookie=df7a118410435386 in_seq=0 out_seq=0 2024-09-21T03:16:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4c99ffb640 1 -- 172.21.15.83:0/984337215 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3cd9bdb60 con 0x55c3cd65c8e0 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4c99ffb640 10 client.15270 handle_command_reply: tid=0 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 client.15270 shutdown 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 msgr2=0x55c3cda4fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b7b0 tx=0x7f4ca8011e60 comp rx=0 tx=0).stop 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 shutdown_connections 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd65c8e0 0x55c3cda61d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd69df60 0x55c3cda5b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4c99ffb640 0 client.15270 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda4ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/984337215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda4fa30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.539+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 >> 172.21.15.83:0/984337215 conn(0x55c3cd692350 msgr2=0x55c3cda470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.540+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 shutdown_connections 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.540+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/984337215 wait complete. 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.540+0000 7f4cc2b2b740 20 client.15270 trim_cache size 0 max 16384 2024-09-21T03:16:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.557+0000 7f4cc2b2b740 1 Processor -- start 2024-09-21T03:16:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 -- start start 2024-09-21T03:16:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda5b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda5be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda600d0 con 0x55c3cd67d280 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda607a0 con 0x55c3cd6750a0 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda60e70 con 0x55c3cd65c8e0 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48598/0 (socket says 172.21.15.83:48598) 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 -- 172.21.15.83:0/1429724845 learned_addr learned my addr 172.21.15.83:0/1429724845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cba353640 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda5be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda5b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 -- 172.21.15.83:0/1429724845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 msgr2=0x55c3cda5b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda5b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 -- 172.21.15.83:0/1429724845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 msgr2=0x55c3cda5be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda5be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.558+0000 7f4cb9b52640 1 -- 172.21.15.83:0/1429724845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3cda46790 con 0x55c3cd67d280 2024-09-21T03:16:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.559+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b560 tx=0x7f4ca8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc742fbb2e0c174 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.559+0000 7f4c9bfff640 1 -- 172.21.15.83:0/1429724845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8011440 con 0x55c3cd67d280 2024-09-21T03:16:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.559+0000 7f4c9bfff640 1 -- 172.21.15.83:0/1429724845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca800fcb0 con 0x55c3cd67d280 2024-09-21T03:16:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.559+0000 7f4c9bfff640 1 -- 172.21.15.83:0/1429724845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca802d430 con 0x55c3cd67d280 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1429724845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 msgr2=0x55c3cda4a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b560 tx=0x7f4ca8011ef0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1429724845 shutdown_connections 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd67d280 0x55c3cda4a9e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd6750a0 0x55c3cda5be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1429724845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda5b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1429724845 >> 172.21.15.83:0/1429724845 conn(0x55c3cd692350 msgr2=0x55c3cda4fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1429724845 shutdown_connections 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1429724845 wait complete. 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 -- start start 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 Processor -- start 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.560+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda61c10 con 0x55c3cd6750a0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda61540 con 0x55c3cd67d280 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3cda622e0 con 0x55c3cd65c8e0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 10 client.0 _ll_register_callbacks cb 0x7f4ca4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48614/0 (socket says 172.21.15.83:48614) 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 -- 172.21.15.83:0/1766067166 learned_addr learned my addr 172.21.15.83:0/1766067166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 -- 172.21.15.83:0/1766067166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 msgr2=0x55c3cda6a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda6a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 -- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 msgr2=0x55c3cda63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 -- 172.21.15.83:0/1766067166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cda641a0 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cba353640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4031440 tx=0x7f4ca4004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fdfc8b2ba88279 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4035040 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4ca4005230 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.561+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca800fcb0 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ca4053050 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3cda69c30 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca800fcb0 con 0x55c3cd6750a0 2024-09-21T03:16:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca4003850 con 0x55c3cd6750a0 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 20 client.15276 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd69df60 0x55c3cdad0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 4 client.15276 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "status"}) -- 0x55c3cd9b9610 con 0x55c3cd69df60 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.562+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd69df60 0x55c3cdad0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4c99ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cb9b52640 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd69df60 0x55c3cdad0790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=240010f7f0b477ea server_cookie=f467fafa5b8088bf in_seq=0 out_seq=0 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4c99ffb640 1 -- 172.21.15.83:0/1766067166 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c3cd9b9610 con 0x55c3cd69df60 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4c99ffb640 10 client.15276 handle_command_reply: tid=0 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 client.15276 shutdown 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 msgr2=0x55c3cda6aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4031440 tx=0x7f4ca4004370 comp rx=0 tx=0).stop 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 shutdown_connections 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3cd69df60 0x55c3cdad0790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3cd67d280 0x55c3cda63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3cd6750a0 0x55c3cda6aa00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 --2- 172.21.15.83:0/1766067166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3cd65c8e0 0x55c3cda6a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4c99ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 >> 172.21.15.83:0/1766067166 conn(0x55c3cd692350 msgr2=0x55c3cda4a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 shutdown_connections 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 1 -- 172.21.15.83:0/1766067166 wait complete. 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.563+0000 7f4cc2b2b740 20 client.15276 trim_cache size 0 max 16384 2024-09-21T03:16:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24620, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 81.729116309999995, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 111.142822157 2024-09-21T03:16:07.572 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 msgr2=0x7f4c9c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 0x7f4c9c0b8aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0e56e0 tx=0x7f4cac0de4a0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 msgr2=0x7f4cb419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00cce0 tx=0x7f4cac00a620 comp rx=0 tx=0).stop 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 shutdown_connections 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4c9c0b6590 0x7f4c9c0b8aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb4108d70 0x7f4cb419fd50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb4108420 0x7f4cb419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 --2- 172.21.15.83:0/1816020029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cb4102ae0 0x7f4cb419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 >> 172.21.15.83:0/1816020029 conn(0x7f4cb4079770 msgr2=0x7f4cb4101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 shutdown_connections 2024-09-21T03:16:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:07.564+0000 7f4cbc5b7640 1 -- 172.21.15.83:0/1816020029 wait complete. 2024-09-21T03:16:08.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:16:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:08 smithi193 ceph-mon[35224]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:08.597 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:08 smithi083 ceph-mon[31219]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:08 smithi142 ceph-mon[34512]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 -- 172.21.15.83:0/3843031015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 msgr2=0x7fec64110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/3843031015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec64110620 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009ca0 tx=0x7fec5c02a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 -- 172.21.15.83:0/3843031015 shutdown_connections 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/3843031015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec64110620 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/3843031015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec641094a0 0x7fec641098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/3843031015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec64103b60 0x7fec64103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 -- 172.21.15.83:0/3843031015 >> 172.21.15.83:0/3843031015 conn(0x7fec6407b850 msgr2=0x7fec6407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 -- 172.21.15.83:0/3843031015 shutdown_connections 2024-09-21T03:16:09.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.074+0000 7fec6b3b7640 1 -- 172.21.15.83:0/3843031015 wait complete. 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 Processor -- start 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 -- start start 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 0x7fec6419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec641094a0 0x7fec6419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec64113af0 con 0x7fec64109df0 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec64113970 con 0x7fec641094a0 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec6b3b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec64113c70 con 0x7fec64103b60 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.075+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48644/0 (socket says 172.21.15.83:48644) 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 -- 172.21.15.83:0/2489633827 learned_addr learned my addr 172.21.15.83:0/2489633827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:09.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec68952640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec641094a0 0x7fec6419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69153640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 0x7fec6419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 -- 172.21.15.83:0/2489633827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 msgr2=0x7fec6419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 0x7fec6419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 -- 172.21.15.83:0/2489633827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec641094a0 msgr2=0x7fec6419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec641094a0 0x7fec6419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 -- 172.21.15.83:0/2489633827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641a0390 con 0x7fec64109df0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69153640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 0x7fec6419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec69954640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec5c005cc0 tx=0x7fec5c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec5a7fc640 1 -- 172.21.15.83:0/2489633827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004040 con 0x7fec64109df0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec5a7fc640 1 -- 172.21.15.83:0/2489633827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c004690 con 0x7fec64109df0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec641a0620 con 0x7fec64109df0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec5a7fc640 1 -- 172.21.15.83:0/2489633827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c030e70 con 0x7fec64109df0 2024-09-21T03:16:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.076+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a0b60 con 0x7fec64109df0 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.079+0000 7fec7192b740 1 Processor -- start 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec5a7fc640 1 -- 172.21.15.83:0/2489633827 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fec5c0305e0 con 0x7fec64109df0 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 -- start start 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cfead1c10 con 0x561cfe78b0a0 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cfeac1040 con 0x561cfe7b3f60 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec7192b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cfeac4e50 con 0x561cfe793280 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48652/0 (socket says 172.21.15.83:48652) 2024-09-21T03:16:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 -- 172.21.15.83:0/2259816185 learned_addr learned my addr 172.21.15.83:0/2259816185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 -- 172.21.15.83:0/2259816185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 msgr2=0x561cfeb5a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 -- 172.21.15.83:0/2259816185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 msgr2=0x561cfeb59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec68952640 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb59f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec69153640 1 -- 172.21.15.83:0/2259816185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfe81f540 con 0x561cfe78b0a0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.080+0000 7fec5a7fc640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 0x7fec500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec68952640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 0x7fec500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec69153640 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fec6000bc20 tx=0x7fec6000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4985140ed9bc89b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec5a7fc640 1 -- 172.21.15.83:0/2489633827 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec5c108880 con 0x7fec64109df0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec4f7fe640 1 -- 172.21.15.83:0/2259816185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60017620 con 0x561cfe78b0a0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec4f7fe640 1 -- 172.21.15.83:0/2259816185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec60017c70 con 0x561cfe78b0a0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.081+0000 7fec4f7fe640 1 -- 172.21.15.83:0/2259816185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6000fc20 con 0x561cfe78b0a0 2024-09-21T03:16:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- 172.21.15.83:0/2259816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 msgr2=0x561cfeb5a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec68952640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 0x7fec500b8aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec54006fd0 tx=0x7fec54008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fec6000bc20 tx=0x7fec6000d710 comp rx=0 tx=0).stop 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- 172.21.15.83:0/2259816185 shutdown_connections 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb5a440 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 --2- 172.21.15.83:0/2259816185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- 172.21.15.83:0/2259816185 >> 172.21.15.83:0/2259816185 conn(0x561cfe7a8350 msgr2=0x561cfeb57df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- 172.21.15.83:0/2259816185 shutdown_connections 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- 172.21.15.83:0/2259816185 wait complete. 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 -- start start 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.082+0000 7fec7192b740 1 Processor -- start 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cfe9225f0 con 0x561cfe78b0a0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cfea5c010 con 0x561cfe7b3f60 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb686e0 con 0x561cfe793280 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec7192b740 10 client.0 _ll_register_callbacks cb 0x561cfeabb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48660/0 (socket says 172.21.15.83:48660) 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 -- 172.21.15.83:0/2174160974 learned_addr learned my addr 172.21.15.83:0/2174160974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 -- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 msgr2=0x561cfeb65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 -- 172.21.15.83:0/2174160974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 msgr2=0x561cfeb719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69954640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 -- 172.21.15.83:0/2174160974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfea89a00 con 0x561cfe78b0a0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec69153640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec6000f7c0 tx=0x7fec6000fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f69f4292c93ed80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.083+0000 7fec4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60026020 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec60017420 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec60017c70 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec60031050 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cfeb74570 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec60017c70 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fec60021940 con 0x561cfe78b0a0 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 10 client.15288 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 20 client.15288 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7728e0 0x561cfeb77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 4 client.15288 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cfead3b60 con 0x561cfe7728e0 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.084+0000 7fec68952640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7728e0 0x561cfeb77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.085+0000 7fec68952640 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7728e0 0x561cfeb77d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd3cc741ed8bc217 server_cookie=95a67c4cf3bdba4e in_seq=0 out_seq=0 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.085+0000 7fec4cff9640 10 client.15288 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec4cff9640 1 -- 172.21.15.83:0/2174160974 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561cfead3b60 con 0x561cfe7728e0 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec4cff9640 10 client.15288 handle_command_reply: tid=0 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 client.15288 shutdown 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 msgr2=0x561cfeb65780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec6000f7c0 tx=0x7fec6000fb50 comp rx=0 tx=0).stop 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 shutdown_connections 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7728e0 0x561cfeb77d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7b3f60 0x561cfeb719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe793280 0x561cfeb65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec4cff9640 0 client.15288 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 --2- 172.21.15.83:0/2174160974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe78b0a0 0x561cfeb65780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 >> 172.21.15.83:0/2174160974 conn(0x561cfe7a8350 msgr2=0x561cfeb5d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 shutdown_connections 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 1 -- 172.21.15.83:0/2174160974 wait complete. 2024-09-21T03:16:09.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.086+0000 7fec7192b740 20 client.15288 trim_cache size 0 max 16384 2024-09-21T03:16:09.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 Processor -- start 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 -- start start 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe7728e0 0x561cfeb60020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe78b0a0 0x561cfeb71940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb76080 con 0x561cfe793280 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb76750 con 0x561cfe78b0a0 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec7192b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb76e20 con 0x561cfe7728e0 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48668/0 (socket says 172.21.15.83:48668) 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.104+0000 7fec69954640 1 -- 172.21.15.83:0/4142619284 learned_addr learned my addr 172.21.15.83:0/4142619284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:09.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69153640 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe7728e0 0x561cfeb60020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec68952640 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe78b0a0 0x561cfeb71940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 -- 172.21.15.83:0/4142619284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe7728e0 msgr2=0x561cfeb60020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe7728e0 0x561cfeb60020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 -- 172.21.15.83:0/4142619284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe78b0a0 msgr2=0x561cfeb71940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe78b0a0 0x561cfeb71940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 -- 172.21.15.83:0/4142619284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfeb66a40 con 0x561cfe793280 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec69954640 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006010 tx=0x7fec5c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc9064ad056f11b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec4effd640 1 -- 172.21.15.83:0/4142619284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0098d0 con 0x561cfe793280 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec4effd640 1 -- 172.21.15.83:0/4142619284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c036040 con 0x561cfe793280 2024-09-21T03:16:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.105+0000 7fec4effd640 1 -- 172.21.15.83:0/4142619284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0d8390 con 0x561cfe793280 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- 172.21.15.83:0/4142619284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 msgr2=0x561cfeb71e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006010 tx=0x7fec5c02a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- 172.21.15.83:0/4142619284 shutdown_connections 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfeb71e80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe78b0a0 0x561cfeb71940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 --2- 172.21.15.83:0/4142619284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe7728e0 0x561cfeb60020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- 172.21.15.83:0/4142619284 >> 172.21.15.83:0/4142619284 conn(0x561cfe7a8350 msgr2=0x561cfeb64000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- 172.21.15.83:0/4142619284 shutdown_connections 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- 172.21.15.83:0/4142619284 wait complete. 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 -- start start 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.106+0000 7fec7192b740 1 Processor -- start 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 0x561cfeb7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe78b0a0 0x561cfeb7e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb77bc0 con 0x561cfe793280 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb774f0 con 0x561cfe7728e0 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cfeb78290 con 0x561cfe78b0a0 2024-09-21T03:16:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec7192b740 10 client.0 _ll_register_callbacks cb 0x7fec6000ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69153640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 0x561cfeb7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 -- 172.21.15.83:0/1135891296 learned_addr learned my addr 172.21.15.83:0/1135891296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec68952640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe78b0a0 0x561cfeb7e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 -- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe78b0a0 msgr2=0x561cfeb7e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe78b0a0 0x561cfeb7e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 -- 172.21.15.83:0/1135891296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 msgr2=0x561cfeb7e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 0x561cfeb7e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69954640 1 -- 172.21.15.83:0/1135891296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfebe6780 con 0x561cfe793280 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.107+0000 7fec69153640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 0x561cfeb7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec69954640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0cbbe0 tx=0x7fec5c0d3300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a446c127b1dfac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0043e0 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec5c004b40 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c036040 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec5c0098d0 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cfebe78d0 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c036040 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fec5c0f3030 con 0x561cfe793280 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 20 client.15294 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7b3f60 0x561cfebeafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 4 client.15294 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.108+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "status"}) -- 0x561cfeacf610 con 0x561cfe7b3f60 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec69153640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7b3f60 0x561cfebeafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec4cff9640 10 client.15294 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec69153640 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7b3f60 0x561cfebeafc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c7454ea01150af3 server_cookie=b444918f32514815 in_seq=0 out_seq=0 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec4cff9640 1 -- 172.21.15.83:0/1135891296 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561cfeacf610 con 0x561cfe7b3f60 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec4cff9640 10 client.15294 handle_command_reply: tid=0 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 client.15294 shutdown 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 msgr2=0x561cfebe6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0cbbe0 tx=0x7fec5c0d3300 comp rx=0 tx=0).stop 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 shutdown_connections 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x561cfe7b3f60 0x561cfebeafc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cfe793280 0x561cfebe6240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cfe78b0a0 0x561cfeb7e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 --2- 172.21.15.83:0/1135891296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfe7728e0 0x561cfeb7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.109+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 >> 172.21.15.83:0/1135891296 conn(0x561cfe7a8350 msgr2=0x561cfeb65530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec4cff9640 0 client.15294 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 shutdown_connections 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec7192b740 1 -- 172.21.15.83:0/1135891296 wait complete. 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec7192b740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-09-21T03:16:09.117 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24628, 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 84.283589090999996, 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 110.082104828 2024-09-21T03:16:09.118 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 msgr2=0x7fec500b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 0x7fec500b8aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec54006fd0 tx=0x7fec54008040 comp rx=0 tx=0).stop 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 msgr2=0x7fec6419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec5c005cc0 tx=0x7fec5c003ba0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 shutdown_connections 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec500b6590 0x7fec500b8aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec64109df0 0x7fec6419fd20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec641094a0 0x7fec6419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 --2- 172.21.15.83:0/2489633827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec64103b60 0x7fec6419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.110+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 >> 172.21.15.83:0/2489633827 conn(0x7fec6407b850 msgr2=0x7fec64107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.111+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 shutdown_connections 2024-09-21T03:16:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:09.111+0000 7fec6b3b7640 1 -- 172.21.15.83:0/2489633827 wait complete. 2024-09-21T03:16:09.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:16:10.191 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:10 smithi193 ceph-mon[35224]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:10.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:10 smithi083 ceph-mon[31219]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 -- 172.21.15.83:0/1495219992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb01094a0 msgr2=0x7facb01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 --2- 172.21.15.83:0/1495219992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb01094a0 0x7facb01098b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 -- 172.21.15.83:0/1495219992 shutdown_connections 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 --2- 172.21.15.83:0/1495219992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0109df0 0x7facb0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 --2- 172.21.15.83:0/1495219992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb01094a0 0x7facb01098b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 --2- 172.21.15.83:0/1495219992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb0103b60 0x7facb0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 -- 172.21.15.83:0/1495219992 >> 172.21.15.83:0/1495219992 conn(0x7facb007b850 msgr2=0x7facb007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 -- 172.21.15.83:0/1495219992 shutdown_connections 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.661+0000 7facb765d640 1 -- 172.21.15.83:0/1495219992 wait complete. 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.662+0000 7facb765d640 1 Processor -- start 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.662+0000 7facb765d640 1 -- start start 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0103b60 0x7facb019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb01094a0 0x7facb019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facb0113b50 con 0x7facb0109df0 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7facb01139d0 con 0x7facb01094a0 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb765d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7facb0113cd0 con 0x7facb0103b60 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47574/0 (socket says 172.21.15.83:47574) 2024-09-21T03:16:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 -- 172.21.15.83:0/2211024248 learned_addr learned my addr 172.21.15.83:0/2211024248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb4bf8640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb01094a0 0x7facb019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb53f9640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0103b60 0x7facb019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 -- 172.21.15.83:0/2211024248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0103b60 msgr2=0x7facb019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0103b60 0x7facb019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 -- 172.21.15.83:0/2211024248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb01094a0 msgr2=0x7facb019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb01094a0 0x7facb019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 -- 172.21.15.83:0/2211024248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb01a0630 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.663+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faca800b970 tx=0x7faca800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.664+0000 7faca67fc640 1 -- 172.21.15.83:0/2211024248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8019070 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.664+0000 7faca67fc640 1 -- 172.21.15.83:0/2211024248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8014440 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.664+0000 7faca67fc640 1 -- 172.21.15.83:0/2211024248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80075d0 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.664+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb01a0920 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.664+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb01a7080 con 0x7facb0109df0 2024-09-21T03:16:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.666+0000 7facbdbd1740 1 Processor -- start 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.666+0000 7faca67fc640 1 -- 172.21.15.83:0/2211024248 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7faca801c070 con 0x7facb0109df0 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 -- start start 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b887f60 0x556e6bc2df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc2e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e6bba5c10 con 0x556e6b867280 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb95040 con 0x556e6b887f60 2024-09-21T03:16:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facbdbd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb98e50 con 0x556e6b85f0a0 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47590/0 (socket says 172.21.15.83:47590) 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 -- 172.21.15.83:0/1398758262 learned_addr learned my addr 172.21.15.83:0/1398758262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b887f60 0x556e6bc2df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb53f9640 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc2e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 -- 172.21.15.83:0/1398758262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 msgr2=0x556e6bc2e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc2e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 -- 172.21.15.83:0/1398758262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b887f60 msgr2=0x556e6bc2df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b887f60 0x556e6bc2df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.667+0000 7facb4bf8640 1 -- 172.21.15.83:0/1398758262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6b8f3540 con 0x556e6b867280 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7faca67fc640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 0x7fac9c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7facb4bf8640 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facac000c00 tx=0x7facac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645e7992db86cb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7facb53f9640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 0x7fac9c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7fac9b7fe640 1 -- 172.21.15.83:0/1398758262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035040 con 0x556e6b867280 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7faca67fc640 1 -- 172.21.15.83:0/2211024248 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7faca80e6900 con 0x7facb0109df0 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7fac9b7fe640 1 -- 172.21.15.83:0/1398758262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facac005020 con 0x556e6b867280 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.668+0000 7fac9b7fe640 1 -- 172.21.15.83:0/1398758262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0044e0 con 0x556e6b867280 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- 172.21.15.83:0/1398758262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 msgr2=0x556e6bc2e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facb53f9640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 0x7fac9c0b8af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x556e6bc2cb80 tx=0x7faca0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facac000c00 tx=0x7facac009680 comp rx=0 tx=0).stop 2024-09-21T03:16:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- 172.21.15.83:0/1398758262 shutdown_connections 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc2e980 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc2e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 --2- 172.21.15.83:0/1398758262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b887f60 0x556e6bc2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- 172.21.15.83:0/1398758262 >> 172.21.15.83:0/1398758262 conn(0x556e6b87c350 msgr2=0x556e6bc2bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- 172.21.15.83:0/1398758262 shutdown_connections 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- 172.21.15.83:0/1398758262 wait complete. 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 -- start start 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.669+0000 7facbdbd1740 1 Processor -- start 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc45970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e6b9f65f0 con 0x556e6b85f0a0 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e6bb30010 con 0x556e6b867280 2024-09-21T03:16:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc3c680 con 0x556e6b887f60 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facbdbd1740 10 client.0 _ll_register_callbacks cb 0x556e6bb8f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47592/0 (socket says 172.21.15.83:47592) 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 -- 172.21.15.83:0/3597940234 learned_addr learned my addr 172.21.15.83:0/3597940234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb5bfa640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc45970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 -- 172.21.15.83:0/3597940234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 msgr2=0x556e6bc45970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc45970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 -- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 msgr2=0x556e6bc39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 -- 172.21.15.83:0/3597940234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6bb5da00 con 0x556e6b85f0a0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7facb53f9640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faca00146e0 tx=0x7faca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dce005447b8189f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.670+0000 7fac98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca001b050 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7faca00158e0 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facac005020 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faca0015ae0 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e6bc48620 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facac005020 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7faca002a430 con 0x556e6b85f0a0 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 20 client.15306 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b8468e0 0x556e6bc4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 4 client.15306 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:10.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.671+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6bba7b60 con 0x556e6b8468e0 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.672+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b8468e0 0x556e6bc4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.672+0000 7fac98ff9640 10 client.15306 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.672+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b8468e0 0x556e6bc4c100 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cfe8f990526651 server_cookie=bb80badb12d72379 in_seq=0 out_seq=0 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7fac98ff9640 1 -- 172.21.15.83:0/3597940234 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e6bba7b60 con 0x556e6b8468e0 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7fac98ff9640 10 client.15306 handle_command_reply: tid=0 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 client.15306 shutdown 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 msgr2=0x556e6bc399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faca00146e0 tx=0x7faca00046c0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 shutdown_connections 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b8468e0 0x556e6bc4c100 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc45970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3597940234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b85f0a0 0x556e6bc399e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 >> 172.21.15.83:0/3597940234 conn(0x556e6b87c350 msgr2=0x556e6bc310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7fac98ff9640 0 client.15306 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.673+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 shutdown_connections 2024-09-21T03:16:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.674+0000 7facbdbd1740 1 -- 172.21.15.83:0/3597940234 wait complete. 2024-09-21T03:16:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.674+0000 7facbdbd1740 20 client.15306 trim_cache size 0 max 16384 2024-09-21T03:16:10.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:10 smithi142 ceph-mon[34512]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.691+0000 7facbdbd1740 1 Processor -- start 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.691+0000 7facbdbd1740 1 -- start start 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b85f0a0 0x556e6bc38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc34f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4a440 con 0x556e6b867280 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4ab10 con 0x556e6b85f0a0 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facbdbd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4b1e0 con 0x556e6b887f60 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47596/0 (socket says 172.21.15.83:47596) 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 -- 172.21.15.83:0/3710039910 learned_addr learned my addr 172.21.15.83:0/3710039910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb53f9640 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b85f0a0 0x556e6bc38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 -- 172.21.15.83:0/3710039910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 msgr2=0x556e6bc34f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc34f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 -- 172.21.15.83:0/3710039910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b85f0a0 msgr2=0x556e6bc38cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b85f0a0 0x556e6bc38cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 -- 172.21.15.83:0/3710039910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6bc26080 con 0x556e6b867280 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7facb4bf8640 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ce199d26af75cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7fac9affd640 1 -- 172.21.15.83:0/3710039910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035040 con 0x556e6b867280 2024-09-21T03:16:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7fac9affd640 1 -- 172.21.15.83:0/3710039910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facac0054d0 con 0x556e6b867280 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.692+0000 7fac9affd640 1 -- 172.21.15.83:0/3710039910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac03d480 con 0x556e6b867280 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 -- 172.21.15.83:0/3710039910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 msgr2=0x556e6bc349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac005fd0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 -- 172.21.15.83:0/3710039910 shutdown_connections 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b887f60 0x556e6bc34f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b867280 0x556e6bc349c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 --2- 172.21.15.83:0/3710039910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b85f0a0 0x556e6bc38cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 -- 172.21.15.83:0/3710039910 >> 172.21.15.83:0/3710039910 conn(0x556e6b87c350 msgr2=0x556e6bc45af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 -- 172.21.15.83:0/3710039910 shutdown_connections 2024-09-21T03:16:10.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.693+0000 7facbdbd1740 1 -- 172.21.15.83:0/3710039910 wait complete. 2024-09-21T03:16:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 -- start start 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 Processor -- start 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4bf80 con 0x556e6b887f60 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4b8b0 con 0x556e6b867280 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e6bc4c650 con 0x556e6b85f0a0 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facbdbd1740 10 client.0 _ll_register_callbacks cb 0x7faca000b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facb5bfa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facb5bfa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47612/0 (socket says 172.21.15.83:47612) 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facb5bfa640 1 -- 172.21.15.83:0/2615425163 learned_addr learned my addr 172.21.15.83:0/2615425163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facb4bf8640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.694+0000 7facb53f9640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 -- 172.21.15.83:0/2615425163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 msgr2=0x556e6bc4e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc4e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 -- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 msgr2=0x556e6bc4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc4ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 -- 172.21.15.83:0/2615425163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6bc53a30 con 0x556e6b887f60 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb4bf8640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facb5bfa640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faca800b940 tx=0x7faca80ac160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5313e458b19aa6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7fac98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80b1270 con 0x556e6b887f60 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facac0054d0 con 0x556e6b887f60 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7faca800cb40 con 0x556e6b887f60 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faca80ad0f0 con 0x556e6b887f60 2024-09-21T03:16:10.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facbdbd1740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e6bc54d60 con 0x556e6b887f60 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.695+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8096030 con 0x556e6b887f60 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7faca80c2c30 con 0x556e6b887f60 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 20 client.15312 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b81bc90 0x556e6bcba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 4 client.15312 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "status"}) -- 0x556e6bba3610 con 0x556e6b81bc90 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facb53f9640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b81bc90 0x556e6bcba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7fac98ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.696+0000 7facb53f9640 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b81bc90 0x556e6bcba890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec27129641d53b3 server_cookie=83a1615a21798494 in_seq=0 out_seq=0 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7fac98ff9640 1 -- 172.21.15.83:0/2615425163 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556e6bba3610 con 0x556e6b81bc90 2024-09-21T03:16:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7fac98ff9640 10 client.15312 handle_command_reply: tid=0 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 client.15312 shutdown 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 msgr2=0x556e6bc4f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faca800b940 tx=0x7faca80ac160 comp rx=0 tx=0).stop 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 shutdown_connections 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556e6b81bc90 0x556e6bcba890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e6b887f60 0x556e6bc4f360 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e6b867280 0x556e6bc4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7fac98ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 --2- 172.21.15.83:0/2615425163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6b85f0a0 0x556e6bc4e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 >> 172.21.15.83:0/2615425163 conn(0x556e6b87c350 msgr2=0x556e6bc349c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 shutdown_connections 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 1 -- 172.21.15.83:0/2615425163 wait complete. 2024-09-21T03:16:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.697+0000 7facbdbd1740 20 client.15312 trim_cache size 0 max 16384 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24647, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 93.099496017000007, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:10.705 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 102.465824767 2024-09-21T03:16:10.706 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 msgr2=0x7fac9c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 0x7fac9c0b8af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x556e6bc2cb80 tx=0x7faca0008040 comp rx=0 tx=0).stop 2024-09-21T03:16:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 msgr2=0x7facb019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faca800b970 tx=0x7faca800be30 comp rx=0 tx=0).stop 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 shutdown_connections 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fac9c0b65e0 0x7fac9c0b8af0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb0109df0 0x7facb019ff00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb01094a0 0x7facb019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 --2- 172.21.15.83:0/2211024248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb0103b60 0x7facb019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 >> 172.21.15.83:0/2211024248 conn(0x7facb007b850 msgr2=0x7facb01076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 shutdown_connections 2024-09-21T03:16:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:10.698+0000 7facb765d640 1 -- 172.21.15.83:0/2211024248 wait complete. 2024-09-21T03:16:11.491 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:16:11.491 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:16:11.499 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:16:11.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:16:11.744 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:12.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 -- 172.21.15.83:0/4121869775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8109df0 msgr2=0x7f99d8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:12.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 --2- 172.21.15.83:0/4121869775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8109df0 0x7f99d8110620 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009ca0 tx=0x7f99d002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 -- 172.21.15.83:0/4121869775 shutdown_connections 2024-09-21T03:16:12.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 --2- 172.21.15.83:0/4121869775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d8109df0 0x7f99d8110620 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 --2- 172.21.15.83:0/4121869775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d81094a0 0x7f99d81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 --2- 172.21.15.83:0/4121869775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 0x7f99d8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 -- 172.21.15.83:0/4121869775 >> 172.21.15.83:0/4121869775 conn(0x7f99d807b850 msgr2=0x7f99d807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 -- 172.21.15.83:0/4121869775 shutdown_connections 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 -- 172.21.15.83:0/4121869775 wait complete. 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.211+0000 7f99dec15640 1 Processor -- start 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 -- start start 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 0x7f99d819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:12.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d8109df0 0x7f99d819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113b70 con 0x7f99d81094a0 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81139f0 con 0x7f99d8103b60 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dec15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113cf0 con 0x7f99d8109df0 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47638/0 (socket says 172.21.15.83:47638) 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 -- 172.21.15.83:0/2829482047 learned_addr learned my addr 172.21.15.83:0/2829482047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dc9b1640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 0x7f99d819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99dd1b2640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d8109df0 0x7f99d819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 -- 172.21.15.83:0/2829482047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d8109df0 msgr2=0x7f99d819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d8109df0 0x7f99d819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 -- 172.21.15.83:0/2829482047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 msgr2=0x7f99d819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 0x7f99d819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.212+0000 7f99d7fff640 1 -- 172.21.15.83:0/2829482047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81a0610 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99d7fff640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99c4006fd0 tx=0x7f99c400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4015020 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99c4011ce0 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4010d90 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81a0900 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.213+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81a7060 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.214+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d8103ff0 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.215+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f99c4018020 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.216+0000 7f99d5ffb640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 0x7f99c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.216+0000 7f99dc9b1640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 0x7f99c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.216+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f99c40e6c00 con 0x7f99d81094a0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.216+0000 7f99dc9b1640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 0x7f99c80b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009990 tx=0x7f99cc009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.220+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99c40ed080 con 0x7f99d81094a0 2024-09-21T03:16:12.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.418+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f99d810bc50 con 0x7f99c80b6560 2024-09-21T03:16:12.420 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:12 smithi083 ceph-mon[31219]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:12.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.419+0000 7f99d5ffb640 1 -- 172.21.15.83:0/2829482047 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f99cc00e850 con 0x7f99c80b6560 2024-09-21T03:16:12.421 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-09-21T03:16:12.421 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-09-21T03:16:12.421 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-09-21T03:16:12.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 msgr2=0x7f99c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:12.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 0x7f99c80b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009990 tx=0x7f99cc009110 comp rx=0 tx=0).stop 2024-09-21T03:16:12.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 msgr2=0x7f99d819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:12.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99c4006fd0 tx=0x7f99c400b820 comp rx=0 tx=0).stop 2024-09-21T03:16:12.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 shutdown_connections 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.422+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99c80b6560 0x7f99c80b8a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99d8109df0 0x7f99d819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d81094a0 0x7f99d819ac40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 --2- 172.21.15.83:0/2829482047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99d8103b60 0x7f99d819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 >> 172.21.15.83:0/2829482047 conn(0x7f99d807b850 msgr2=0x7f99d8107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 shutdown_connections 2024-09-21T03:16:12.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:12.423+0000 7f99dec15640 1 -- 172.21.15.83:0/2829482047 wait complete. 2024-09-21T03:16:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:12 smithi193 ceph-mon[35224]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:12 smithi142 ceph-mon[34512]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T03:16:13.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:16:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 -- 172.21.15.83:0/867601392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac01083a0 msgr2=0x7f6ac01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 --2- 172.21.15.83:0/867601392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac01083a0 0x7f6ac01087b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009900 tx=0x7f6ab402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 -- 172.21.15.83:0/867601392 shutdown_connections 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 --2- 172.21.15.83:0/867601392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0108cf0 0x7f6ac010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 --2- 172.21.15.83:0/867601392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac01083a0 0x7f6ac01087b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 --2- 172.21.15.83:0/867601392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac0102a60 0x7f6ac0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 -- 172.21.15.83:0/867601392 >> 172.21.15.83:0/867601392 conn(0x7f6ac0079730 msgr2=0x7f6ac0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 -- 172.21.15.83:0/867601392 shutdown_connections 2024-09-21T03:16:13.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.852+0000 7f6ac8718640 1 -- 172.21.15.83:0/867601392 wait complete. 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 Processor -- start 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 -- start start 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 0x7f6ac019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 0x7f6ac019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0113ac0 con 0x7f6ac0108cf0 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0113940 con 0x7f6ac01083a0 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac8718640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0113c40 con 0x7f6ac0102a60 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47654/0 (socket says 172.21.15.83:47654) 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 -- 172.21.15.83:0/2158091129 learned_addr learned my addr 172.21.15.83:0/2158091129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac5cb3640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 0x7f6ac019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:13.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 -- 172.21.15.83:0/2158091129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 msgr2=0x7f6ac019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac64b4640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 0x7f6ac019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 0x7f6ac019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 -- 172.21.15.83:0/2158091129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 msgr2=0x7f6ac019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 0x7f6ac019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 -- 172.21.15.83:0/2158091129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac01a0540 con 0x7f6ac0108cf0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac5cb3640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 0x7f6ac019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac64b4640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 0x7f6ac019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.853+0000 7f6ac6cb5640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000cad0 tx=0x7f6ab000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.854+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000ae20 con 0x7f6ac0108cf0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.854+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac01a0830 con 0x7f6ac0108cf0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.854+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab0005020 con 0x7f6ac0108cf0 2024-09-21T03:16:13.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.854+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000f6b0 con 0x7f6ac0108cf0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.854+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac01a6f90 con 0x7f6ac0108cf0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.855+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac0102e70 con 0x7f6ac0108cf0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.856+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f6ab00051d0 con 0x7f6ac0108cf0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.857+0000 7f6abf7fe640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 0x7f6aa80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.857+0000 7f6ac64b4640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 0x7f6aa80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.857+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6ab00e7920 con 0x7f6ac0108cf0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.858+0000 7f6ac64b4640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 0x7f6aa80b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006fd0 tx=0x7f6ab8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:13.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:13.861+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ab0014030 con 0x7f6ac0108cf0 2024-09-21T03:16:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.048+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6ac0063800 con 0x7f6aa80b6560 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.060+0000 7f6abf7fe640 1 -- 172.21.15.83:0/2158091129 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6ac0063800 con 0x7f6aa80b6560 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (109s) 21s ago 108s 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 d57899cc9133 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi142 running (106s) 39s ago 105s 94.7M - 19.3.0-5120-g18432218 ddce307e1f32 144994b3b26d 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi193 running (2m) 39s ago 2m 88.2M - 19.3.0-5120-g18432218 ddce307e1f32 34f6a2854465 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (2m) 21s ago 2m 82.5M - 19.3.0-5120-g18432218 ddce307e1f32 41475ed58ad8 2024-09-21T03:16:14.062 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi142 running (2m) 39s ago 2m 85.5M - 19.3.0-5120-g18432218 ddce307e1f32 fe46fd12cb07 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi193 running (2m) 39s ago 2m 85.6M - 19.3.0-5120-g18432218 ddce307e1f32 0568e50348aa 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (112s) 21s ago 111s 87.2M - 19.3.0-5120-g18432218 ddce307e1f32 e2060f161d42 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi142 running (117s) 39s ago 117s 85.7M - 19.3.0-5120-g18432218 ddce307e1f32 6348f3961e6e 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi193 running (115s) 39s ago 114s 84.9M - 19.3.0-5120-g18432218 ddce307e1f32 507f099162a6 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (2m) 21s ago 2m 84.3M - 19.3.0-5120-g18432218 ddce307e1f32 3ac39b45586d 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi142 running (2m) 39s ago 2m 84.8M - 19.3.0-5120-g18432218 ddce307e1f32 ca0a5e702f66 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi193 running (2m) 39s ago 2m 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 7e2a7e1d8a23 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (9m) 21s ago 9m 585M - 19.3.0-5120-g18432218 ddce307e1f32 f7ed1f718536 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi142 *:8443,8765 running (8m) 39s ago 8m 523M - 19.3.0-5120-g18432218 ddce307e1f32 d3572e5ba617 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi193 *:8443,8765 running (8m) 39s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 310d420b04b9 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (9m) 21s ago 10m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 9561cf4f5431 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi142 running (8m) 39s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 c3a7274f2e34 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi193 running (8m) 39s ago 8m 108M 2048M 19.3.0-5120-g18432218 ddce307e1f32 e19f79293868 2024-09-21T03:16:14.063 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (7m) 21s ago 7m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 7112a04ac180 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi142 running (6m) 39s ago 6m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 03cebe8bf29d 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi193 running (6m) 39s ago 6m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3da7ea20fb52 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (6m) 21s ago 6m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ef482d744698 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi142 running (5m) 39s ago 5m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6368d8d0c4a3 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi193 running (5m) 39s ago 5m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 40cc946c838e 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (4m) 21s ago 4m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 919c38ab6280 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi142 running (4m) 39s ago 4m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 d4d56a66bb0e 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi193 running (3m) 39s ago 3m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 390d18c296bb 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (3m) 21s ago 3m 118M 4272M 19.3.0-5120-g18432218 ddce307e1f32 620715511de6 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi142 running (3m) 39s ago 3m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 7baf4d65d926 2024-09-21T03:16:14.064 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi193 running (2m) 39s ago 2m 108M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6402f5d9319d 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 msgr2=0x7f6aa80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 0x7f6aa80b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006fd0 tx=0x7f6ab8008040 comp rx=0 tx=0).stop 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 msgr2=0x7f6ac019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000cad0 tx=0x7f6ab000cf90 comp rx=0 tx=0).stop 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 shutdown_connections 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6aa80b6560 0x7f6aa80b8a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ac0108cf0 0x7f6ac019fe10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:14.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac01083a0 0x7f6ac019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 --2- 172.21.15.83:0/2158091129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ac0102a60 0x7f6ac019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 >> 172.21.15.83:0/2158091129 conn(0x7f6ac0079730 msgr2=0x7f6ac0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 shutdown_connections 2024-09-21T03:16:14.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:14.063+0000 7f6ac8718640 1 -- 172.21.15.83:0/2158091129 wait complete. 2024-09-21T03:16:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:14 smithi083 ceph-mon[31219]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:14.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:14 smithi083 ceph-mon[31219]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:14.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:14 smithi193 ceph-mon[35224]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:14 smithi193 ceph-mon[35224]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:14 smithi142 ceph-mon[34512]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:14 smithi142 ceph-mon[34512]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-21T03:16:14.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:16:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:15.470 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:15 smithi083 ceph-mon[31219]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:15.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 -- 172.21.15.83:0/3999140804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c109ec0 msgr2=0x7f015c1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 --2- 172.21.15.83:0/3999140804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c109ec0 0x7f015c1106f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0154009ca0 tx=0x7f015402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 -- 172.21.15.83:0/3999140804 shutdown_connections 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 --2- 172.21.15.83:0/3999140804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c109ec0 0x7f015c1106f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 --2- 172.21.15.83:0/3999140804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 0x7f015c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 --2- 172.21.15.83:0/3999140804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c103c30 0x7f015c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 -- 172.21.15.83:0/3999140804 >> 172.21.15.83:0/3999140804 conn(0x7f015c07b8d0 msgr2=0x7f015c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 -- 172.21.15.83:0/3999140804 shutdown_connections 2024-09-21T03:16:15.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.560+0000 7f016298a640 1 -- 172.21.15.83:0/3999140804 wait complete. 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 Processor -- start 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 -- start start 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 0x7f015c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c109ec0 0x7f015c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f015c113ba0 con 0x7f015c103c30 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f015c113a20 con 0x7f015c109570 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f016298a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f015c113d20 con 0x7f015c109ec0 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47682/0 (socket says 172.21.15.83:47682) 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 -- 172.21.15.83:0/3098621904 learned_addr learned my addr 172.21.15.83:0/3098621904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f0153fff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 0x7f015c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f0160f27640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c109ec0 0x7f015c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 -- 172.21.15.83:0/3098621904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c109ec0 msgr2=0x7f015c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c109ec0 0x7f015c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 -- 172.21.15.83:0/3098621904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 msgr2=0x7f015c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.561+0000 7f015bfff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 0x7f015c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f015bfff640 1 -- 172.21.15.83:0/3098621904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c1a8890 con 0x7f015c103c30 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f015bfff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f014c00b4a0 tx=0x7f014c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c015020 con 0x7f015c103c30 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f014c011ce0 con 0x7f015c103c30 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c010dc0 con 0x7f015c103c30 2024-09-21T03:16:15.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c1a8b80 con 0x7f015c103c30 2024-09-21T03:16:15.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.562+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c1a9070 con 0x7f015c103c30 2024-09-21T03:16:15.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:15 smithi193 ceph-mon[35224]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.563+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c1040c0 con 0x7f015c103c30 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.564+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f014c018020 con 0x7f015c103c30 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.565+0000 7f0159ffb640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 0x7f01480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.565+0000 7f0153fff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 0x7f01480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.565+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f014c0e71c0 con 0x7f015c103c30 2024-09-21T03:16:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.566+0000 7f0153fff640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 0x7f01480b8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f015c106520 tx=0x7f0144009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:15.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.568+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f014c01f020 con 0x7f015c103c30 2024-09-21T03:16:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:15 smithi142 ceph-mon[34512]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.754+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f015c063800 con 0x7f01480b6560 2024-09-21T03:16:15.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.760+0000 7f0159ffb640 1 -- 172.21.15.83:0/3098621904 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f015c063800 con 0x7f01480b6560 2024-09-21T03:16:15.762 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:16:15.762 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 41s ago 2m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T03:16:15.762 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 41s ago 8m smithi083=x;smithi142=y;smithi193=z;count:3 2024-09-21T03:16:15.762 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 41s ago 8m smithi083:172.21.15.83=a;smithi142:172.21.15.142=b;smithi193:172.21.15.193=c;count:3 2024-09-21T03:16:15.762 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 41s ago - 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 msgr2=0x7f01480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 0x7f01480b8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f015c106520 tx=0x7f0144009110 comp rx=0 tx=0).stop 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 msgr2=0x7f015c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f014c00b4a0 tx=0x7f014c00b8a0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 shutdown_connections 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01480b6560 0x7f01480b8a70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015c109ec0 0x7f015c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c109570 0x7f015c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 --2- 172.21.15.83:0/3098621904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f015c103c30 0x7f015c075710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.762+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 >> 172.21.15.83:0/3098621904 conn(0x7f015c07b8d0 msgr2=0x7f015c107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.763+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 shutdown_connections 2024-09-21T03:16:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:15.763+0000 7f016298a640 1 -- 172.21.15.83:0/3098621904 wait complete. 2024-09-21T03:16:16.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:16 smithi083 ceph-mon[31219]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:16.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:16 smithi083 ceph-mon[31219]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:16.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:16:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:16 smithi193 ceph-mon[35224]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:16 smithi193 ceph-mon[35224]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:16 smithi142 ceph-mon[34512]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:16 smithi142 ceph-mon[34512]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 -- 172.21.15.83:0/2647459563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 msgr2=0x7fc748102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/2647459563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc748102c80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc744009900 tx=0x7fc74402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 -- 172.21.15.83:0/2647459563 shutdown_connections 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/2647459563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc748108b00 0x7fc74810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/2647459563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7481081b0 0x7fc7481085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/2647459563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc748102c80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 -- 172.21.15.83:0/2647459563 >> 172.21.15.83:0/2647459563 conn(0x7fc7480fe2c0 msgr2=0x7fc748100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:17.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 -- 172.21.15.83:0/2647459563 shutdown_connections 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.242+0000 7fc74dd9f640 1 -- 172.21.15.83:0/2647459563 wait complete. 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 Processor -- start 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 -- start start 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 0x7fc74819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc748108b00 0x7fc74819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc748112820 con 0x7fc748102870 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc7481126a0 con 0x7fc7481081b0 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74dd9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7481129a0 con 0x7fc748108b00 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74cd9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74cd9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47696/0 (socket says 172.21.15.83:47696) 2024-09-21T03:16:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc74cd9d640 1 -- 172.21.15.83:0/3409616114 learned_addr learned my addr 172.21.15.83:0/3409616114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.243+0000 7fc73ffff640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 0x7fc74819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 -- 172.21.15.83:0/3409616114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc748108b00 msgr2=0x7fc74819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74d59e640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc748108b00 0x7fc74819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc748108b00 0x7fc74819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 -- 172.21.15.83:0/3409616114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 msgr2=0x7fc74819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 0x7fc74819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 -- 172.21.15.83:0/3409616114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7481a6350 con 0x7fc748102870 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc73ffff640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 0x7fc74819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74cd9d640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc744006010 tx=0x7fc7440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74403f030 con 0x7fc748102870 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc744003800 con 0x7fc748102870 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7481a65e0 con 0x7fc748102870 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744035070 con 0x7fc748102870 2024-09-21T03:16:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.244+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7481a6b40 con 0x7fc748102870 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.245+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc748102c80 con 0x7fc748102870 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.247+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fc744003cb0 con 0x7fc748102870 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.247+0000 7fc73dffb640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 0x7fc7300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.247+0000 7fc73ffff640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 0x7fc7300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.248+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc744108050 con 0x7fc748102870 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.248+0000 7fc73ffff640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 0x7fc7300b8aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc748107d60 tx=0x7fc73800a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.250+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7440ce840 con 0x7fc748102870 2024-09-21T03:16:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.438+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc748063800 con 0x7fc7300b6590 2024-09-21T03:16:17.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:17 smithi083 ceph-mon[31219]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.440+0000 7fc73dffb640 1 -- 172.21.15.83:0/3409616114 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc748063800 con 0x7fc7300b6590 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stdout:smithi142 172.21.15.142 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stdout:smithi193 172.21.15.193 2024-09-21T03:16:17.441 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 msgr2=0x7fc7300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 0x7fc7300b8aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc748107d60 tx=0x7fc73800a5c0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 msgr2=0x7fc74819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc744006010 tx=0x7fc7440042f0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 shutdown_connections 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc7300b6590 0x7fc7300b8aa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc748108b00 0x7fc74819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7481081b0 0x7fc74819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 --2- 172.21.15.83:0/3409616114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc748102870 0x7fc74819b150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 >> 172.21.15.83:0/3409616114 conn(0x7fc7480fe2c0 msgr2=0x7fc7481006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 shutdown_connections 2024-09-21T03:16:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:17.442+0000 7fc74dd9f640 1 -- 172.21.15.83:0/3409616114 wait complete. 2024-09-21T03:16:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:17 smithi193 ceph-mon[35224]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:17 smithi142 ceph-mon[34512]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:18.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:16:18.464 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:18 smithi083 ceph-mon[31219]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:18.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:18 smithi193 ceph-mon[35224]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:18.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:18 smithi142 ceph-mon[34512]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:18.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 -- 172.21.15.83:0/4066739407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd190107b80 msgr2=0x7fd190107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:18.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 --2- 172.21.15.83:0/4066739407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd190107b80 0x7fd190107f90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd178009900 tx=0x7fd17802a410 comp rx=0 tx=0).stop 2024-09-21T03:16:18.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 -- 172.21.15.83:0/4066739407 shutdown_connections 2024-09-21T03:16:18.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 --2- 172.21.15.83:0/4066739407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1901084d0 0x7fd19010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 --2- 172.21.15.83:0/4066739407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd190107b80 0x7fd190107f90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 --2- 172.21.15.83:0/4066739407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190102240 0x7fd190102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 -- 172.21.15.83:0/4066739407 >> 172.21.15.83:0/4066739407 conn(0x7fd190079f90 msgr2=0x7fd19007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 -- 172.21.15.83:0/4066739407 shutdown_connections 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.898+0000 7fd19641f640 1 -- 172.21.15.83:0/4066739407 wait complete. 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 Processor -- start 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 -- start start 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd190102240 0x7fd19019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190107b80 0x7fd19019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd190112160 con 0x7fd1901084d0 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd190111fe0 con 0x7fd190102240 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd19641f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1901122e0 con 0x7fd190107b80 2024-09-21T03:16:18.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd1949bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd1949bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47708/0 (socket says 172.21.15.83:47708) 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd1949bc640 1 -- 172.21.15.83:0/710814639 learned_addr learned my addr 172.21.15.83:0/710814639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.899+0000 7fd187fff640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190107b80 0x7fd19019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd18ffff640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd190102240 0x7fd19019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 -- 172.21.15.83:0/710814639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190107b80 msgr2=0x7fd19019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190107b80 0x7fd19019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 -- 172.21.15.83:0/710814639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd190102240 msgr2=0x7fd19019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd190102240 0x7fd19019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 -- 172.21.15.83:0/710814639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901a6600 con 0x7fd1901084d0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd1949bc640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd18800eaa0 tx=0x7fd18800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18800cdb0 con 0x7fd1901084d0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1901a68f0 con 0x7fd1901084d0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd188005020 con 0x7fd1901084d0 2024-09-21T03:16:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1880116d0 con 0x7fd1901084d0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.900+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a6e00 con 0x7fd1901084d0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.901+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd190102650 con 0x7fd1901084d0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.902+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fd1880051d0 con 0x7fd1901084d0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.903+0000 7fd18dffb640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 0x7fd17c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.903+0000 7fd18ffff640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 0x7fd17c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.903+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd1880e7a80 con 0x7fd1901084d0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.904+0000 7fd18ffff640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 0x7fd17c0b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd180005fd0 tx=0x7fd180005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:18.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:18.906+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd188016030 con 0x7fd1901084d0 2024-09-21T03:16:19.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.091+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd1901013f0 con 0x7fd17c0b6560 2024-09-21T03:16:19.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.093+0000 7fd18dffb640 1 -- 172.21.15.83:0/710814639 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd1901013f0 con 0x7fd17c0b6560 2024-09-21T03:16:19.095 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:16:19.095 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:16:19.095 INFO:teuthology.orchestra.run.smithi083.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:16:19.095 INFO:teuthology.orchestra.run.smithi083.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.096+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 msgr2=0x7fd17c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.096+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 0x7fd17c0b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd180005fd0 tx=0x7fd180005850 comp rx=0 tx=0).stop 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.096+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 msgr2=0x7fd19019bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.096+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd18800eaa0 tx=0x7fd18800ef60 comp rx=0 tx=0).stop 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.096+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 shutdown_connections 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd17c0b6560 0x7fd17c0b8a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1901084d0 0x7fd19019bd70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:19.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd190107b80 0x7fd19019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:19.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 --2- 172.21.15.83:0/710814639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd190102240 0x7fd19019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:19.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 >> 172.21.15.83:0/710814639 conn(0x7fd190079f90 msgr2=0x7fd190100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:19.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 shutdown_connections 2024-09-21T03:16:19.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:19.097+0000 7fd19641f640 1 -- 172.21.15.83:0/710814639 wait complete. 2024-09-21T03:16:19.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:19 smithi083 ceph-mon[31219]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:19.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:19 smithi083 ceph-mon[31219]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:19 smithi193 ceph-mon[35224]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:19 smithi193 ceph-mon[35224]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:19 smithi142 ceph-mon[34512]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:19 smithi142 ceph-mon[34512]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:19.810 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:16:19.820 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:16:19.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:16:20.073 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 -- 172.21.15.83:0/3825253874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df01083d0 msgr2=0x7f3df01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 --2- 172.21.15.83:0/3825253874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df01083d0 0x7f3df01087e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009900 tx=0x7f3de402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 -- 172.21.15.83:0/3825253874 shutdown_connections 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 --2- 172.21.15.83:0/3825253874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df0108d20 0x7f3df010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 --2- 172.21.15.83:0/3825253874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df01083d0 0x7f3df01087e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 --2- 172.21.15.83:0/3825253874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0102a90 0x7f3df0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 -- 172.21.15.83:0/3825253874 >> 172.21.15.83:0/3825253874 conn(0x7f3df0079750 msgr2=0x7f3df0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 -- 172.21.15.83:0/3825253874 shutdown_connections 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.564+0000 7f3df516b640 1 -- 172.21.15.83:0/3825253874 wait complete. 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 Processor -- start 2024-09-21T03:16:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 -- start start 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df01083d0 0x7f3df019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0108d20 0x7f3df019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01129d0 con 0x7f3df0102a90 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0112850 con 0x7f3df01083d0 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.565+0000 7f3df516b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0112b50 con 0x7f3df0108d20 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39060/0 (socket says 172.21.15.83:39060) 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 -- 172.21.15.83:0/3500206134 learned_addr learned my addr 172.21.15.83:0/3500206134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3def59e640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0108d20 0x7f3df019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3dee59c640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df01083d0 0x7f3df019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 -- 172.21.15.83:0/3500206134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0108d20 msgr2=0x7f3df019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0108d20 0x7f3df019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 -- 172.21.15.83:0/3500206134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df01083d0 msgr2=0x7f3df019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df01083d0 0x7f3df019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 -- 172.21.15.83:0/3500206134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a6510 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3deed9d640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c920 tx=0x7f3de000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0007c70 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a6800 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0007e10 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0018820 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.566+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a6d90 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.567+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df0103d10 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.569+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f3de000f450 con 0x7f3df0102a90 2024-09-21T03:16:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.569+0000 7f3dd7fff640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.570+0000 7f3dee59c640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.570+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3de00e78b0 con 0x7f3df0102a90 2024-09-21T03:16:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.570+0000 7f3dee59c640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 0x7f3dd80b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.572+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3de0014030 con 0x7f3df0102a90 2024-09-21T03:16:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.795+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3df0102ea0 con 0x7f3df0102a90 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.795+0000 7f3dd7fff640 1 -- 172.21.15.83:0/3500206134 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3134 (secure 0 0 0) 0x7f3df0102ea0 con 0x7f3df0102a90 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:e26 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:btime 2024-09-21T03:14:48:753858+0000 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:16:20.797 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 26 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-21T03:14:37.183732+0000 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-09-21T03:14:48.753854+0000 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-09-21T03:16:20.798 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 5 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2,3,4 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=24647,1=24628,2=24620,3=24613,4=24601} 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3] 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-09-21T03:16:20.799 INFO:teuthology.orchestra.run.smithi083.stdout:qdb_cluster leader: 24647 members: 24601,24613,24620,24628,24647 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:24647} state up:active seq 4 addr [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{0:24593} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/3055596594,v1:172.21.15.142:6835/3055596594] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{1:24628} state up:active seq 8 addr [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{1:24571} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/817680520,v1:172.21.15.193:6833/817680520] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{2:24620} state up:active seq 9 addr [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{2:24569} state up:standby-replay seq 1 addr [v2:172.21.15.142:6832/1409264490,v1:172.21.15.142:6833/1409264490] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{3:24613} state up:active seq 10 addr [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{3:14850} state up:standby-replay seq 1 addr [v2:172.21.15.83:6840/2540451962,v1:172.21.15.83:6841/2540451962] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{4:24601} state up:active seq 11 addr [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{4:14838} state up:standby-replay seq 1 addr [v2:172.21.15.83:6838/3046510766,v1:172.21.15.83:6839/3046510766] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:20.800 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14784} state up:standby seq 1 addr [v2:172.21.15.83:6834/3297642632,v1:172.21.15.83:6835/3297642632] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.801 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:14796} state up:standby seq 1 addr [v2:172.21.15.83:6836/3179576739,v1:172.21.15.83:6837/3179576739] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:16:20.801 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:16:20.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 msgr2=0x7f3dd80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:20.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 0x7f3dd80b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4031400 comp rx=0 tx=0).stop 2024-09-21T03:16:20.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 msgr2=0x7f3df019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c920 tx=0x7f3de000cde0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 shutdown_connections 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3dd80b6560 0x7f3dd80b8a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3df0108d20 0x7f3df019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3df01083d0 0x7f3df019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 --2- 172.21.15.83:0/3500206134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3df0102a90 0x7f3df019b2e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.797+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 >> 172.21.15.83:0/3500206134 conn(0x7f3df0079750 msgr2=0x7f3df01052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.798+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 shutdown_connections 2024-09-21T03:16:20.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:20.798+0000 7f3df516b640 1 -- 172.21.15.83:0/3500206134 wait complete. 2024-09-21T03:16:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3500206134' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:16:21.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3500206134' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:16:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3500206134' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:16:21.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:16:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:21 smithi083 ceph-mon[31219]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:22.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:21 smithi142 ceph-mon[34512]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:22.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:21 smithi193 ceph-mon[35224]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 -- 172.21.15.83:0/192824043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe904108d30 msgr2=0x7fe90410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 --2- 172.21.15.83:0/192824043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe904108d30 0x7fe90410f560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009ca0 tx=0x7fe8fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 -- 172.21.15.83:0/192824043 shutdown_connections 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 --2- 172.21.15.83:0/192824043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe904108d30 0x7fe90410f560 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 --2- 172.21.15.83:0/192824043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9041083e0 0x7fe9041087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 --2- 172.21.15.83:0/192824043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe904102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 -- 172.21.15.83:0/192824043 >> 172.21.15.83:0/192824043 conn(0x7fe904079750 msgr2=0x7fe904079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 -- 172.21.15.83:0/192824043 shutdown_connections 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.507+0000 7fe90b74a640 1 -- 172.21.15.83:0/192824043 wait complete. 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 Processor -- start 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 -- start start 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe90419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe904108d30 0x7fe90419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112a50 con 0x7fe9041083e0 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041128d0 con 0x7fe904102aa0 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe90b74a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112bd0 con 0x7fe904108d30 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe908ce5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe908ce5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39068/0 (socket says 172.21.15.83:39068) 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe908ce5640 1 -- 172.21.15.83:0/1297793120 learned_addr learned my addr 172.21.15.83:0/1297793120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe9094e6640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe90419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe909ce7640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe904108d30 0x7fe90419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.508+0000 7fe908ce5640 1 -- 172.21.15.83:0/1297793120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe904108d30 msgr2=0x7fe90419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe908ce5640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe904108d30 0x7fe90419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe908ce5640 1 -- 172.21.15.83:0/1297793120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 msgr2=0x7fe90419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe908ce5640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe90419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe908ce5640 1 -- 172.21.15.83:0/1297793120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9041a03a0 con 0x7fe9041083e0 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe9094e6640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe90419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe908ce5640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c8c0 tx=0x7fe90000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900007bb0 con 0x7fe9041083e0 2024-09-21T03:16:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9041a0690 con 0x7fe9041083e0 2024-09-21T03:16:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe900007d50 con 0x7fe9041083e0 2024-09-21T03:16:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9041a6eb0 con 0x7fe9041083e0 2024-09-21T03:16:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.509+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9000187c0 con 0x7fe9041083e0 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.510+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e8005290 con 0x7fe9041083e0 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.512+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fe900018960 con 0x7fe9041083e0 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.512+0000 7fe8fa7fc640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 0x7fe8f00bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.513+0000 7fe9094e6640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 0x7fe8f00bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.513+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe9000e7780 con 0x7fe9041083e0 2024-09-21T03:16:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.513+0000 7fe9094e6640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 0x7fe8f00bcf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.516+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe900014030 con 0x7fe9041083e0 2024-09-21T03:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.696+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe8e8005850 con 0x7fe9041083e0 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.698+0000 7fe8fa7fc640 1 -- 172.21.15.83:0/1297793120 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5645 (secure 0 0 0) 0x7fe9000adfa0 con 0x7fe9041083e0 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 77 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-21T03:06:13.570352+0000 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-09-21T03:15:00.250430+0000 2024-09-21T03:16:22.699 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 26 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 9 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/4290610745,v1:172.21.15.83:6803/4290610745] [v2:172.21.15.83:6804/4290610745,v1:172.21.15.83:6805/4290610745] exists,up f96fcc94-0c90-4aaa-83e9-93e0c3cdc835 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 14 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/3543209885,v1:172.21.15.142:6801/3543209885] [v2:172.21.15.142:6802/3543209885,v1:172.21.15.142:6803/3543209885] exists,up 949b227b-4c32-4ca5-a1db-df21fe923196 2024-09-21T03:16:22.700 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 19 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/905402231,v1:172.21.15.193:6801/905402231] [v2:172.21.15.193:6802/905402231,v1:172.21.15.193:6803/905402231] exists,up 49bca58f-0193-459d-915a-a295270c7fb4 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 26 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/4077666910,v1:172.21.15.83:6811/4077666910] [v2:172.21.15.83:6812/4077666910,v1:172.21.15.83:6813/4077666910] exists,up 280e4e6e-e486-492c-b238-36b38206052c 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/4220832012,v1:172.21.15.142:6809/4220832012] [v2:172.21.15.142:6810/4220832012,v1:172.21.15.142:6811/4220832012] exists,up 0c6f14a7-871f-4d09-a704-71022463636c 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2703219317,v1:172.21.15.193:6809/2703219317] [v2:172.21.15.193:6810/2703219317,v1:172.21.15.193:6811/2703219317] exists,up 0de58bca-db3b-4600-83bb-9df3665dc790 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 42 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/2541470109,v1:172.21.15.83:6819/2541470109] [v2:172.21.15.83:6820/2541470109,v1:172.21.15.83:6821/2541470109] exists,up 2a5a5708-35f7-4e4b-a46c-3c1afab03a79 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 47 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/3641783594,v1:172.21.15.142:6817/3641783594] [v2:172.21.15.142:6818/3641783594,v1:172.21.15.142:6819/3641783594] exists,up 07842e54-9f66-42d4-9a74-2458dbdd8c37 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 53 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/1886733655,v1:172.21.15.193:6817/1886733655] [v2:172.21.15.193:6818/1886733655,v1:172.21.15.193:6819/1886733655] exists,up 2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 59 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/400576662,v1:172.21.15.83:6827/400576662] [v2:172.21.15.83:6828/400576662,v1:172.21.15.83:6829/400576662] exists,up fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 63 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/2781310607,v1:172.21.15.142:6825/2781310607] [v2:172.21.15.142:6826/2781310607,v1:172.21.15.142:6827/2781310607] exists,up f60a1b3a-0053-460c-848e-aa177514017d 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/1672057754,v1:172.21.15.193:6825/1672057754] [v2:172.21.15.193:6826/1672057754,v1:172.21.15.193:6827/1672057754] exists,up 920f94f1-463c-4ffd-bd02-2470631a6746 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.4 [2,11] 2024-09-21T03:16:22.701 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.13 [2,11] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1a [8,4] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.3 [4,9] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.c [3,9] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.1d [4,8] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.21 [5,9] 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1976219591 expires 2024-09-22T03:07:59.080195+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3984963619 expires 2024-09-22T03:06:36.366094+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/3130587124 expires 2024-09-22T03:07:01.129903+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2346813165 expires 2024-09-22T03:07:59.080195+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/1246429082 expires 2024-09-22T03:06:36.366094+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/493695771 expires 2024-09-22T03:07:59.080195+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/1246429082 expires 2024-09-22T03:06:36.366094+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/211211530 expires 2024-09-22T03:07:01.129903+0000 2024-09-21T03:16:22.702 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1905391534 expires 2024-09-22T03:06:36.366094+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/817883806 expires 2024-09-22T03:06:36.366094+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3621693467 expires 2024-09-22T03:07:01.129903+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2962867597 expires 2024-09-22T03:07:01.129903+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3776769458 expires 2024-09-22T03:07:59.080195+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/3130587124 expires 2024-09-22T03:07:01.129903+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/493695771 expires 2024-09-22T03:07:59.080195+0000 2024-09-21T03:16:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 msgr2=0x7fe8f00bcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 0x7fe8f00bcf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f4008040 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 msgr2=0x7fe90419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c8c0 tx=0x7fe90000cd80 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 shutdown_connections 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8f00baa20 0x7fe8f00bcf30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe904108d30 0x7fe90419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9041083e0 0x7fe90419aa90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 --2- 172.21.15.83:0/1297793120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe904102aa0 0x7fe90419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 >> 172.21.15.83:0/1297793120 conn(0x7fe904079750 msgr2=0x7fe904101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 shutdown_connections 2024-09-21T03:16:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:22.701+0000 7fe90b74a640 1 -- 172.21.15.83:0/1297793120 wait complete. 2024-09-21T03:16:23.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:22 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1297793120' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:16:23.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:22 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1297793120' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:16:23.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:22 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1297793120' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:16:23.453 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:16:23.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:16:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:24.095 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:23 smithi083 ceph-mon[31219]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:24.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:23 smithi142 ceph-mon[34512]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 -- 172.21.15.83:0/2000708202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9701101e0 msgr2=0x7fa970112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2000708202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9701101e0 0x7fa970112600 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa968009ca0 tx=0x7fa96802a950 comp rx=0 tx=0).stop 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 -- 172.21.15.83:0/2000708202 shutdown_connections 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2000708202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9701101e0 0x7fa970112600 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2000708202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102f80 0x7fa97010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2000708202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa970102630 0x7fa970102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 -- 172.21.15.83:0/2000708202 >> 172.21.15.83:0/2000708202 conn(0x7fa970079730 msgr2=0x7fa970079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 -- 172.21.15.83:0/2000708202 shutdown_connections 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.183+0000 7fa9775aa640 1 -- 172.21.15.83:0/2000708202 wait complete. 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 Processor -- start 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 -- start start 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102630 0x7fa97019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9701101e0 0x7fa9701a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa970115c50 con 0x7fa970102f80 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa970115ad0 con 0x7fa9701101e0 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa9775aa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa970115dd0 con 0x7fa970102630 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39084/0 (socket says 172.21.15.83:39084) 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 -- 172.21.15.83:0/2269511599 learned_addr learned my addr 172.21.15.83:0/2269511599 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa975b47640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9701101e0 0x7fa9701a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa975346640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102630 0x7fa97019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 -- 172.21.15.83:0/2269511599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102630 msgr2=0x7fa97019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102630 0x7fa97019c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 -- 172.21.15.83:0/2269511599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9701101e0 msgr2=0x7fa9701a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9701101e0 0x7fa9701a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.184+0000 7fa974b45640 1 -- 172.21.15.83:0/2269511599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9701a2510 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa974b45640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa96000d950 tx=0x7fa96000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960017030 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9701a2800 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa960004020 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600047a0 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.185+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9701a8f60 con 0x7fa970102f80 2024-09-21T03:16:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.186+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa970102a40 con 0x7fa970102f80 2024-09-21T03:16:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.188+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa960009a30 con 0x7fa970102f80 2024-09-21T03:16:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.188+0000 7fa9667fc640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 0x7fa95c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.188+0000 7fa975346640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 0x7fa95c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.188+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9600e7590 con 0x7fa970102f80 2024-09-21T03:16:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.189+0000 7fa975346640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 0x7fa95c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009b60 tx=0x7fa96c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.191+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9600add80 con 0x7fa970102f80 2024-09-21T03:16:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:23 smithi193 ceph-mon[35224]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.405+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa97019d6d0 con 0x7fa970102f80 2024-09-21T03:16:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.406+0000 7fa9667fc640 1 -- 172.21.15.83:0/2269511599 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7fa9600b2160 con 0x7fa970102f80 2024-09-21T03:16:24.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:24.408 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:16:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 msgr2=0x7fa95c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 0x7fa95c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009b60 tx=0x7fa96c005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 msgr2=0x7fa97019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa96000d950 tx=0x7fa96000de10 comp rx=0 tx=0).stop 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 shutdown_connections 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa95c0b6560 0x7fa95c0b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9701101e0 0x7fa9701a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa970102f80 0x7fa97019cb40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 --2- 172.21.15.83:0/2269511599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa970102630 0x7fa97019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.408+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 >> 172.21.15.83:0/2269511599 conn(0x7fa970079730 msgr2=0x7fa970109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.409+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 shutdown_connections 2024-09-21T03:16:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:24.409+0000 7fa9775aa640 1 -- 172.21.15.83:0/2269511599 wait complete. 2024-09-21T03:16:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:24 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2269511599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:25.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:16:25.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:24 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2269511599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:24 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2269511599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 -- 172.21.15.83:0/3304542915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf001083d0 msgr2=0x7faf001087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 --2- 172.21.15.83:0/3304542915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf001083d0 0x7faf001087e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faef0009ca0 tx=0x7faef002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 -- 172.21.15.83:0/3304542915 shutdown_connections 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 --2- 172.21.15.83:0/3304542915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 0x7faf0010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 --2- 172.21.15.83:0/3304542915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf001083d0 0x7faf001087e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 --2- 172.21.15.83:0/3304542915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf00102a90 0x7faf00102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 -- 172.21.15.83:0/3304542915 >> 172.21.15.83:0/3304542915 conn(0x7faf00079750 msgr2=0x7faf00079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 -- 172.21.15.83:0/3304542915 shutdown_connections 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.800+0000 7faf0713a640 1 -- 172.21.15.83:0/3304542915 wait complete. 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 Processor -- start 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 -- start start 2024-09-21T03:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf001083d0 0x7faf0019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 0x7faf0019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112990 con 0x7faf00102a90 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112810 con 0x7faf00108d20 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf0713a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112b10 con 0x7faf001083d0 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf04ed6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf04ed6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39112/0 (socket says 172.21.15.83:39112) 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.801+0000 7faf04ed6640 1 -- 172.21.15.83:0/4093542231 learned_addr learned my addr 172.21.15.83:0/4093542231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faefffff640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf001083d0 0x7faf0019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 -- 172.21.15.83:0/4093542231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf001083d0 msgr2=0x7faf0019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf001083d0 0x7faf0019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 -- 172.21.15.83:0/4093542231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 msgr2=0x7faf0019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf056d7640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 0x7faf0019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 0x7faf0019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 -- 172.21.15.83:0/4093542231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001a04c0 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf04ed6640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faef800d950 tx=0x7faef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8017030 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf001a07b0 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef8004020 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80047a0 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.802+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a6f10 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.803+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf00103d10 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.805+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7faef8009a30 con 0x7faf00102a90 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.806+0000 7faefdffb640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 0x7faeec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.806+0000 7faefffff640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 0x7faeec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.806+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7faef80e74a0 con 0x7faf00102a90 2024-09-21T03:16:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.806+0000 7faefffff640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 0x7faeec0b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faef0005e30 tx=0x7faef0009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:25.809+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faef80adc90 con 0x7faf00102a90 2024-09-21T03:16:26.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.025+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf00102ea0 con 0x7faf00102a90 2024-09-21T03:16:26.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:25 smithi083 ceph-mon[31219]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:26.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.026+0000 7faefdffb640 1 -- 172.21.15.83:0/4093542231 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7faef80b2070 con 0x7faf00102a90 2024-09-21T03:16:26.028 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:26.028 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-09-21T03:14:48:753858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:14:48.753854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:16:26.028 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 msgr2=0x7faeec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 0x7faeec0b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faef0005e30 tx=0x7faef0009360 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 msgr2=0x7faf0019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faef800d950 tx=0x7faef800de10 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 shutdown_connections 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faeec0b6560 0x7faeec0b8a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00108d20 0x7faf0019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf001083d0 0x7faf0019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 --2- 172.21.15.83:0/4093542231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf00102a90 0x7faf0019a550 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 >> 172.21.15.83:0/4093542231 conn(0x7faf00079750 msgr2=0x7faf00101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 shutdown_connections 2024-09-21T03:16:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:26.028+0000 7faf0713a640 1 -- 172.21.15.83:0/4093542231 wait complete. 2024-09-21T03:16:26.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:25 smithi142 ceph-mon[34512]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:25 smithi193 ceph-mon[35224]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:26.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:16:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:27.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:26 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4093542231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:27.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:26 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4093542231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:27.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:26 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4093542231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 -- 172.21.15.83:0/691533765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 msgr2=0x7f41281106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 --2- 172.21.15.83:0/691533765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f41281106b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4120009ca0 tx=0x7f412002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 -- 172.21.15.83:0/691533765 shutdown_connections 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 --2- 172.21.15.83:0/691533765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f41281106b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 --2- 172.21.15.83:0/691533765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 0x7f4128106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 --2- 172.21.15.83:0/691533765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 0x7f41281024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 -- 172.21.15.83:0/691533765 >> 172.21.15.83:0/691533765 conn(0x7f4128079f90 msgr2=0x7f412807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:27.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 -- 172.21.15.83:0/691533765 shutdown_connections 2024-09-21T03:16:27.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.552+0000 7f412dd77640 1 -- 172.21.15.83:0/691533765 wait complete. 2024-09-21T03:16:27.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.553+0000 7f412dd77640 1 Processor -- start 2024-09-21T03:16:27.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.553+0000 7f412dd77640 1 -- start start 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.553+0000 7f412dd77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 0x7f412819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f412dd77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 0x7f412819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f412dd77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f412dd77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113bf0 con 0x7f4128106c40 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f412dd77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113a70 con 0x7f41281062f0 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f412dd77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113d70 con 0x7f41281020a0 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39120/0 (socket says 172.21.15.83:39120) 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 -- 172.21.15.83:0/3245108697 learned_addr learned my addr 172.21.15.83:0/3245108697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4126ffd640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 0x7f412819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f41277fe640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 0x7f412819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:27.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 -- 172.21.15.83:0/3245108697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 msgr2=0x7f412819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 0x7f412819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 -- 172.21.15.83:0/3245108697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 msgr2=0x7f412819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 0x7f412819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 -- 172.21.15.83:0/3245108697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41281a0610 con 0x7f4128106c40 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4127fff640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4120006010 tx=0x7f4120003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120004240 con 0x7f4128106c40 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.554+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120003e10 con 0x7f4128106c40 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.555+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41281a08a0 con 0x7f4128106c40 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.555+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120030e10 con 0x7f4128106c40 2024-09-21T03:16:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.555+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a7060 con 0x7f4128106c40 2024-09-21T03:16:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.556+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4128102530 con 0x7f4128106c40 2024-09-21T03:16:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.557+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4120004b40 con 0x7f4128106c40 2024-09-21T03:16:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.558+0000 7f4124ff9640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 0x7f41100b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.558+0000 7f41277fe640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 0x7f41100b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.558+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4120108830 con 0x7f4128106c40 2024-09-21T03:16:27.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.558+0000 7f41277fe640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 0x7f41100b8af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f412806bb10 tx=0x7f4118009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.561+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41200cf020 con 0x7f4128106c40 2024-09-21T03:16:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.738+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f412810bd60 con 0x7f4128106c40 2024-09-21T03:16:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.739+0000 7f4124ff9640 1 -- 172.21.15.83:0/3245108697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7f4120009800 con 0x7f4128106c40 2024-09-21T03:16:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 msgr2=0x7f41100b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 0x7f41100b8af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f412806bb10 tx=0x7f4118009110 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 msgr2=0x7f412819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4120006010 tx=0x7f4120003c20 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 shutdown_connections 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41100b65e0 0x7f41100b8af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4128106c40 0x7f412819fee0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41281062f0 0x7f412819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 --2- 172.21.15.83:0/3245108697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41281020a0 0x7f412819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 >> 172.21.15.83:0/3245108697 conn(0x7f4128079f90 msgr2=0x7f41281055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 shutdown_connections 2024-09-21T03:16:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:27.741+0000 7f412dd77640 1 -- 172.21.15.83:0/3245108697 wait complete. 2024-09-21T03:16:28.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:27 smithi083 ceph-mon[31219]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:28.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:27 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3245108697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:28.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:27 smithi142 ceph-mon[34512]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:28.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:27 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3245108697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:28.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:27 smithi193 ceph-mon[35224]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:28.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:27 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3245108697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:28.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:14:37.183732+0000', 'modified': '2024-09-21T03:14:48.753854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24647, 'mds_1': 24628, 'mds_2': 24620, 'mds_3': 24613, 'mds_4': 24601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/3046510766', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3046510766}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3046510766}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/2540451962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 2540451962}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 2540451962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/1409264490', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 1409264490}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 1409264490}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24571': {'gid': 24571, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/817680520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 817680520}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 817680520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24593': {'gid': 24593, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/3055596594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 3055596594}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 3055596594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24601': {'gid': 24601, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.193:6835/1130551949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1130551949}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1130551949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.193:6837/3456789844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 3456789844}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 3456789844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.142:6837/1691832808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1691832808}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1691832808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/357057598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 357057598}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 357057598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24647': {'gid': 24647, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.142:6839/3152549242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3152549242}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3152549242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24647, 'qdb_cluster': [24601, 24613, 24620, 24628, 24647]} 2024-09-21T03:16:28.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T03:16:28.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T03:16:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 -- 172.21.15.83:0/3482961737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32881083a0 msgr2=0x7f32881087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 --2- 172.21.15.83:0/3482961737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32881083a0 0x7f32881087b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f327c009900 tx=0x7f327c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 -- 172.21.15.83:0/3482961737 shutdown_connections 2024-09-21T03:16:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 --2- 172.21.15.83:0/3482961737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3288108cf0 0x7f328810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 --2- 172.21.15.83:0/3482961737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32881083a0 0x7f32881087b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 --2- 172.21.15.83:0/3482961737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 0x7f3288102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 -- 172.21.15.83:0/3482961737 >> 172.21.15.83:0/3482961737 conn(0x7f3288079760 msgr2=0x7f3288079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 -- 172.21.15.83:0/3482961737 shutdown_connections 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.121+0000 7f328d712640 1 -- 172.21.15.83:0/3482961737 wait complete. 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 Processor -- start 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 -- start start 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 0x7f328819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32881083a0 0x7f328819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3288113ad0 con 0x7f3288108cf0 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3288113950 con 0x7f32881083a0 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f328d712640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3288113c50 con 0x7f3288102a60 2024-09-21T03:16:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f32877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f32877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39142/0 (socket says 172.21.15.83:39142) 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f32877fe640 1 -- 172.21.15.83:0/2506540852 learned_addr learned my addr 172.21.15.83:0/2506540852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.122+0000 7f3286ffd640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 0x7f328819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32867fc640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32881083a0 0x7f328819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 -- 172.21.15.83:0/2506540852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 msgr2=0x7f328819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 0x7f328819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 -- 172.21.15.83:0/2506540852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32881083a0 msgr2=0x7f328819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32881083a0 0x7f328819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 -- 172.21.15.83:0/2506540852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32881a03f0 con 0x7f3288108cf0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f32877fe640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f328000b970 tx=0x7f328000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f326ffff640 1 -- 172.21.15.83:0/2506540852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280019070 con 0x7f3288108cf0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f326ffff640 1 -- 172.21.15.83:0/2506540852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3280014440 con 0x7f3288108cf0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32881a06e0 con 0x7f3288108cf0 2024-09-21T03:16:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f326ffff640 1 -- 172.21.15.83:0/2506540852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32800075d0 con 0x7f3288108cf0 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.123+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32881a6f00 con 0x7f3288108cf0 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 Processor -- start 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f326ffff640 1 -- 172.21.15.83:0/2506540852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f328001c070 con 0x7f3288108cf0 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 -- start start 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f804f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f805980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f77cc10 con 0x55ba2f4360a0 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f76c040 con 0x55ba2f43e280 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3293c86740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f76fe50 con 0x55ba2f45ef60 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39148/0 (socket says 172.21.15.83:39148) 2024-09-21T03:16:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 -- 172.21.15.83:0/4151403220 learned_addr learned my addr 172.21.15.83:0/4151403220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f32867fc640 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f805980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f32867fc640 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f804f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 -- 172.21.15.83:0/4151403220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 msgr2=0x55ba2f804f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f804f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 -- 172.21.15.83:0/4151403220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 msgr2=0x55ba2f805980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f3286ffd640 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f805980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.126+0000 7f326ffff640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 0x7f32700b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f3286ffd640 1 -- 172.21.15.83:0/4151403220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2f4ca540 con 0x55ba2f4360a0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f32877fe640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 0x7f32700b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f3286ffd640 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f327800cc60 tx=0x7f327800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba63256d1cd2a688 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f326ffff640 1 -- 172.21.15.83:0/2506540852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f32800e6970 con 0x7f3288108cf0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f326d7fa640 1 -- 172.21.15.83:0/4151403220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278014070 con 0x55ba2f4360a0 2024-09-21T03:16:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f326d7fa640 1 -- 172.21.15.83:0/4151403220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780047a0 con 0x55ba2f4360a0 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.127+0000 7f326d7fa640 1 -- 172.21.15.83:0/4151403220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278010690 con 0x55ba2f4360a0 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- 172.21.15.83:0/4151403220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 msgr2=0x55ba2f805440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f32877fe640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 0x7f32700b8aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32800a9cd0 tx=0x7f3280079d40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f327800cc60 tx=0x7f327800e720 comp rx=0 tx=0).stop 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- 172.21.15.83:0/4151403220 shutdown_connections 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f805980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f805440 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 --2- 172.21.15.83:0/4151403220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f804f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- 172.21.15.83:0/4151403220 >> 172.21.15.83:0/4151403220 conn(0x55ba2f453350 msgr2=0x55ba2f802df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- 172.21.15.83:0/4151403220 shutdown_connections 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- 172.21.15.83:0/4151403220 wait complete. 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.128+0000 7f3293c86740 1 -- start start 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 Processor -- start 2024-09-21T03:16:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f810fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f81ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f5cd5f0 con 0x55ba2f4360a0 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f707010 con 0x55ba2f43e280 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f8136e0 con 0x55ba2f45ef60 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3293c86740 10 client.0 _ll_register_callbacks cb 0x55ba2f766340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39162/0 (socket says 172.21.15.83:39162) 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 -- 172.21.15.83:0/1260953261 learned_addr learned my addr 172.21.15.83:0/1260953261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f3286ffd640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f810fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 -- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 msgr2=0x55ba2f81ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f81ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 -- 172.21.15.83:0/1260953261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 msgr2=0x55ba2f810fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f810fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 -- 172.21.15.83:0/1260953261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2f734a00 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f32867fc640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f327c005ab0 tx=0x7f327c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4842e39c9dd03175 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.129+0000 7f326aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c035040 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f327c004450 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32780047a0 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f327c005020 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 10 client.15384 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba2f81f6a0 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780047a0 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f327c03a040 con 0x55ba2f4360a0 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 10 client.15384 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 20 client.15384 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f41d8e0 0x55ba2f8230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 4 client.15384 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.130+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2f77eb60 con 0x55ba2f41d8e0 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.131+0000 7f3286ffd640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f41d8e0 0x55ba2f8230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.131+0000 7f326aff5640 10 client.15384 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.131+0000 7f3286ffd640 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f41d8e0 0x55ba2f8230a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc2b50b50ed5c81e server_cookie=733234357d3c365f in_seq=0 out_seq=0 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f326aff5640 1 -- 172.21.15.83:0/1260953261 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba2f77eb60 con 0x55ba2f41d8e0 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f326aff5640 10 client.15384 handle_command_reply: tid=0 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f3293c86740 1 client.15384 shutdown 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 msgr2=0x55ba2f810aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f327c005ab0 tx=0x7f327c031740 comp rx=0 tx=0).stop 2024-09-21T03:16:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 shutdown_connections 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.132+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f41d8e0 0x55ba2f8230a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f81ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f43e280 0x55ba2f810fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 --2- 172.21.15.83:0/1260953261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f4360a0 0x55ba2f810aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 >> 172.21.15.83:0/1260953261 conn(0x55ba2f453350 msgr2=0x55ba2f808090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f326aff5640 0 client.15384 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 shutdown_connections 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 1 -- 172.21.15.83:0/1260953261 wait complete. 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.133+0000 7f3293c86740 20 client.15384 trim_cache size 0 max 16384 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.150+0000 7f3293c86740 1 Processor -- start 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 -- start start 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f4360a0 0x55ba2f810a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 0x55ba2f80ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f8213e0 con 0x55ba2f43e280 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f821ab0 con 0x55ba2f45ef60 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3293c86740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f822180 con 0x55ba2f4360a0 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39174/0 (socket says 172.21.15.83:39174) 2024-09-21T03:16:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 -- 172.21.15.83:0/633888864 learned_addr learned my addr 172.21.15.83:0/633888864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f32867fc640 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 0x55ba2f80ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 -- 172.21.15.83:0/633888864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f4360a0 msgr2=0x55ba2f810a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f4360a0 0x55ba2f810a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 -- 172.21.15.83:0/633888864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 msgr2=0x55ba2f80ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 0x55ba2f80ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 -- 172.21.15.83:0/633888864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2f7fd080 con 0x55ba2f43e280 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f32867fc640 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 0x55ba2f80ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f3286ffd640 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f327800f010 tx=0x7f327800c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d57af745a233ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f326cff9640 1 -- 172.21.15.83:0/633888864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278014040 con 0x55ba2f43e280 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.151+0000 7f326cff9640 1 -- 172.21.15.83:0/633888864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278004220 con 0x55ba2f43e280 2024-09-21T03:16:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f326cff9640 1 -- 172.21.15.83:0/633888864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278010430 con 0x55ba2f43e280 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 -- 172.21.15.83:0/633888864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 msgr2=0x55ba2f810fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f327800f010 tx=0x7f327800c770 comp rx=0 tx=0).stop 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 -- 172.21.15.83:0/633888864 shutdown_connections 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f45ef60 0x55ba2f80ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f810fc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 --2- 172.21.15.83:0/633888864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f4360a0 0x55ba2f810a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 -- 172.21.15.83:0/633888864 >> 172.21.15.83:0/633888864 conn(0x55ba2f453350 msgr2=0x55ba2f80ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 -- 172.21.15.83:0/633888864 shutdown_connections 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.152+0000 7f3293c86740 1 -- 172.21.15.83:0/633888864 wait complete. 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 -- start start 2024-09-21T03:16:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 Processor -- start 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 0x55ba2f82b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f8252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f822f20 con 0x55ba2f43e280 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f822850 con 0x55ba2f4360a0 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba2f8235f0 con 0x55ba2f45ef60 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3293c86740 10 client.0 _ll_register_callbacks cb 0x7f327c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39184/0 (socket says 172.21.15.83:39184) 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f3286ffd640 1 -- 172.21.15.83:0/3938664916 learned_addr learned my addr 172.21.15.83:0/3938664916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f32867fc640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 0x55ba2f82b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.153+0000 7f32867fc640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f8252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 -- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 msgr2=0x55ba2f8252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f8252d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 -- 172.21.15.83:0/3938664916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 msgr2=0x55ba2f82b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 0x55ba2f82b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 -- 172.21.15.83:0/3938664916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2f825810 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f32867fc640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 0x55ba2f82b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3286ffd640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f327800ec20 tx=0x7f32780047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db2a0631db6be62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327803a060 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3278003cb0 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278004220 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3278014030 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3293c86740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba2f82b180 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278004220 con 0x55ba2f43e280 2024-09-21T03:16:29.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f3278010b20 con 0x55ba2f43e280 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.154+0000 7f3293c86740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 10 client.15390 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 20 client.15390 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f3f2c90 0x55ba2f8913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 4 client.15390 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "status"}) -- 0x55ba2f77a610 con 0x55ba2f3f2c90 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f32867fc640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f3f2c90 0x55ba2f8913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f32867fc640 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f3f2c90 0x55ba2f8913b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce88cb4133906821 server_cookie=bb6d7f7735e61b6 in_seq=0 out_seq=0 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f326aff5640 10 client.15390 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f326aff5640 1 -- 172.21.15.83:0/3938664916 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ba2f77a610 con 0x55ba2f3f2c90 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f326aff5640 10 client.15390 handle_command_reply: tid=0 2024-09-21T03:16:29.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.155+0000 7f3293c86740 1 client.15390 shutdown 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 msgr2=0x55ba2f824d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f327800ec20 tx=0x7f32780047a0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 shutdown_connections 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55ba2f3f2c90 0x55ba2f8913b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba2f45ef60 0x55ba2f8252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2f43e280 0x55ba2f824d90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f326aff5640 0 client.15390 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 --2- 172.21.15.83:0/3938664916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2f4360a0 0x55ba2f82b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 >> 172.21.15.83:0/3938664916 conn(0x55ba2f453350 msgr2=0x55ba2f80ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 shutdown_connections 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 1 -- 172.21.15.83:0/3938664916 wait complete. 2024-09-21T03:16:29.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f3293c86740 20 client.15390 trim_cache size 0 max 16384 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24601, 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 101.298142892, 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:29.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:29.165 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 137.90884715499999 2024-09-21T03:16:29.165 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:29.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 msgr2=0x7f32700b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 0x7f32700b8aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32800a9cd0 tx=0x7f3280079d40 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 msgr2=0x7f328819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f328000b970 tx=0x7f328000be30 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 shutdown_connections 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32700b6590 0x7f32700b8aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3288108cf0 0x7f328819fd80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32881083a0 0x7f328819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 --2- 172.21.15.83:0/2506540852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3288102a60 0x7f328819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.156+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 >> 172.21.15.83:0/2506540852 conn(0x7f3288079760 msgr2=0x7f32881064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.157+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 shutdown_connections 2024-09-21T03:16:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:29.157+0000 7f328d712640 1 -- 172.21.15.83:0/2506540852 wait complete. 2024-09-21T03:16:29.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:16:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:30.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:30 smithi083 ceph-mon[31219]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:30 smithi193 ceph-mon[35224]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/3856656392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b681049b0 msgr2=0x7f0b68104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/3856656392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b681049b0 0x7f0b68104dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009ca0 tx=0x7f0b5802a8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/3856656392 shutdown_connections 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/3856656392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b6810de10 0x7f0b68110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/3856656392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b68105300 0x7f0b6810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/3856656392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b681049b0 0x7f0b68104dc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/3856656392 >> 172.21.15.83:0/3856656392 conn(0x7f0b68079730 msgr2=0x7f0b68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/3856656392 shutdown_connections 2024-09-21T03:16:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.631+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/3856656392 wait complete. 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 Processor -- start 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 -- start start 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b681049b0 0x7f0b6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 0x7f0b6819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68113ad0 con 0x7f0b68105300 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68113950 con 0x7f0b681049b0 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6f7a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68113c50 con 0x7f0b6810de10 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47846/0 (socket says 172.21.15.83:47846) 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.632+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/727169178 learned_addr learned my addr 172.21.15.83:0/727169178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 0x7f0b6819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b681049b0 0x7f0b6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/727169178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 msgr2=0x7f0b6819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 0x7f0b6819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/727169178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b681049b0 msgr2=0x7f0b6819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b681049b0 0x7f0b6819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/727169178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b681a0410 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 0x7f0b6819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b667fc640 1 -- 172.21.15.83:0/727169178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c011830 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b667fc640 1 -- 172.21.15.83:0/727169178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c016070 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681a0700 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b667fc640 1 -- 172.21.15.83:0/727169178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c010e00 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.633+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a6ee0 con 0x7f0b68105300 2024-09-21T03:16:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 Processor -- start 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b667fc640 1 -- 172.21.15.83:0/727169178 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0b5c01b460 con 0x7f0b68105300 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 -- start start 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 0x559199982440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x559199982980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5591998f9c10 con 0x5591995dbf60 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5591998e9040 con 0x5591995b30a0 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b75d15740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591998ece50 con 0x5591995bb280 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6dd3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6dd3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47854/0 (socket says 172.21.15.83:47854) 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6dd3e640 1 -- 172.21.15.83:0/3220542765 learned_addr learned my addr 172.21.15.83:0/3220542765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 0x559199982440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x559199982980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.636+0000 7f0b6dd3e640 1 -- 172.21.15.83:0/3220542765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 msgr2=0x559199982980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x559199982980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6dd3e640 1 -- 172.21.15.83:0/3220542765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 msgr2=0x559199982440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 0x559199982440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6dd3e640 1 -- 172.21.15.83:0/3220542765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559199647540 con 0x5591995dbf60 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x559199982980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 0x559199982440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b667fc640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 0x7f0b500b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009bd0 tx=0x7f0b6000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cc27f245760234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 0x7f0b500b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b4f7fe640 1 -- 172.21.15.83:0/3220542765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60016070 con 0x5591995dbf60 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b667fc640 1 -- 172.21.15.83:0/727169178 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0b5c0e6940 con 0x7f0b68105300 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b4f7fe640 1 -- 172.21.15.83:0/3220542765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b60011920 con 0x5591995dbf60 2024-09-21T03:16:30.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.637+0000 7f0b4f7fe640 1 -- 172.21.15.83:0/3220542765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60010790 con 0x5591995dbf60 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- 172.21.15.83:0/3220542765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 msgr2=0x559199981f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009bd0 tx=0x7f0b6000e6c0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 0x7f0b500b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559199980b80 tx=0x7f0b58009430 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- 172.21.15.83:0/3220542765 shutdown_connections 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x559199982980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995b30a0 0x559199982440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3220542765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995dbf60 0x559199981f00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- 172.21.15.83:0/3220542765 >> 172.21.15.83:0/3220542765 conn(0x5591995d0350 msgr2=0x55919997fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- 172.21.15.83:0/3220542765 shutdown_connections 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- 172.21.15.83:0/3220542765 wait complete. 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 -- start start 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.638+0000 7f0b75d15740 1 Processor -- start 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995bb280 0x55919998dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995dbf60 0x559199999a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55919974a5f0 con 0x5591995b30a0 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559199884010 con 0x5591995bb280 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591999906e0 con 0x5591995dbf60 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b75d15740 10 client.0 _ll_register_callbacks cb 0x5591998e3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47864/0 (socket says 172.21.15.83:47864) 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 -- 172.21.15.83:0/3223422408 learned_addr learned my addr 172.21.15.83:0/3223422408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995bb280 0x55919998dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 -- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995dbf60 msgr2=0x559199999a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995dbf60 0x559199999a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995dbf60 0x559199999a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 -- 172.21.15.83:0/3223422408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995bb280 msgr2=0x55919998dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995bb280 0x55919998dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.639+0000 7f0b6d53d640 1 -- 172.21.15.83:0/3223422408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591998b1a00 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b58035000 tx=0x7f0b580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a647ec225ea25f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58004740 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0b580305e0 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b60011920 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b580307e0 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 10 client.15402 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55919999c550 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b60011920 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0b580495b0 con 0x5591995b30a0 2024-09-21T03:16:30.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 20 client.15402 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55919959a8e0 0x5591999a0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 4 client.15402 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.640+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591998fbb60 con 0x55919959a8e0 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.641+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55919959a8e0 0x5591999a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.641+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55919959a8e0 0x5591999a0030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3779d0455eca80cb server_cookie=9fd5de7aa4adb00d in_seq=0 out_seq=0 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.641+0000 7f0b4cff9640 10 client.15402 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b4cff9640 1 -- 172.21.15.83:0/3223422408 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591998fbb60 con 0x55919959a8e0 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b4cff9640 10 client.15402 handle_command_reply: tid=0 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 client.15402 shutdown 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 msgr2=0x55919998d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b58035000 tx=0x7f0b580095c0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 shutdown_connections 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55919959a8e0 0x5591999a0030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995dbf60 0x559199999a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b4cff9640 0 client.15402 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591995bb280 0x55919998dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 --2- 172.21.15.83:0/3223422408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998d7b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 >> 172.21.15.83:0/3223422408 conn(0x5591995d0350 msgr2=0x5591999850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 shutdown_connections 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.642+0000 7f0b75d15740 1 -- 172.21.15.83:0/3223422408 wait complete. 2024-09-21T03:16:30.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.643+0000 7f0b75d15740 20 client.15402 trim_cache size 0 max 16384 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.660+0000 7f0b75d15740 1 Processor -- start 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 -- start start 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x55919999de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x55919998c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55919999e370 con 0x5591995b30a0 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55919999ea40 con 0x55919959a8e0 2024-09-21T03:16:30.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b75d15740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55919999f110 con 0x5591995bb280 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47866/0 (socket says 172.21.15.83:47866) 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/1837969641 learned_addr learned my addr 172.21.15.83:0/1837969641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/1837969641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 msgr2=0x55919998c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x55919998c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/1837969641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 msgr2=0x55919999de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x55919999de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x55919999de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/1837969641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559199989320 con 0x5591995b30a0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.661+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0e43a0 tx=0x7f0b5c0d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e8bca077cf70b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b4effd640 1 -- 172.21.15.83:0/1837969641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0d00b0 con 0x5591995b30a0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b4effd640 1 -- 172.21.15.83:0/1837969641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0b1510 con 0x5591995b30a0 2024-09-21T03:16:30.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b4effd640 1 -- 172.21.15.83:0/1837969641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0ab700 con 0x5591995b30a0 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 -- 172.21.15.83:0/1837969641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 msgr2=0x55919998c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0e43a0 tx=0x7f0b5c0d0040 comp rx=0 tx=0).stop 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 -- 172.21.15.83:0/1837969641 shutdown_connections 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x55919998c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x55919998c030 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 --2- 172.21.15.83:0/1837969641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x55919999de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 -- 172.21.15.83:0/1837969641 >> 172.21.15.83:0/1837969641 conn(0x5591995d0350 msgr2=0x55919997f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 -- 172.21.15.83:0/1837969641 shutdown_connections 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.662+0000 7f0b75d15740 1 -- 172.21.15.83:0/1837969641 wait complete. 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 -- start start 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 Processor -- start 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x5591999a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x5591999a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55919999feb0 con 0x5591995b30a0 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55919999f7e0 con 0x55919959a8e0 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591999a0580 con 0x5591995bb280 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b75d15740 10 client.0 _ll_register_callbacks cb 0x7f0b58009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b6cd3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47880/0 (socket says 172.21.15.83:47880) 2024-09-21T03:16:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.663+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/4051019624 learned_addr learned my addr 172.21.15.83:0/4051019624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x5591999a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6dd3e640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x5591999a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 msgr2=0x5591999a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x5591999a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/4051019624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 msgr2=0x5591999a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x5591999a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 -- 172.21.15.83:0/4051019624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591999a48d0 con 0x5591995b30a0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b6cd3c640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c000c00 tx=0x7f0b5c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cc0f2c7ac3e323 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0b1040 con 0x5591995b30a0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0b5c0b11e0 con 0x5591995b30a0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c0b1510 con 0x5591995b30a0 2024-09-21T03:16:30.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0d00b0 con 0x5591995b30a0 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b75d15740 10 client.15408 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559199a0b780 con 0x5591995b30a0 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.664+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0b1510 con 0x5591995b30a0 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0b5c0ab700 con 0x5591995b30a0 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 20 client.15408 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5591995dbf60 0x559199a0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 4 client.15408 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "status"}) -- 0x5591998f7610 con 0x5591995dbf60 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5591995dbf60 0x559199a0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b4cff9640 10 client.15408 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:30.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.665+0000 7f0b6d53d640 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5591995dbf60 0x559199a0eb30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d26f40aac2439ac6 server_cookie=e2f9d9ad76a06f9c in_seq=0 out_seq=0 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b4cff9640 1 -- 172.21.15.83:0/4051019624 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5591998f7610 con 0x5591995dbf60 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b4cff9640 10 client.15408 handle_command_reply: tid=0 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 client.15408 shutdown 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 msgr2=0x5591999a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c000c00 tx=0x7f0b5c010820 comp rx=0 tx=0).stop 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 shutdown_connections 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5591995dbf60 0x559199a0eb30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591995bb280 0x5591999a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591995b30a0 0x5591999a2d50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b4cff9640 0 client.15408 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 --2- 172.21.15.83:0/4051019624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55919959a8e0 0x5591999a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 >> 172.21.15.83:0/4051019624 conn(0x5591995d0350 msgr2=0x55919997f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 shutdown_connections 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 1 -- 172.21.15.83:0/4051019624 wait complete. 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.666+0000 7f0b75d15740 20 client.15408 trim_cache size 0 max 16384 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24613, 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 103.821151629, 2024-09-21T03:16:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 136.81202766000001 2024-09-21T03:16:30.674 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 msgr2=0x7f0b500b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 0x7f0b500b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559199980b80 tx=0x7f0b58009430 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 msgr2=0x7f0b6819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c00b8e0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 shutdown_connections 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0b500b65e0 0x7f0b500b8af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b6810de10 0x7f0b6819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b68105300 0x7f0b6819aa40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 --2- 172.21.15.83:0/727169178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b681049b0 0x7f0b6819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 >> 172.21.15.83:0/727169178 conn(0x7f0b68079730 msgr2=0x7f0b681007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:30.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 shutdown_connections 2024-09-21T03:16:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:30.667+0000 7f0b6f7a1640 1 -- 172.21.15.83:0/727169178 wait complete. 2024-09-21T03:16:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:30 smithi142 ceph-mon[34512]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:31.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:16:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 -- 172.21.15.83:0/3368384206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4102ac0 msgr2=0x7f77a4102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 --2- 172.21.15.83:0/3368384206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4102ac0 0x7f77a4102ed0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f779402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 -- 172.21.15.83:0/3368384206 shutdown_connections 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 --2- 172.21.15.83:0/3368384206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4108d00 0x7f77a410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 --2- 172.21.15.83:0/3368384206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 0x7f77a41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 --2- 172.21.15.83:0/3368384206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4102ac0 0x7f77a4102ed0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 -- 172.21.15.83:0/3368384206 >> 172.21.15.83:0/3368384206 conn(0x7f77a4079720 msgr2=0x7f77a4079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 -- 172.21.15.83:0/3368384206 shutdown_connections 2024-09-21T03:16:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.232+0000 7f77a967a640 1 -- 172.21.15.83:0/3368384206 wait complete. 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.233+0000 7f77a967a640 1 Processor -- start 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.233+0000 7f77a967a640 1 -- start start 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4102ac0 0x7f77a41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 0x7f77a4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41129c0 con 0x7f77a4108d00 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4112840 con 0x7f77a4102ac0 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4112b40 con 0x7f77a41083b0 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47908/0 (socket says 172.21.15.83:47908) 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 -- 172.21.15.83:0/1969696370 learned_addr learned my addr 172.21.15.83:0/1969696370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4102ac0 0x7f77a41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a27fc640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 0x7f77a4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 -- 172.21.15.83:0/1969696370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 msgr2=0x7f77a4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 0x7f77a4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 -- 172.21.15.83:0/1969696370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4102ac0 msgr2=0x7f77a41990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4102ac0 0x7f77a41990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 -- 172.21.15.83:0/1969696370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a8790 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a37fe640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f779c00eaa0 tx=0x7f779c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f778bfff640 1 -- 172.21.15.83:0/1969696370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c00cdb0 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a8a80 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f778bfff640 1 -- 172.21.15.83:0/1969696370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779c005020 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.234+0000 7f778bfff640 1 -- 172.21.15.83:0/1969696370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0116d0 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.235+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a8fc0 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.237+0000 7f77afbee740 1 Processor -- start 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.237+0000 7f778bfff640 1 -- 172.21.15.83:0/1969696370 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f779c0051d0 con 0x7f77a4108d00 2024-09-21T03:16:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.237+0000 7f77afbee740 1 -- start start 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.237+0000 7f77afbee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7db3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.237+0000 7f77afbee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77afbee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7db4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77afbee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602b7d2bc10 con 0x5602b79e50a0 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77afbee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602b7d1b040 con 0x5602b79ed280 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77afbee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b7d1ee50 con 0x5602b7a0df60 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47910/0 (socket says 172.21.15.83:47910) 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 -- 172.21.15.83:0/753382179 learned_addr learned my addr 172.21.15.83:0/753382179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a27fc640 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7db3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a27fc640 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7db4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f778bfff640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 0x7f778c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 -- 172.21.15.83:0/753382179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 msgr2=0x5602b7db3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7db3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 -- 172.21.15.83:0/753382179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 msgr2=0x5602b7db4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7db4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 -- 172.21.15.83:0/753382179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b7a79540 con 0x5602b79e50a0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a37fe640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 0x7f778c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f7794032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a8ff4e9b9b053 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f778bfff640 1 -- 172.21.15.83:0/1969696370 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f779c0e7610 con 0x7f77a4108d00 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77897fa640 1 -- 172.21.15.83:0/753382179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794040070 con 0x5602b79e50a0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77897fa640 1 -- 172.21.15.83:0/753382179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794032e50 con 0x5602b79e50a0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.238+0000 7f77897fa640 1 -- 172.21.15.83:0/753382179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794035040 con 0x5602b79e50a0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 -- 172.21.15.83:0/753382179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 msgr2=0x5602b7db4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77a37fe640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 0x7f778c0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ad400 tx=0x7f779c082040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f7794032c60 comp rx=0 tx=0).stop 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 -- 172.21.15.83:0/753382179 shutdown_connections 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7db4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7db4440 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 --2- 172.21.15.83:0/753382179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7db3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 -- 172.21.15.83:0/753382179 >> 172.21.15.83:0/753382179 conn(0x5602b7a02350 msgr2=0x5602b7db1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 -- 172.21.15.83:0/753382179 shutdown_connections 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.239+0000 7f77afbee740 1 -- 172.21.15.83:0/753382179 wait complete. 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 -- start start 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 Processor -- start 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b79e50a0 0x5602b7dbf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b7a0df60 0x5602b7dcb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602b7b7c5f0 con 0x5602b79ed280 2024-09-21T03:16:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602b7cb6010 con 0x5602b7a0df60 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dc2680 con 0x5602b79e50a0 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77afbee740 10 client.0 _ll_register_callbacks cb 0x5602b7d15340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47918/0 (socket says 172.21.15.83:47918) 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.240+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3362227675 learned_addr learned my addr 172.21.15.83:0/3362227675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3362227675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b79e50a0 msgr2=0x5602b7dbf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b79e50a0 0x5602b7dbf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b7a0df60 msgr2=0x5602b7dcb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b7a0df60 0x5602b7dcb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3362227675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b7ce3a00 con 0x5602b79ed280 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7794005530 tx=0x7f77940050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc232b62619e0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779404b020 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794032e50 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f7794004040 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7794056050 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77afbee740 10 client.15420 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602b7dce570 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794032e50 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7794004c40 con 0x5602b79ed280 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.241+0000 7f77afbee740 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77afbee740 10 client.15420 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77afbee740 20 client.15420 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79cc8e0 0x5602b7dd1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77afbee740 4 client.15420 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602b7d2db60 con 0x5602b79cc8e0 2024-09-21T03:16:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77a27fc640 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79cc8e0 0x5602b7dd1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f7786ff5640 10 client.15420 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.242+0000 7f77a27fc640 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79cc8e0 0x5602b7dd1bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f1c4bbf78f1456b server_cookie=6e7475f106062435 in_seq=0 out_seq=0 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f7786ff5640 1 -- 172.21.15.83:0/3362227675 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602b7d2db60 con 0x5602b79cc8e0 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f7786ff5640 10 client.15420 handle_command_reply: tid=0 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 client.15420 shutdown 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 msgr2=0x5602b7dbfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7794005530 tx=0x7f77940050e0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 shutdown_connections 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79cc8e0 0x5602b7dd1bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b7a0df60 0x5602b7dcb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dbfda0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 --2- 172.21.15.83:0/3362227675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b79e50a0 0x5602b7dbf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 >> 172.21.15.83:0/3362227675 conn(0x5602b7a02350 msgr2=0x5602b7db7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.243+0000 7f7786ff5640 0 client.15420 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.244+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 shutdown_connections 2024-09-21T03:16:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.244+0000 7f77afbee740 1 -- 172.21.15.83:0/3362227675 wait complete. 2024-09-21T03:16:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.244+0000 7f77afbee740 20 client.15420 trim_cache size 0 max 16384 2024-09-21T03:16:32.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 Processor -- start 2024-09-21T03:16:32.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 -- start start 2024-09-21T03:16:32.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79e50a0 0x5602b7db1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dacad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dcfef0 con 0x5602b79ed280 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dd05c0 con 0x5602b79e50a0 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77afbee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dd0c90 con 0x5602b7a0df60 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47920/0 (socket says 172.21.15.83:47920) 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.262+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3423450997 learned_addr learned my addr 172.21.15.83:0/3423450997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a27fc640 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dacad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a27fc640 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79e50a0 0x5602b7db1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3423450997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 msgr2=0x5602b7dacad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dacad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3423450997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79e50a0 msgr2=0x5602b7db1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79e50a0 0x5602b7db1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 -- 172.21.15.83:0/3423450997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b7db61f0 con 0x5602b79ed280 2024-09-21T03:16:32.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7794006010 tx=0x7f7794031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4673b1619137798c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f7788ff9640 1 -- 172.21.15.83:0/3423450997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779405f070 con 0x5602b79ed280 2024-09-21T03:16:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f7788ff9640 1 -- 172.21.15.83:0/3423450997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794005db0 con 0x5602b79ed280 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.263+0000 7f7788ff9640 1 -- 172.21.15.83:0/3423450997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779404b040 con 0x5602b79ed280 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 -- 172.21.15.83:0/3423450997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 msgr2=0x5602b7dba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7794006010 tx=0x7f7794031400 comp rx=0 tx=0).stop 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 -- 172.21.15.83:0/3423450997 shutdown_connections 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dacad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79ed280 0x5602b7dba020 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 --2- 172.21.15.83:0/3423450997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79e50a0 0x5602b7db1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 -- 172.21.15.83:0/3423450997 >> 172.21.15.83:0/3423450997 conn(0x5602b7a02350 msgr2=0x5602b7dbf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 -- 172.21.15.83:0/3423450997 shutdown_connections 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.264+0000 7f77afbee740 1 -- 172.21.15.83:0/3423450997 wait complete. 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 -- start start 2024-09-21T03:16:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 Processor -- start 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7dd3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dd3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dd1a30 con 0x5602b79e50a0 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dd1360 con 0x5602b79ed280 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b7dd2100 con 0x5602b7a0df60 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77afbee740 10 client.0 _ll_register_callbacks cb 0x7f779800c4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77a27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77a27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47930/0 (socket says 172.21.15.83:47930) 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77a27fc640 1 -- 172.21.15.83:0/2878222448 learned_addr learned my addr 172.21.15.83:0/2878222448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.265+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7dd3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dd3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 -- 172.21.15.83:0/2878222448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 msgr2=0x5602b7dd3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dd3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 -- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 msgr2=0x5602b7dd3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7dd3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 -- 172.21.15.83:0/2878222448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b7dd4670 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77a27fc640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f77a406a890 tx=0x7f779800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1db772a176da35b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f7786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798013110 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f77980132b0 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794005db0 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77980134b0 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77afbee740 10 client.15426 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602b7e3d080 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.266+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794005db0 con 0x5602b79e50a0 2024-09-21T03:16:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7798012a80 con 0x5602b79e50a0 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 10 client.15426 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 20 client.15426 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79a1c90 0x5602b7e40940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 4 client.15426 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "status"}) -- 0x5602b7d29610 con 0x5602b79a1c90 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79a1c90 0x5602b7e40940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f77a2ffd640 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79a1c90 0x5602b7e40940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2933656035bc1b4f server_cookie=53e93bbe1afedb97 in_seq=0 out_seq=0 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.267+0000 7f7786ff5640 10 client.15426 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f7786ff5640 1 -- 172.21.15.83:0/2878222448 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5602b7d29610 con 0x5602b79a1c90 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f7786ff5640 10 client.15426 handle_command_reply: tid=0 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 client.15426 shutdown 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 msgr2=0x5602b7dda590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f77a406a890 tx=0x7f779800cbc0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 shutdown_connections 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5602b79a1c90 0x5602b7e40940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b7a0df60 0x5602b7dd3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5602b79ed280 0x5602b7dd3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 --2- 172.21.15.83:0/2878222448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602b79e50a0 0x5602b7dda590 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f7786ff5640 0 client.15426 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 >> 172.21.15.83:0/2878222448 conn(0x5602b7a02350 msgr2=0x5602b7db4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 shutdown_connections 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 1 -- 172.21.15.83:0/2878222448 wait complete. 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77afbee740 20 client.15426 trim_cache size 0 max 16384 2024-09-21T03:16:32.274 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24620, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 106.432797904, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 135.84650412799999 2024-09-21T03:16:32.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 msgr2=0x7f778c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 0x7f778c0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ad400 tx=0x7f779c082040 comp rx=0 tx=0).stop 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 msgr2=0x7f77a4199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.268+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f779c00eaa0 tx=0x7f779c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 shutdown_connections 2024-09-21T03:16:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f778c0b6560 0x7f778c0b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77a4108d00 0x7f77a4199b60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a41083b0 0x7f77a4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 --2- 172.21.15.83:0/1969696370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a4102ac0 0x7f77a41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 >> 172.21.15.83:0/1969696370 conn(0x7f77a4079720 msgr2=0x7f77a4106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 shutdown_connections 2024-09-21T03:16:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:32.269+0000 7f77a967a640 1 -- 172.21.15.83:0/1969696370 wait complete. 2024-09-21T03:16:32.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:32 smithi193 ceph-mon[35224]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:32 smithi083 ceph-mon[31219]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:32 smithi142 ceph-mon[34512]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:33.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:16:33.251 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 -- 172.21.15.83:0/3497257838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc9810de10 msgr2=0x7efc98110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 --2- 172.21.15.83:0/3497257838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc9810de10 0x7efc98110300 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc90009900 tx=0x7efc9002a410 comp rx=0 tx=0).stop 2024-09-21T03:16:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 -- 172.21.15.83:0/3497257838 shutdown_connections 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 --2- 172.21.15.83:0/3497257838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc9810de10 0x7efc98110300 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 --2- 172.21.15.83:0/3497257838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 --2- 172.21.15.83:0/3497257838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc981049b0 0x7efc98104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 -- 172.21.15.83:0/3497257838 >> 172.21.15.83:0/3497257838 conn(0x7efc98079730 msgr2=0x7efc98079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 -- 172.21.15.83:0/3497257838 shutdown_connections 2024-09-21T03:16:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 -- 172.21.15.83:0/3497257838 wait complete. 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.702+0000 7efca00c7640 1 Processor -- start 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 -- start start 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 0x7efc9819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc98113ad0 con 0x7efc981049b0 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc98113950 con 0x7efc9810de10 2024-09-21T03:16:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efca00c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc98113c50 con 0x7efc98105300 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47948/0 (socket says 172.21.15.83:47948) 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 -- 172.21.15.83:0/2511346351 learned_addr learned my addr 172.21.15.83:0/2511346351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9e664640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 0x7efc9819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 -- 172.21.15.83:0/2511346351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 msgr2=0x7efc9819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9d662640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 -- 172.21.15.83:0/2511346351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 msgr2=0x7efc9819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 0x7efc9819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 -- 172.21.15.83:0/2511346351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc981a03f0 con 0x7efc981049b0 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9d662640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9e664640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 0x7efc9819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.703+0000 7efc9de63640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ccb0 tx=0x7efc8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.704+0000 7efc96ffd640 1 -- 172.21.15.83:0/2511346351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88003990 con 0x7efc981049b0 2024-09-21T03:16:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.704+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc981a06e0 con 0x7efc981049b0 2024-09-21T03:16:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.704+0000 7efc96ffd640 1 -- 172.21.15.83:0/2511346351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc88004020 con 0x7efc981049b0 2024-09-21T03:16:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.704+0000 7efc96ffd640 1 -- 172.21.15.83:0/2511346351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8800f6b0 con 0x7efc981049b0 2024-09-21T03:16:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.704+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc98102650 con 0x7efc981049b0 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.706+0000 7efc96ffd640 1 -- 172.21.15.83:0/2511346351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7efc880044b0 con 0x7efc981049b0 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.706+0000 7efca663b740 1 Processor -- start 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 -- start start 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 0x55cae46d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cae4647c10 con 0x55cae4329f60 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cae4637040 con 0x55cae43010a0 2024-09-21T03:16:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efca663b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cae463ae50 con 0x55cae4309280 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9e664640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc96ffd640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 0x7efc800b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9e664640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47964/0 (socket says 172.21.15.83:47964) 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9e664640 1 -- 172.21.15.83:0/3658749748 learned_addr learned my addr 172.21.15.83:0/3658749748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9de63640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 0x55cae46d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9e664640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 0x7efc800b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.707+0000 7efc9d662640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 -- 172.21.15.83:0/3658749748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 msgr2=0x55cae46d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 -- 172.21.15.83:0/3658749748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 msgr2=0x55cae46d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 0x55cae46d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc96ffd640 1 -- 172.21.15.83:0/2511346351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7efc880e7840 con 0x7efc981049b0 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 -- 172.21.15.83:0/3658749748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae4395540 con 0x55cae4329f60 2024-09-21T03:16:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9de63640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 0x55cae46d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc9e664640 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc90009900 tx=0x7efc90031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c218306e7bd2597 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc7ffff640 1 -- 172.21.15.83:0/3658749748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9004b020 con 0x55cae4329f60 2024-09-21T03:16:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc7ffff640 1 -- 172.21.15.83:0/3658749748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900043b0 con 0x55cae4329f60 2024-09-21T03:16:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.708+0000 7efc7ffff640 1 -- 172.21.15.83:0/3658749748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90035070 con 0x55cae4329f60 2024-09-21T03:16:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- 172.21.15.83:0/3658749748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 msgr2=0x55cae46cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efc9e664640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 0x7efc800b8af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc90049000 tx=0x7efc90048000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc90009900 tx=0x7efc90031760 comp rx=0 tx=0).stop 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- 172.21.15.83:0/3658749748 shutdown_connections 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae43010a0 0x55cae46d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 --2- 172.21.15.83:0/3658749748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae4329f60 0x55cae46cff00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- 172.21.15.83:0/3658749748 >> 172.21.15.83:0/3658749748 conn(0x55cae431e350 msgr2=0x55cae46cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- 172.21.15.83:0/3658749748 shutdown_connections 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- 172.21.15.83:0/3658749748 wait complete. 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.709+0000 7efca663b740 1 -- start start 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 Processor -- start 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 0x55cae46e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cae44985f0 con 0x55cae43010a0 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cae45d2010 con 0x55cae4329f60 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cae46de6e0 con 0x55cae4309280 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efca663b740 10 client.0 _ll_register_callbacks cb 0x55cae4631340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47980/0 (socket says 172.21.15.83:47980) 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 -- 172.21.15.83:0/265630954 learned_addr learned my addr 172.21.15.83:0/265630954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9d662640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9e664640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 0x55cae46e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 -- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 msgr2=0x55cae46dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46dbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.710+0000 7efc9de63640 1 -- 172.21.15.83:0/265630954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 msgr2=0x55cae46e79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc9de63640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 0x55cae46e79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc9de63640 1 -- 172.21.15.83:0/265630954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae45ffa00 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc9e664640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 0x55cae46e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc9de63640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efc880e52a0 tx=0x7efc88076450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd344dc314f7eee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8801d070 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7efc880790c0 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc900043b0 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc880a90a0 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cae46ea630 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900043b0 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7efc880a9280 con 0x55cae43010a0 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:33.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 20 client.15438 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae42e88e0 0x55cae46ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 4 client.15438 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.711+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae4649b60 con 0x55cae42e88e0 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.712+0000 7efc9d662640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae42e88e0 0x55cae46ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.712+0000 7efc7d7fa640 10 client.15438 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.712+0000 7efc9d662640 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae42e88e0 0x55cae46ee010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3ab77e48ded4aa0 server_cookie=f7f730192b14c45c in_seq=0 out_seq=0 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efc7d7fa640 1 -- 172.21.15.83:0/265630954 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cae4649b60 con 0x55cae42e88e0 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efc7d7fa640 10 client.15438 handle_command_reply: tid=0 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 client.15438 shutdown 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 msgr2=0x55cae46db990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efc880e52a0 tx=0x7efc88076450 comp rx=0 tx=0).stop 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 shutdown_connections 2024-09-21T03:16:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae42e88e0 0x55cae46ee010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4329f60 0x55cae46e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efc7d7fa640 0 client.15438 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae4309280 0x55cae46dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 --2- 172.21.15.83:0/265630954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46db990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 >> 172.21.15.83:0/265630954 conn(0x55cae431e350 msgr2=0x55cae46d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 shutdown_connections 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 1 -- 172.21.15.83:0/265630954 wait complete. 2024-09-21T03:16:33.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.713+0000 7efca663b740 20 client.15438 trim_cache size 0 max 16384 2024-09-21T03:16:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 Processor -- start 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 -- start start 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae43010a0 0x55cae46e7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cae46ec350 con 0x55cae42e88e0 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cae46eca20 con 0x55cae4309280 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efca663b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cae46ed0f0 con 0x55cae43010a0 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.731+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47984/0 (socket says 172.21.15.83:47984) 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 -- 172.21.15.83:0/2251662631 learned_addr learned my addr 172.21.15.83:0/2251662631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9e664640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 -- 172.21.15.83:0/2251662631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae43010a0 msgr2=0x55cae46e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9d662640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae43010a0 0x55cae46e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae43010a0 0x55cae46e7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 -- 172.21.15.83:0/2251662631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 msgr2=0x55cae46e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 -- 172.21.15.83:0/2251662631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae46ee340 con 0x55cae42e88e0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9e664640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc9de63640 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f80 tx=0x7efc88004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5234e4f7d476729c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc7f7fe640 1 -- 172.21.15.83:0/2251662631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8801d050 con 0x55cae42e88e0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc7f7fe640 1 -- 172.21.15.83:0/2251662631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc880ad440 con 0x55cae42e88e0 2024-09-21T03:16:33.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.732+0000 7efc7f7fe640 1 -- 172.21.15.83:0/2251662631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880a9280 con 0x55cae42e88e0 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- 172.21.15.83:0/2251662631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 msgr2=0x55cae46d6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f80 tx=0x7efc88004420 comp rx=0 tx=0).stop 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- 172.21.15.83:0/2251662631 shutdown_connections 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae43010a0 0x55cae46e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 --2- 172.21.15.83:0/2251662631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae42e88e0 0x55cae46d6020 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- 172.21.15.83:0/2251662631 >> 172.21.15.83:0/2251662631 conn(0x55cae431e350 msgr2=0x55cae46cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- 172.21.15.83:0/2251662631 shutdown_connections 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- 172.21.15.83:0/2251662631 wait complete. 2024-09-21T03:16:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 -- start start 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.733+0000 7efca663b740 1 Processor -- start 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae42e88e0 0x55cae46f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cae46ede90 con 0x55cae43010a0 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cae46ed7c0 con 0x55cae4309280 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cae46ee560 con 0x55cae42e88e0 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efca663b740 10 client.0 _ll_register_callbacks cb 0x7efc880a98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48000/0 (socket says 172.21.15.83:48000) 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 -- 172.21.15.83:0/2803503727 learned_addr learned my addr 172.21.15.83:0/2803503727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9de63640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae42e88e0 0x55cae46f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9e664640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 -- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae42e88e0 msgr2=0x55cae46f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae42e88e0 0x55cae46f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 -- 172.21.15.83:0/2803503727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 msgr2=0x55cae46f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46f0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.734+0000 7efc9d662640 1 -- 172.21.15.83:0/2803503727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae46f0990 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc9d662640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc98102400 tx=0x7efc8c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386eb749f7cc9693 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0169a0 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7efc8c016b40 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc880ad440 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc8c016d40 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 10 client.15444 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cae46f6300 con 0x55cae43010a0 2024-09-21T03:16:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc880ad440 con 0x55cae43010a0 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7efc8c014410 con 0x55cae43010a0 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 20 client.15444 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae4329f60 0x55cae475cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 4 client.15444 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.735+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "status"}) -- 0x55cae4645610 con 0x55cae4329f60 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efc9de63640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae4329f60 0x55cae475cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efc9de63640 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae4329f60 0x55cae475cb80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33f89cef3e2ca457 server_cookie=14f73a6e925dc699 in_seq=0 out_seq=0 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efc7d7fa640 10 client.15444 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efc7d7fa640 1 -- 172.21.15.83:0/2803503727 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55cae4645610 con 0x55cae4329f60 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efc7d7fa640 10 client.15444 handle_command_reply: tid=0 2024-09-21T03:16:33.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efca663b740 1 client.15444 shutdown 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 msgr2=0x55cae46efe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc98102400 tx=0x7efc8c00cf00 comp rx=0 tx=0).stop 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 shutdown_connections 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.736+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cae4329f60 0x55cae475cb80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cae4309280 0x55cae46f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cae43010a0 0x55cae46efe10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efc7d7fa640 0 client.15444 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 --2- 172.21.15.83:0/2803503727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cae42e88e0 0x55cae46f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 >> 172.21.15.83:0/2803503727 conn(0x55cae431e350 msgr2=0x55cae46cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 shutdown_connections 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 1 -- 172.21.15.83:0/2803503727 wait complete. 2024-09-21T03:16:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca663b740 20 client.15444 trim_cache size 0 max 16384 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24628, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 108.91057596, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 134.70909279099999 2024-09-21T03:16:33.744 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 msgr2=0x7efc800b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 0x7efc800b8af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc90049000 tx=0x7efc90048000 comp rx=0 tx=0).stop 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 msgr2=0x7efc9819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ccb0 tx=0x7efc8800a620 comp rx=0 tx=0).stop 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 shutdown_connections 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efc800b65e0 0x7efc800b8af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc9810de10 0x7efc9819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc98105300 0x7efc9819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 --2- 172.21.15.83:0/2511346351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc981049b0 0x7efc9819a480 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:33.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 >> 172.21.15.83:0/2511346351 conn(0x7efc98079730 msgr2=0x7efc98100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:33.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 shutdown_connections 2024-09-21T03:16:33.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:33.737+0000 7efca00c7640 1 -- 172.21.15.83:0/2511346351 wait complete. 2024-09-21T03:16:34.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:34 smithi083 ceph-mon[31219]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:34.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:34 smithi193 ceph-mon[35224]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:34.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:16:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:34 smithi142 ceph-mon[34512]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T03:16:34.818 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:35.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 -- 172.21.15.83:0/359615716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 msgr2=0x7f2ce8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 --2- 172.21.15.83:0/359615716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce8110620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009900 tx=0x7f2ce002a410 comp rx=0 tx=0).stop 2024-09-21T03:16:35.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 -- 172.21.15.83:0/359615716 shutdown_connections 2024-09-21T03:16:35.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 --2- 172.21.15.83:0/359615716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce8110620 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 --2- 172.21.15.83:0/359615716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce81094a0 0x7f2ce81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 --2- 172.21.15.83:0/359615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 0x7f2ce8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 -- 172.21.15.83:0/359615716 >> 172.21.15.83:0/359615716 conn(0x7f2ce807b850 msgr2=0x7f2ce807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 -- 172.21.15.83:0/359615716 shutdown_connections 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.350+0000 7f2cf0231640 1 -- 172.21.15.83:0/359615716 wait complete. 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 Processor -- start 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 -- start start 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 0x7f2ce819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce81094a0 0x7f2ce819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8113ab0 con 0x7f2ce8109df0 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8113930 con 0x7f2ce8103b60 2024-09-21T03:16:35.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cf0231640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8113c30 con 0x7f2ce81094a0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cee7ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cee7ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48006/0 (socket says 172.21.15.83:48006) 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cee7ce640 1 -- 172.21.15.83:0/855824526 learned_addr learned my addr 172.21.15.83:0/855824526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 0x7f2ce819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cee7ce640 1 -- 172.21.15.83:0/855824526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce81094a0 msgr2=0x7f2ce819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.351+0000 7f2cee7ce640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce81094a0 0x7f2ce819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cee7ce640 1 -- 172.21.15.83:0/855824526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 msgr2=0x7f2ce819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cee7ce640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 0x7f2ce819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cee7ce640 1 -- 172.21.15.83:0/855824526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce81a0350 con 0x7f2ce8109df0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cee7ce640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004770 tx=0x7f2ce0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2ce6ffd640 1 -- 172.21.15.83:0/855824526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce003f030 con 0x7f2ce8109df0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2ce6ffd640 1 -- 172.21.15.83:0/855824526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ce0003aa0 con 0x7f2ce8109df0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce81a05e0 con 0x7f2ce8109df0 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2ce6ffd640 1 -- 172.21.15.83:0/855824526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0035070 con 0x7f2ce8109df0 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.352+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce81a0ac0 con 0x7f2ce8109df0 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.354+0000 7f2cf67a5740 1 Processor -- start 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2ce6ffd640 1 -- 172.21.15.83:0/855824526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2ce0042030 con 0x7f2ce8109df0 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 -- start start 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefb19f60 0x562fefebff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefec0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fefe37c10 con 0x562fefaf9280 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fefe27040 con 0x562fefb19f60 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fefe2ae50 con 0x562fefaf10a0 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48022/0 (socket says 172.21.15.83:48022) 2024-09-21T03:16:35.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2ced7cc640 1 -- 172.21.15.83:0/46003879 learned_addr learned my addr 172.21.15.83:0/46003879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.355+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefb19f60 0x562fefebff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefec0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ce6ffd640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 0x7f2cd00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 -- 172.21.15.83:0/46003879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 msgr2=0x562fefec0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefec0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 -- 172.21.15.83:0/46003879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefb19f60 msgr2=0x562fefebff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefb19f60 0x562fefebff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 -- 172.21.15.83:0/46003879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fefb85540 con 0x562fefaf9280 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2cee7ce640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 0x7f2cd00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005fd0 tx=0x7f2cd80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0c3bd33e662ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ce6ffd640 1 -- 172.21.15.83:0/855824526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2ce0107ca0 con 0x7f2ce8109df0 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ccffff640 1 -- 172.21.15.83:0/46003879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800d8e0 con 0x562fefaf9280 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ccffff640 1 -- 172.21.15.83:0/46003879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd80103d0 con 0x562fefaf9280 2024-09-21T03:16:35.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.356+0000 7f2ccffff640 1 -- 172.21.15.83:0/46003879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800f620 con 0x562fefaf9280 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 -- 172.21.15.83:0/46003879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 msgr2=0x562fefec0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cee7ce640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 0x7f2cd00b8aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00cb000 tx=0x7f2ce009b070 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005fd0 tx=0x7f2cd80095e0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 -- 172.21.15.83:0/46003879 shutdown_connections 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefaf9280 0x562fefec0980 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefec0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/46003879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefb19f60 0x562fefebff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 -- 172.21.15.83:0/46003879 >> 172.21.15.83:0/46003879 conn(0x562fefb0e350 msgr2=0x562fefebddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 -- 172.21.15.83:0/46003879 shutdown_connections 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.357+0000 7f2cf67a5740 1 -- 172.21.15.83:0/46003879 wait complete. 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 -- start start 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 Processor -- start 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefecb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 0x562fefecbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fefc885f0 con 0x562fefb19f60 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fefdc2010 con 0x562fefaf9280 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fefece6e0 con 0x562fefaf10a0 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cf67a5740 10 client.0 _ll_register_callbacks cb 0x562fefe21340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48024/0 (socket says 172.21.15.83:48024) 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3274438116 learned_addr learned my addr 172.21.15.83:0/3274438116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 0x562fefecbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.358+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefecb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3274438116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 msgr2=0x562fefecb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefecb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 msgr2=0x562fefecbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 0x562fefecbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3274438116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fefdefa00 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 0x562fefecbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009e70 tx=0x7f2cd8010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff36b22fc301e9cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8010780 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2cd800d6e0 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd80103d0 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2cd8035050 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2cf67a5740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fefeda600 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd80103d0 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.359+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2cd802d020 con 0x562fefb19f60 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 10 client.15456 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 20 client.15456 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefad88e0 0x562fefeddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 4 client.15456 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fefe39b60 con 0x562fefad88e0 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefad88e0 0x562fefeddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2ccd7fa640 10 client.15456 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.360+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefad88e0 0x562fefeddbc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82028230dd5ac46b server_cookie=e82c5c11cff3d53 in_seq=0 out_seq=0 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/3274438116 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fefe39b60 con 0x562fefad88e0 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2ccd7fa640 10 client.15456 handle_command_reply: tid=0 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 client.15456 shutdown 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 msgr2=0x562fefed7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009e70 tx=0x7f2cd8010e10 comp rx=0 tx=0).stop 2024-09-21T03:16:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 shutdown_connections 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefad88e0 0x562fefeddbc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7a00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf9280 0x562fefecbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2ccd7fa640 0 client.15456 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3274438116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf10a0 0x562fefecb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 >> 172.21.15.83:0/3274438116 conn(0x562fefb0e350 msgr2=0x562fefec3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 shutdown_connections 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.361+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3274438116 wait complete. 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.362+0000 7f2cf67a5740 20 client.15456 trim_cache size 0 max 16384 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 Processor -- start 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 -- start start 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefec6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefed7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fefedbf00 con 0x562fefb19f60 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fefedc5d0 con 0x562fefaf10a0 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fefedcca0 con 0x562fefaf9280 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.379+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48030/0 (socket says 172.21.15.83:48030) 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3307396243 learned_addr learned my addr 172.21.15.83:0/3307396243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefec6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3307396243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 msgr2=0x562fefed7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefed7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3307396243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 msgr2=0x562fefec6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefec6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 -- 172.21.15.83:0/3307396243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fefeb9520 con 0x562fefb19f60 2024-09-21T03:16:35.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8002210 tx=0x7f2cd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759f6a8d139cb4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ccf7fe640 1 -- 172.21.15.83:0/3307396243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8009ca0 con 0x562fefb19f60 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ccf7fe640 1 -- 172.21.15.83:0/3307396243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd802a040 con 0x562fefb19f60 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.380+0000 7f2ccf7fe640 1 -- 172.21.15.83:0/3307396243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8010630 con 0x562fefb19f60 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3307396243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 msgr2=0x562fefed7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8002210 tx=0x7f2cd8005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3307396243 shutdown_connections 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefed7eb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefed7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/3307396243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefec6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3307396243 >> 172.21.15.83:0/3307396243 conn(0x562fefb0e350 msgr2=0x562fefec6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3307396243 shutdown_connections 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- 172.21.15.83:0/3307396243 wait complete. 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 -- start start 2024-09-21T03:16:35.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.381+0000 7f2cf67a5740 1 Processor -- start 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefee6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefedfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fefedda40 con 0x562fefb19f60 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fefedd370 con 0x562fefaf10a0 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fefede110 con 0x562fefaf9280 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 10 client.0 _ll_register_callbacks cb 0x7f2cd80123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48042/0 (socket says 172.21.15.83:48042) 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 -- 172.21.15.83:0/899771952 learned_addr learned my addr 172.21.15.83:0/899771952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefee6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefedfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 -- 172.21.15.83:0/899771952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 msgr2=0x562fefedfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefedfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 -- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 msgr2=0x562fefee6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefee6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 -- 172.21.15.83:0/899771952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fefee0a50 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ced7cc640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800fa40 tx=0x7f2cd80095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add439ddd8fb278 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8003970 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd802a040 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.382+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2cd8003b10 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2cd8003d10 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562feff49150 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd802a040 con 0x562fefb19f60 2024-09-21T03:16:35.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2cd802d070 con 0x562fefb19f60 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 20 client.15462 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefaadc90 0x562feff4c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 4 client.15462 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "status"}) -- 0x562fefe35610 con 0x562fefaadc90 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.383+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefaadc90 0x562feff4c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5108497e-77c6-11ef-baf5-efdc52797490", 2024-09-21T03:16:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24647, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 117.78666492399999, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 26, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 77, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 127.15299405499999 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cedfcd640 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefaadc90 0x562feff4c720 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e85836e84dae095 server_cookie=e18a5aa7fe65e60c in_seq=0 out_seq=0 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2ccd7fa640 10 client.15462 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2ccd7fa640 1 -- 172.21.15.83:0/899771952 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562fefe35610 con 0x562fefaadc90 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2ccd7fa640 10 client.15462 handle_command_reply: tid=0 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 client.15462 shutdown 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 msgr2=0x562fefee02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800fa40 tx=0x7f2cd80095b0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 shutdown_connections 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562fefaadc90 0x562feff4c720 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fefb19f60 0x562fefee02c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2ccd7fa640 0 client.15462 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fefaf9280 0x562fefedfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 --2- 172.21.15.83:0/899771952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fefaf10a0 0x562fefee6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 >> 172.21.15.83:0/899771952 conn(0x562fefb0e350 msgr2=0x562fefebd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 shutdown_connections 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 1 -- 172.21.15.83:0/899771952 wait complete. 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.384+0000 7f2cf67a5740 20 client.15462 trim_cache size 0 max 16384 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 msgr2=0x7f2cd00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 0x7f2cd00b8aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00cb000 tx=0x7f2ce009b070 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 msgr2=0x7f2ce819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004770 tx=0x7f2ce0004270 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 shutdown_connections 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2cd00b6590 0x7f2cd00b8aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ce8109df0 0x7f2ce819fce0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce81094a0 0x7f2ce819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 --2- 172.21.15.83:0/855824526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ce8103b60 0x7f2ce819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 >> 172.21.15.83:0/855824526 conn(0x7f2ce807b850 msgr2=0x7f2ce8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 shutdown_connections 2024-09-21T03:16:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:35.385+0000 7f2cf0231640 1 -- 172.21.15.83:0/855824526 wait complete. 2024-09-21T03:16:36.082 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:16:36.113 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:16:36.113 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T03:16:36.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:36.113 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-09-21T03:16:36.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:36.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-09-21T03:16:36.196 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-09-21T03:16:36.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:36.198 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-21T03:16:36.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:16:36.198 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-21T03:16:36.199 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-21T03:16:36.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:16:36.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:36 smithi083 ceph-mon[31219]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:36.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:36 smithi193 ceph-mon[35224]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:36 smithi142 ceph-mon[34512]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 -- 172.21.15.83:0/3989596487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec10de90 msgr2=0x7f01ec110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 --2- 172.21.15.83:0/3989596487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec10de90 0x7f01ec110380 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009ca0 tx=0x7f01e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 -- 172.21.15.83:0/3989596487 shutdown_connections 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 --2- 172.21.15.83:0/3989596487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec10de90 0x7f01ec110380 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 --2- 172.21.15.83:0/3989596487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec105310 0x7f01ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 --2- 172.21.15.83:0/3989596487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01ec1048e0 0x7f01ec104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 -- 172.21.15.83:0/3989596487 >> 172.21.15.83:0/3989596487 conn(0x7f01ec079770 msgr2=0x7f01ec079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 -- 172.21.15.83:0/3989596487 shutdown_connections 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.981+0000 7f01f3710640 1 -- 172.21.15.83:0/3989596487 wait complete. 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 Processor -- start 2024-09-21T03:16:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 -- start start 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec105310 0x7f01ec19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01ec10de90 0x7f01ec19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec113af0 con 0x7f01ec1048e0 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec113970 con 0x7f01ec10de90 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f3710640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec113c70 con 0x7f01ec105310 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48046/0 (socket says 172.21.15.83:48046) 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 -- 172.21.15.83:0/686303601 learned_addr learned my addr 172.21.15.83:0/686303601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 -- 172.21.15.83:0/686303601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec105310 msgr2=0x7f01ec19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec105310 0x7f01ec19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 -- 172.21.15.83:0/686303601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01ec10de90 msgr2=0x7f01ec19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01ec10de90 0x7f01ec19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 -- 172.21.15.83:0/686303601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec1a04b0 con 0x7f01ec1048e0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.982+0000 7f01f14ac640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d950 tx=0x7f01dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.983+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc017030 con 0x7f01ec1048e0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.983+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01dc004020 con 0x7f01ec1048e0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.983+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec1a07a0 con 0x7f01ec1048e0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.983+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0047a0 con 0x7f01ec1048e0 2024-09-21T03:16:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.983+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1a6f00 con 0x7f01ec1048e0 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.984+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec06c1e0 con 0x7f01ec1048e0 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.985+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f01dc009a30 con 0x7f01ec1048e0 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.986+0000 7f01ea7fc640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 0x7f01d40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.986+0000 7f01f0cab640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 0x7f01d40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.986+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f01dc0e6ea0 con 0x7f01ec1048e0 2024-09-21T03:16:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.987+0000 7f01f0cab640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 0x7f01d40b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01ec100170 tx=0x7f01e0005b60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:36.991+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01dc0ad690 con 0x7f01ec1048e0 2024-09-21T03:16:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.206+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f01ec104cf0 con 0x7f01ec1048e0 2024-09-21T03:16:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.207+0000 7f01ea7fc640 1 -- 172.21.15.83:0/686303601 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f01dc0b1a70 con 0x7f01ec1048e0 2024-09-21T03:16:37.208 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 msgr2=0x7f01d40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 0x7f01d40b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01ec100170 tx=0x7f01e0005b60 comp rx=0 tx=0).stop 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 msgr2=0x7f01ec19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d950 tx=0x7f01dc00de10 comp rx=0 tx=0).stop 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 shutdown_connections 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f01d40b65e0 0x7f01d40b8af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01ec10de90 0x7f01ec19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec105310 0x7f01ec19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 --2- 172.21.15.83:0/686303601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec1048e0 0x7f01ec19a540 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 >> 172.21.15.83:0/686303601 conn(0x7f01ec079770 msgr2=0x7f01ec107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 shutdown_connections 2024-09-21T03:16:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:37.209+0000 7f01f3710640 1 -- 172.21.15.83:0/686303601 wait complete. 2024-09-21T03:16:37.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:37 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/686303601' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:16:37.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:37 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/686303601' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:16:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:37 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/686303601' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:16:38.004 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:16:38.004 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:16:38.004 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-09-21T03:16:38.004 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:16:38.005 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-21T03:16:38.005 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:16:38.005 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:16:38.005 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:16:38.005 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-21T03:16:38.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:38.005 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:16:38.023 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:16:38.024 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:16:38.025 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:16:38.025 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:16:38.025 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42135sec preferred_lft 42135sec 2024-09-21T03:16:38.025 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link noprefixroute 2024-09-21T03:16:38.025 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:16:38.025 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:16:38.025 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:16:38.025 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-09-21T03:16:38.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:16:38.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-09-21T03:16:38.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-09-21T03:16:38.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:16:38.026 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-09-21T03:16:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:38.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:38.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:38.238 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:16:38.262 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-09-21T03:16:38.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:38.264 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-09-21T03:16:38.329 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-09-21T03:16:38.330 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:16:38.330 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:16:38.330 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:16:38.330 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:16:38.330 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:16:38.330 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-09-21T03:16:38.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:16:38.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:16:38.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:16:38.331 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-09-21T03:16:38.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:38.518 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:38 smithi083 ceph-mon[31219]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:38 smithi193 ceph-mon[35224]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:38.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:38.623 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-09-21T03:16:38.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:38.639 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-09-21T03:16:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:38 smithi142 ceph-mon[34512]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T03:16:38.693 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:16:38.694 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-09-21T03:16:38.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-21T03:16:38.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-21T03:16:38.694 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-09-21T03:16:38.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:38.810 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-21T03:16:38.810 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:16:38.810 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-21T03:16:38.811 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-09-21T03:16:38.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:38.939 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:16:38.939 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-09-21T03:16:38.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-09-21T03:16:38.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:16:38.940 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-09-21T03:16:38.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:39.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:16:39.016 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /tmp/cephfs 2024-09-21T03:16:39.031 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-21T03:16:39.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:16:39.032 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /tmp/cephfs 2024-09-21T03:16:39.086 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T03:16:39.087 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T03:16:39.087 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/5a50105c-b77d-4a93-a145-d0128cc748da 2024-09-21T03:16:39.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:39.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/5a50105c-b77d-4a93-a145-d0128cc748da /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-09-21T03:16:39.200 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-21T03:16:39.200 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T03:16:39.329 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-21T03:16:39.329 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T03:16:39.329 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.142:3300,172.21.15.193:3300" 2024-09-21T03:16:39.329 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@5108497e-77c6-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/5a50105c-b77d-4a93-a145-d0128cc748da 2024-09-21T03:16:39.329 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.83:3300/172.21.15.142:3300/172.21.15.193:3300" will pass to kernel 2024-09-21T03:16:39.331 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:16:39.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:16:39.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /tmp/cephfs 2024-09-21T03:16:39.360 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:16:39.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:16:39.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:39.457 INFO:teuthology.orchestra.run.smithi083.stdout:client25007 2024-09-21T03:16:39.457 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:16:39.458 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:16:39.474 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:16:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:16:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:39.570 INFO:teuthology.orchestra.run.smithi083.stdout:client25007 2024-09-21T03:16:39.570 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:16:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:16:39.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:39.626 INFO:teuthology.orchestra.run.smithi083.stdout:client25007 2024-09-21T03:16:39.627 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5108497e-77c6-11ef-baf5-efdc52797490.client25007/status') 2024-09-21T03:16:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:16:39.685 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.25007 (3)172.21.15.83:0/13344951 2024-09-21T03:16:39.685 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-09-21T03:16:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-09-21T03:16:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-09-21T03:16:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:69 bytes copied, 4.5581e-05 s, 1.5 MB/s 2024-09-21T03:16:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:16:39.692 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:16:39.704 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:16:39.704 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:16:39.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 -- 172.21.15.83:0/729413601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109400 msgr2=0x7f5738109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 --2- 172.21.15.83:0/729413601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109400 0x7f5738109810 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f572c009900 tx=0x7f572c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 -- 172.21.15.83:0/729413601 shutdown_connections 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 --2- 172.21.15.83:0/729413601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738109d50 0x7f5738110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 --2- 172.21.15.83:0/729413601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109400 0x7f5738109810 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 --2- 172.21.15.83:0/729413601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738103ad0 0x7f5738103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:39.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 -- 172.21.15.83:0/729413601 >> 172.21.15.83:0/729413601 conn(0x7f57380ff510 msgr2=0x7f5738101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 -- 172.21.15.83:0/729413601 shutdown_connections 2024-09-21T03:16:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 -- 172.21.15.83:0/729413601 wait complete. 2024-09-21T03:16:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.814+0000 7f573d6be640 1 Processor -- start 2024-09-21T03:16:39.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 -- start start 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738103ad0 0x7f5738075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738109400 0x7f5738075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5738113a10 con 0x7f5738109d50 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5738113890 con 0x7f5738109400 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f573d6be640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5738113b90 con 0x7f5738103ad0 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60372/0 (socket says 172.21.15.83:60372) 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 -- 172.21.15.83:0/947021376 learned_addr learned my addr 172.21.15.83:0/947021376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f5736ffd640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738103ad0 0x7f5738075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57367fc640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738109400 0x7f5738075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 -- 172.21.15.83:0/947021376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738103ad0 msgr2=0x7f5738075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738103ad0 0x7f5738075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 -- 172.21.15.83:0/947021376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738109400 msgr2=0x7f5738075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738109400 0x7f5738075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.815+0000 7f57377fe640 1 -- 172.21.15.83:0/947021376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57381a86a0 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f57377fe640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f573000cce0 tx=0x7f573000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573000f450 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f573000faa0 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57381a8990 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730018b70 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.816+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381a8e60 con 0x7f5738109d50 2024-09-21T03:16:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.817+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5738103ee0 con 0x7f5738109d50 2024-09-21T03:16:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.819+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f5730018d10 con 0x7f5738109d50 2024-09-21T03:16:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.820+0000 7f571ffff640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 0x7f57200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.820+0000 7f5736ffd640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 0x7f57200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.820+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f57300e7850 con 0x7f5738109d50 2024-09-21T03:16:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.820+0000 7f5736ffd640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 0x7f57200b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5738077160 tx=0x7f5728005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:39.823+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5730014030 con 0x7f5738109d50 2024-09-21T03:16:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.039+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f5738076a10 con 0x7f5738109d50 2024-09-21T03:16:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.270+0000 7f571ffff640 1 -- 172.21.15.83:0/947021376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f5738076a10 con 0x7f5738109d50 2024-09-21T03:16:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.273+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 msgr2=0x7f57200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.273+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 0x7f57200b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5738077160 tx=0x7f5728005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.273+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 msgr2=0x7f5738076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.273+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f573000cce0 tx=0x7f573000a620 comp rx=0 tx=0).stop 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 shutdown_connections 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57200b6560 0x7f57200b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738109d50 0x7f5738076190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738109400 0x7f5738075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 --2- 172.21.15.83:0/947021376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5738103ad0 0x7f5738075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 >> 172.21.15.83:0/947021376 conn(0x7f57380ff510 msgr2=0x7f5738101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 shutdown_connections 2024-09-21T03:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.274+0000 7f573d6be640 1 -- 172.21.15.83:0/947021376 wait complete. 2024-09-21T03:16:40.289 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:16:40.299 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T03:16:40.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- 172.21.15.83:0/639450323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 msgr2=0x7f67381086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 --2- 172.21.15.83:0/639450323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f67381086d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f672c009900 tx=0x7f672c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- 172.21.15.83:0/639450323 shutdown_connections 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 --2- 172.21.15.83:0/639450323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 0x7f673810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 --2- 172.21.15.83:0/639450323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f67381086d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 --2- 172.21.15.83:0/639450323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 0x7f6738102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- 172.21.15.83:0/639450323 >> 172.21.15.83:0/639450323 conn(0x7f67380fe350 msgr2=0x7f67381007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- 172.21.15.83:0/639450323 shutdown_connections 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- 172.21.15.83:0/639450323 wait complete. 2024-09-21T03:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 Processor -- start 2024-09-21T03:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.479+0000 7f673ff89640 1 -- start start 2024-09-21T03:16:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 0x7f673819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 0x7f673819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67381128d0 con 0x7f67381082c0 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6738112750 con 0x7f6738108c10 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6738112a50 con 0x7f6738102990 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60396/0 (socket says 172.21.15.83:60396) 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 -- 172.21.15.83:0/3826987337 learned_addr learned my addr 172.21.15.83:0/3826987337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673dd25640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 0x7f673819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673e526640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 0x7f673819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 -- 172.21.15.83:0/3826987337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 msgr2=0x7f673819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 0x7f673819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 -- 172.21.15.83:0/3826987337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 msgr2=0x7f673819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 0x7f673819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 -- 172.21.15.83:0/3826987337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381a64c0 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673d524640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f672c02a930 tx=0x7f672c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c03f030 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381a6750 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f672c003aa0 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.480+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c035070 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.481+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381a6c90 con 0x7f67381082c0 2024-09-21T03:16:40.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.481+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738102da0 con 0x7f67381082c0 2024-09-21T03:16:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.483+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f672c042030 con 0x7f67381082c0 2024-09-21T03:16:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.484+0000 7f672affd640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 0x7f67240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.484+0000 7f673dd25640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 0x7f67240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.484+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f672c107fd0 con 0x7f67381082c0 2024-09-21T03:16:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.484+0000 7f673dd25640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 0x7f67240b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6734005fd0 tx=0x7f6734005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.487+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f672c0ce7c0 con 0x7f67381082c0 2024-09-21T03:16:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:40 smithi193 ceph-mon[35224]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:40 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:16:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:40 smithi083 ceph-mon[31219]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:40 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:16:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.664+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67381a7010 con 0x7f67381082c0 2024-09-21T03:16:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.665+0000 7f672affd640 1 -- 172.21.15.83:0/3826987337 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7f672c0d2ba0 con 0x7f67381082c0 2024-09-21T03:16:40.667 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:40.667 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:16:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 msgr2=0x7f67240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 0x7f67240b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6734005fd0 tx=0x7f6734005e20 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 msgr2=0x7f673819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f672c02a930 tx=0x7f672c004270 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 shutdown_connections 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f67240b6560 0x7f67240b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6738108c10 0x7f673819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67381082c0 0x7f673819b740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 --2- 172.21.15.83:0/3826987337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6738102990 0x7f673819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 >> 172.21.15.83:0/3826987337 conn(0x7f67380fe350 msgr2=0x7f6738104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 shutdown_connections 2024-09-21T03:16:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.668+0000 7f673ff89640 1 -- 172.21.15.83:0/3826987337 wait complete. 2024-09-21T03:16:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:40 smithi142 ceph-mon[34512]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T03:16:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:40 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:16:40.681 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:09:57.414337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:14:33.119862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:14:35.204610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:16:40.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:16:40.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.792+0000 7f61cc6b4640 1 -- 172.21.15.83:0/3267988198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 msgr2=0x7f61c41065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.792+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/3267988198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c41065d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009900 tx=0x7f61b402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 -- 172.21.15.83:0/3267988198 shutdown_connections 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/3267988198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4106b10 0x7f61c41105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/3267988198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c41065d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/3267988198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4101fe0 0x7f61c41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 -- 172.21.15.83:0/3267988198 >> 172.21.15.83:0/3267988198 conn(0x7f61c4079e10 msgr2=0x7f61c407a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 -- 172.21.15.83:0/3267988198 shutdown_connections 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 -- 172.21.15.83:0/3267988198 wait complete. 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 Processor -- start 2024-09-21T03:16:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 -- start start 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.793+0000 7f61cc6b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4101fe0 0x7f61c419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4106b10 0x7f61c419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41139e0 con 0x7f61c41061c0 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4113860 con 0x7f61c4101fe0 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4113b60 con 0x7f61c4106b10 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60410/0 (socket says 172.21.15.83:60410) 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 -- 172.21.15.83:0/2602948126 learned_addr learned my addr 172.21.15.83:0/2602948126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61ca450640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4101fe0 0x7f61c419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cac51640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4106b10 0x7f61c419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 -- 172.21.15.83:0/2602948126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4106b10 msgr2=0x7f61c419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4106b10 0x7f61c419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 -- 172.21.15.83:0/2602948126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4101fe0 msgr2=0x7f61c419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4101fe0 0x7f61c419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 -- 172.21.15.83:0/2602948126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c41a0370 con 0x7f61c41061c0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61c9c4f640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f61b402a930 tx=0x7f61b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b403f030 con 0x7f61c41061c0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b4003aa0 con 0x7f61c41061c0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c41a0600 con 0x7f61c41061c0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4035070 con 0x7f61c41061c0 2024-09-21T03:16:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.794+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c41a6df0 con 0x7f61c41061c0 2024-09-21T03:16:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.795+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c406b590 con 0x7f61c41061c0 2024-09-21T03:16:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.797+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f61b4042030 con 0x7f61c41061c0 2024-09-21T03:16:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.797+0000 7f61bb7fe640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 0x7f61b00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.798+0000 7f61ca450640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 0x7f61b00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.798+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f61b4107e80 con 0x7f61c41061c0 2024-09-21T03:16:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.798+0000 7f61ca450640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 0x7f61b00b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005fd0 tx=0x7f61c000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:40.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.801+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61b40d2a50 con 0x7f61c41061c0 2024-09-21T03:16:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.983+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61c410bba0 con 0x7f61c41061c0 2024-09-21T03:16:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.983+0000 7f61bb7fe640 1 -- 172.21.15.83:0/2602948126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f61b40d7990 con 0x7f61c41061c0 2024-09-21T03:16:40.985 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 msgr2=0x7f61b00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 0x7f61b00b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005fd0 tx=0x7f61c000b390 comp rx=0 tx=0).stop 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 msgr2=0x7f61c419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f61b402a930 tx=0x7f61b4004270 comp rx=0 tx=0).stop 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 shutdown_connections 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f61b00b6560 0x7f61b00b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c4106b10 0x7f61c419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c41061c0 0x7f61c419a9e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 --2- 172.21.15.83:0/2602948126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61c4101fe0 0x7f61c419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 >> 172.21.15.83:0/2602948126 conn(0x7f61c4079e10 msgr2=0x7f61c4109560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 shutdown_connections 2024-09-21T03:16:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:40.985+0000 7f61cc6b4640 1 -- 172.21.15.83:0/2602948126 wait complete. 2024-09-21T03:16:40.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 -- 172.21.15.83:0/563663701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 msgr2=0x7f4e58102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 --2- 172.21.15.83:0/563663701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e58102e30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009900 tx=0x7f4e4802a410 comp rx=0 tx=0).stop 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 -- 172.21.15.83:0/563663701 shutdown_connections 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 --2- 172.21.15.83:0/563663701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 --2- 172.21.15.83:0/563663701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e58108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 --2- 172.21.15.83:0/563663701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e58102e30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 -- 172.21.15.83:0/563663701 >> 172.21.15.83:0/563663701 conn(0x7f4e58079630 msgr2=0x7f4e58079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 -- 172.21.15.83:0/563663701 shutdown_connections 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.109+0000 7f4e5f007640 1 -- 172.21.15.83:0/563663701 wait complete. 2024-09-21T03:16:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 Processor -- start 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 -- start start 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e5819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58113a30 con 0x7f4e58102a20 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581138b0 con 0x7f4e58108300 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5f007640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58113bb0 con 0x7f4e58108c50 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60432/0 (socket says 172.21.15.83:60432) 2024-09-21T03:16:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 -- 172.21.15.83:0/3539363481 learned_addr learned my addr 172.21.15.83:0/3539363481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e57fff640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e5819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5d5a4640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 -- 172.21.15.83:0/3539363481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 msgr2=0x7f4e5819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 -- 172.21.15.83:0/3539363481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 msgr2=0x7f4e5819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e5819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 -- 172.21.15.83:0/3539363481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581a0380 con 0x7f4e58102a20 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e57fff640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e5819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5d5a4640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.110+0000 7f4e5cda3640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005940 tx=0x7f4e480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.111+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4803f030 con 0x7f4e58102a20 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.111+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e48003cb0 con 0x7f4e58102a20 2024-09-21T03:16:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.111+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581a0610 con 0x7f4e58102a20 2024-09-21T03:16:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.111+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48035070 con 0x7f4e58102a20 2024-09-21T03:16:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.111+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a6e00 con 0x7f4e58102a20 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.112+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58102e30 con 0x7f4e58102a20 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.113+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4e48042030 con 0x7f4e58102a20 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.114+0000 7f4e55ffb640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 0x7f4e400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.114+0000 7f4e57fff640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 0x7f4e400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.114+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4e48108020 con 0x7f4e58102a20 2024-09-21T03:16:41.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.115+0000 7f4e57fff640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 0x7f4e400b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0058c0 tx=0x7f4e4c005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:41.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.117+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e480ce810 con 0x7f4e58102a20 2024-09-21T03:16:41.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:16:41.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:41 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:41.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3826987337' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:41.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2602948126' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.301+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e5802e640 con 0x7f4e58102a20 2024-09-21T03:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.301+0000 7f4e55ffb640 1 -- 172.21.15.83:0/3539363481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f4e480d2bf0 con 0x7f4e58102a20 2024-09-21T03:16:41.303 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-09-21T03:16:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 msgr2=0x7f4e400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 0x7f4e400b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0058c0 tx=0x7f4e4c005850 comp rx=0 tx=0).stop 2024-09-21T03:16:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 msgr2=0x7f4e5819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005940 tx=0x7f4e480042f0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 shutdown_connections 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4e400b6560 0x7f4e400b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e58108c50 0x7f4e5819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e58108300 0x7f4e5819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 --2- 172.21.15.83:0/3539363481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e58102a20 0x7f4e5819a450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 >> 172.21.15.83:0/3539363481 conn(0x7f4e58079630 msgr2=0x7f4e581063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 shutdown_connections 2024-09-21T03:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.304+0000 7f4e5f007640 1 -- 172.21.15.83:0/3539363481 wait complete. 2024-09-21T03:16:41.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 -- 172.21.15.83:0/4068783544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 msgr2=0x7f25b4103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/4068783544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b4103ec0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009900 tx=0x7f25a402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 -- 172.21.15.83:0/4068783544 shutdown_connections 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/4068783544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 0x7f25b4110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/4068783544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 0x7f25b41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/4068783544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b4103ec0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 -- 172.21.15.83:0/4068783544 >> 172.21.15.83:0/4068783544 conn(0x7f25b40ff510 msgr2=0x7f25b4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 -- 172.21.15.83:0/4068783544 shutdown_connections 2024-09-21T03:16:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 -- 172.21.15.83:0/4068783544 wait complete. 2024-09-21T03:16:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.428+0000 7f25ba1ed640 1 Processor -- start 2024-09-21T03:16:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 -- start start 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 0x7f25b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 0x7f25b419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25b41139f0 con 0x7f25b4103ab0 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4113870 con 0x7f25b41093e0 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25ba1ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4113b70 con 0x7f25b4109d30 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60450/0 (socket says 172.21.15.83:60450) 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 -- 172.21.15.83:0/418240727 learned_addr learned my addr 172.21.15.83:0/418240727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b2ffd640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 0x7f25b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b3fff640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 0x7f25b419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 -- 172.21.15.83:0/418240727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 msgr2=0x7f25b419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 0x7f25b419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 -- 172.21.15.83:0/418240727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 msgr2=0x7f25b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 0x7f25b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.429+0000 7f25b37fe640 1 -- 172.21.15.83:0/418240727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41a0380 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25b37fe640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005940 tx=0x7f25a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a403f030 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a4003cb0 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41a0610 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4035070 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.430+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b41a6e00 con 0x7f25b4103ab0 2024-09-21T03:16:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.431+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b4104d20 con 0x7f25b4103ab0 2024-09-21T03:16:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.432+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f25a4042030 con 0x7f25b4103ab0 2024-09-21T03:16:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.433+0000 7f25b0ff9640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 0x7f259c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.433+0000 7f25b2ffd640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 0x7f259c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.433+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f25a4107cc0 con 0x7f25b4103ab0 2024-09-21T03:16:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.433+0000 7f25b2ffd640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 0x7f259c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25b4108f60 tx=0x7f25a800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.437+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25a40ce4b0 con 0x7f25b4103ab0 2024-09-21T03:16:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:16:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:41 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3826987337' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2602948126' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:16:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.617+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25b410bb00 con 0x7f25b4103ab0 2024-09-21T03:16:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.617+0000 7f25b0ff9640 1 -- 172.21.15.83:0/418240727 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f25a40d2890 con 0x7f25b4103ab0 2024-09-21T03:16:41.620 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-09-21T03:16:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 msgr2=0x7f259c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 0x7f259c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25b4108f60 tx=0x7f25a800b410 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 msgr2=0x7f25b419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005940 tx=0x7f25a40042f0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 shutdown_connections 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f259c0b6560 0x7f259c0b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4109d30 0x7f25b419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b41093e0 0x7f25b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 --2- 172.21.15.83:0/418240727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25b4103ab0 0x7f25b419a450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 >> 172.21.15.83:0/418240727 conn(0x7f25b40ff510 msgr2=0x7f25b4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 shutdown_connections 2024-09-21T03:16:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:41.620+0000 7f25ba1ed640 1 -- 172.21.15.83:0/418240727 wait complete. 2024-09-21T03:16:41.633 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T03:16:41.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:16:41.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/947021376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:16:41.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:41 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:41.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3826987337' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:41.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2602948126' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:16:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 -- 172.21.15.83:0/3462278780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 msgr2=0x7fccfc103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3462278780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc103fd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fccec009ca0 tx=0x7fccec02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 -- 172.21.15.83:0/3462278780 shutdown_connections 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3462278780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc109e00 0x7fccfc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3462278780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc1094b0 0x7fccfc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3462278780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc103fd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 -- 172.21.15.83:0/3462278780 >> 172.21.15.83:0/3462278780 conn(0x7fccfc07b910 msgr2=0x7fccfc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 -- 172.21.15.83:0/3462278780 shutdown_connections 2024-09-21T03:16:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.388+0000 7fcd030fa640 1 -- 172.21.15.83:0/3462278780 wait complete. 2024-09-21T03:16:42.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 Processor -- start 2024-09-21T03:16:42.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 -- start start 2024-09-21T03:16:42.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 0x7fccfc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 0x7fccfc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113ac0 con 0x7fccfc103bc0 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113940 con 0x7fccfc1094b0 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd030fa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc113c40 con 0x7fccfc109e00 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.389+0000 7fcd00e96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60466/0 (socket says 172.21.15.83:60466) 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 -- 172.21.15.83:0/3344264765 learned_addr learned my addr 172.21.15.83:0/3344264765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd01697640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 0x7fccfc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fccfbfff640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 0x7fccfc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 -- 172.21.15.83:0/3344264765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 msgr2=0x7fccfc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:42.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 0x7fccfc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 -- 172.21.15.83:0/3344264765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 msgr2=0x7fccfc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 0x7fccfc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 -- 172.21.15.83:0/3344264765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1a03e0 con 0x7fccfc103bc0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fccfbfff640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 0x7fccfc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd01697640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 0x7fccfc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd00e96640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fccec033040 tx=0x7fccec0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec003b70 con 0x7fccfc103bc0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec003d10 con 0x7fccfc103bc0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec02f9e0 con 0x7fccfc103bc0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.390+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1a0670 con 0x7fccfc103bc0 2024-09-21T03:16:42.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.391+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1a6e30 con 0x7fccfc103bc0 2024-09-21T03:16:42.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.391+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc104df0 con 0x7fccfc103bc0 2024-09-21T03:16:42.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.393+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fccec004b40 con 0x7fccfc103bc0 2024-09-21T03:16:42.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.394+0000 7fccf9ffb640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 0x7fcce80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:42.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.394+0000 7fccfbfff640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 0x7fcce80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:42.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.394+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fccec1088d0 con 0x7fccfc103bc0 2024-09-21T03:16:42.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.394+0000 7fccfbfff640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 0x7fcce80b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fccfc07bf20 tx=0x7fcce4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:42.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.397+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccec0cf0c0 con 0x7fccfc103bc0 2024-09-21T03:16:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:42 smithi193 ceph-mon[35224]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:42 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3539363481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:16:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:42 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/418240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:16:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.614+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccfc103fd0 con 0x7fccfc103bc0 2024-09-21T03:16:42.617 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:42 smithi083 ceph-mon[31219]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:42.617 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:42 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3539363481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:16:42.617 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:42 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/418240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:16:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.615+0000 7fccf9ffb640 1 -- 172.21.15.83:0/3344264765 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fccec0d34a0 con 0x7fccfc103bc0 2024-09-21T03:16:42.618 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:42.618 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"btime":"2024-09-21T03:16:40:269796+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:16:40.040340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:16:42.618 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-09-21T03:16:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.617+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 msgr2=0x7fcce80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.617+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 0x7fcce80b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fccfc07bf20 tx=0x7fcce4005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:42.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.617+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 msgr2=0x7fccfc19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.617+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fccec033040 tx=0x7fccec0316f0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 shutdown_connections 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcce80b65e0 0x7fcce80b8af0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccfc109e00 0x7fccfc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccfc1094b0 0x7fccfc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 --2- 172.21.15.83:0/3344264765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccfc103bc0 0x7fccfc19a470 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 >> 172.21.15.83:0/3344264765 conn(0x7fccfc07b910 msgr2=0x7fccfc107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 shutdown_connections 2024-09-21T03:16:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:42.618+0000 7fcd030fa640 1 -- 172.21.15.83:0/3344264765 wait complete. 2024-09-21T03:16:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:42 smithi142 ceph-mon[34512]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T03:16:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:42 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3539363481' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:16:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:42 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/418240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:16:43.282 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T03:16:43.282 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T03:16:43.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:16:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 -- 172.21.15.83:0/3102994240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381082b0 msgr2=0x7fa2381086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 --2- 172.21.15.83:0/3102994240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381082b0 0x7fa2381086c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa228009900 tx=0x7fa22802a410 comp rx=0 tx=0).stop 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 -- 172.21.15.83:0/3102994240 shutdown_connections 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 --2- 172.21.15.83:0/3102994240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238108c00 0x7fa23810f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 --2- 172.21.15.83:0/3102994240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381082b0 0x7fa2381086c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 --2- 172.21.15.83:0/3102994240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa238102980 0x7fa238102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 -- 172.21.15.83:0/3102994240 >> 172.21.15.83:0/3102994240 conn(0x7fa2380fe380 msgr2=0x7fa2381007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 -- 172.21.15.83:0/3102994240 shutdown_connections 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.391+0000 7fa23f37b640 1 -- 172.21.15.83:0/3102994240 wait complete. 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 Processor -- start 2024-09-21T03:16:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 -- start start 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238102980 0x7fa238199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2381082b0 0x7fa238199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381128e0 con 0x7fa238108c00 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa238112760 con 0x7fa238102980 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23f37b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa238112a60 con 0x7fa2381082b0 2024-09-21T03:16:43.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.392+0000 7fa23d918640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60488/0 (socket says 172.21.15.83:60488) 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 -- 172.21.15.83:0/992375770 learned_addr learned my addr 172.21.15.83:0/992375770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d117640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238102980 0x7fa238199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23c916640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2381082b0 0x7fa238199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 -- 172.21.15.83:0/992375770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2381082b0 msgr2=0x7fa238199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2381082b0 0x7fa238199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 -- 172.21.15.83:0/992375770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238102980 msgr2=0x7fa238199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238102980 0x7fa238199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 -- 172.21.15.83:0/992375770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23819f200 con 0x7fa238108c00 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23d918640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa23000cce0 tx=0x7fa23000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000f450 con 0x7fa238108c00 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23000faa0 con 0x7fa238108c00 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230018a80 con 0x7fa238108c00 2024-09-21T03:16:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23819f4f0 con 0x7fa238108c00 2024-09-21T03:16:43.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.393+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a5d10 con 0x7fa238108c00 2024-09-21T03:16:43.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.394+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa238103bf0 con 0x7fa238108c00 2024-09-21T03:16:43.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.396+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa23000fc50 con 0x7fa238108c00 2024-09-21T03:16:43.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.397+0000 7fa22e7fc640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 0x7fa2240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.397+0000 7fa23d117640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 0x7fa2240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.397+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2300e7d60 con 0x7fa238108c00 2024-09-21T03:16:43.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.398+0000 7fa23d117640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 0x7fa2240b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa234009990 tx=0x7fa234009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.400+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa230014030 con 0x7fa238108c00 2024-09-21T03:16:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:43 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3344264765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:43 smithi193 ceph-mon[35224]: pgmap v327: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T03:16:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.577+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa23810aa60 con 0x7fa2240b6560 2024-09-21T03:16:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.582+0000 7fa22e7fc640 1 -- 172.21.15.83:0/992375770 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259134 (secure 0 0 0) 0x7fa23400e850 con 0x7fa2240b6560 2024-09-21T03:16:43.587 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:43.588 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 msgr2=0x7fa2240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 0x7fa2240b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa234009990 tx=0x7fa234009110 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 msgr2=0x7fa23819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa23000cce0 tx=0x7fa23000a620 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 shutdown_connections 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2240b6560 0x7fa2240b8a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238108c00 0x7fa23819eb90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2381082b0 0x7fa238199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 --2- 172.21.15.83:0/992375770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa238102980 0x7fa238199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 >> 172.21.15.83:0/992375770 conn(0x7fa2380fe380 msgr2=0x7fa238107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:43.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 shutdown_connections 2024-09-21T03:16:43.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.585+0000 7fa23f37b640 1 -- 172.21.15.83:0/992375770 wait complete. 2024-09-21T03:16:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:43 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3344264765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:43 smithi083 ceph-mon[31219]: pgmap v327: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T03:16:43.599 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-21T03:16:43.202369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663178,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1260,"num_read_kb":739,"num_write":383,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351404,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590420,"statfs":{"total":1151940427776,"available":1151580590080,"internally_reserved":0,"allocated":10727424,"data_stored":6970794,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":77,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003256"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461448+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:59.461448+0000","last_peered":"2024-09-21T03:15:59.461448+0000","last_clean":"2024-09-21T03:15:59.461448+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:59.461448+0000","last_undegraded":"2024-09-21T03:15:59.461448+0000","last_fullsized":"2024-09-21T03:15:59.461448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454514+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:59.454514+0000","last_peered":"2024-09-21T03:15:59.454514+0000","last_clean":"2024-09-21T03:15:59.454514+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:59.454514+0000","last_undegraded":"2024-09-21T03:15:59.454514+0000","last_fullsized":"2024-09-21T03:15:59.454514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462578+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:59.462578+0000","last_peered":"2024-09-21T03:15:59.462578+0000","last_clean":"2024-09-21T03:15:59.462578+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:59.462578+0000","last_undegraded":"2024-09-21T03:15:59.462578+0000","last_fullsized":"2024-09-21T03:15:59.462578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459054+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:59.459054+0000","last_peered":"2024-09-21T03:15:59.459054+0000","last_clean":"2024-09-21T03:15:59.459054+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:59.459054+0000","last_undegraded":"2024-09-21T03:15:59.459054+0000","last_fullsized":"2024-09-21T03:15:59.459054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436242+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:59.436242+0000","last_peered":"2024-09-21T03:15:59.436242+0000","last_clean":"2024-09-21T03:15:59.436242+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:59.436242+0000","last_undegraded":"2024-09-21T03:15:59.436242+0000","last_fullsized":"2024-09-21T03:15:59.436242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458740+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:59.458740+0000","last_peered":"2024-09-21T03:15:59.458740+0000","last_clean":"2024-09-21T03:15:59.458740+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:59.458740+0000","last_undegraded":"2024-09-21T03:15:59.458740+0000","last_fullsized":"2024-09-21T03:15:59.458740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445973+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:59.445973+0000","last_peered":"2024-09-21T03:15:59.445973+0000","last_clean":"2024-09-21T03:15:59.445973+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:59.445973+0000","last_undegraded":"2024-09-21T03:15:59.445973+0000","last_fullsized":"2024-09-21T03:15:59.445973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452622+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:59.452622+0000","last_peered":"2024-09-21T03:15:59.452622+0000","last_clean":"2024-09-21T03:15:59.452622+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:59.452622+0000","last_undegraded":"2024-09-21T03:15:59.452622+0000","last_fullsized":"2024-09-21T03:15:59.452622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458000+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:59.458000+0000","last_peered":"2024-09-21T03:15:59.458000+0000","last_clean":"2024-09-21T03:15:59.458000+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:59.458000+0000","last_undegraded":"2024-09-21T03:15:59.458000+0000","last_fullsized":"2024-09-21T03:15:59.458000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459668+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:59.459668+0000","last_peered":"2024-09-21T03:15:59.459668+0000","last_clean":"2024-09-21T03:15:59.459668+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:59.459668+0000","last_undegraded":"2024-09-21T03:15:59.459668+0000","last_fullsized":"2024-09-21T03:15:59.459668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460855+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:59.460855+0000","last_peered":"2024-09-21T03:15:59.460855+0000","last_clean":"2024-09-21T03:15:59.460855+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:59.460855+0000","last_undegraded":"2024-09-21T03:15:59.460855+0000","last_fullsized":"2024-09-21T03:15:59.460855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.442827+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:59.442827+0000","last_peered":"2024-09-21T03:15:59.442827+0000","last_clean":"2024-09-21T03:15:59.442827+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:59.442827+0000","last_undegraded":"2024-09-21T03:15:59.442827+0000","last_fullsized":"2024-09-21T03:15:59.442827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452352+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:59.452352+0000","last_peered":"2024-09-21T03:15:59.452352+0000","last_clean":"2024-09-21T03:15:59.452352+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:59.452352+0000","last_undegraded":"2024-09-21T03:15:59.452352+0000","last_fullsized":"2024-09-21T03:15:59.452352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454556+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:59.454556+0000","last_peered":"2024-09-21T03:15:59.454556+0000","last_clean":"2024-09-21T03:15:59.454556+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:59.454556+0000","last_undegraded":"2024-09-21T03:15:59.454556+0000","last_fullsized":"2024-09-21T03:15:59.454556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453334+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:59.453334+0000","last_peered":"2024-09-21T03:15:59.453334+0000","last_clean":"2024-09-21T03:15:59.453334+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:59.453334+0000","last_undegraded":"2024-09-21T03:15:59.453334+0000","last_fullsized":"2024-09-21T03:15:59.453334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462531+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:59.462531+0000","last_peered":"2024-09-21T03:15:59.462531+0000","last_clean":"2024-09-21T03:15:59.462531+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:59.462531+0000","last_undegraded":"2024-09-21T03:15:59.462531+0000","last_fullsized":"2024-09-21T03:15:59.462531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461567+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:59.461567+0000","last_peered":"2024-09-21T03:15:59.461567+0000","last_clean":"2024-09-21T03:15:59.461567+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:59.461567+0000","last_undegraded":"2024-09-21T03:15:59.461567+0000","last_fullsized":"2024-09-21T03:15:59.461567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.437070+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:59.437070+0000","last_peered":"2024-09-21T03:15:59.437070+0000","last_clean":"2024-09-21T03:15:59.437070+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:59.437070+0000","last_undegraded":"2024-09-21T03:15:59.437070+0000","last_fullsized":"2024-09-21T03:15:59.437070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448112+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:59.448112+0000","last_peered":"2024-09-21T03:15:59.448112+0000","last_clean":"2024-09-21T03:15:59.448112+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:59.448112+0000","last_undegraded":"2024-09-21T03:15:59.448112+0000","last_fullsized":"2024-09-21T03:15:59.448112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452845+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:59.452845+0000","last_peered":"2024-09-21T03:15:59.452845+0000","last_clean":"2024-09-21T03:15:59.452845+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:59.452845+0000","last_undegraded":"2024-09-21T03:15:59.452845+0000","last_fullsized":"2024-09-21T03:15:59.452845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447478+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:59.447478+0000","last_peered":"2024-09-21T03:15:59.447478+0000","last_clean":"2024-09-21T03:15:59.447478+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:59.447478+0000","last_undegraded":"2024-09-21T03:15:59.447478+0000","last_fullsized":"2024-09-21T03:15:59.447478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462703+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:59.462703+0000","last_peered":"2024-09-21T03:15:59.462703+0000","last_clean":"2024-09-21T03:15:59.462703+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:59.462703+0000","last_undegraded":"2024-09-21T03:15:59.462703+0000","last_fullsized":"2024-09-21T03:15:59.462703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448989+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:59.448989+0000","last_peered":"2024-09-21T03:15:59.448989+0000","last_clean":"2024-09-21T03:15:59.448989+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:59.448989+0000","last_undegraded":"2024-09-21T03:15:59.448989+0000","last_fullsized":"2024-09-21T03:15:59.448989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451608+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:59.451608+0000","last_peered":"2024-09-21T03:15:59.451608+0000","last_clean":"2024-09-21T03:15:59.451608+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:59.451608+0000","last_undegraded":"2024-09-21T03:15:59.451608+0000","last_fullsized":"2024-09-21T03:15:59.451608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436971+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:59.436971+0000","last_peered":"2024-09-21T03:15:59.436971+0000","last_clean":"2024-09-21T03:15:59.436971+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:59.436971+0000","last_undegraded":"2024-09-21T03:15:59.436971+0000","last_fullsized":"2024-09-21T03:15:59.436971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.438447+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:59.438447+0000","last_peered":"2024-09-21T03:15:59.438447+0000","last_clean":"2024-09-21T03:15:59.438447+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:59.438447+0000","last_undegraded":"2024-09-21T03:15:59.438447+0000","last_fullsized":"2024-09-21T03:15:59.438447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445817+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:59.445817+0000","last_peered":"2024-09-21T03:15:59.445817+0000","last_clean":"2024-09-21T03:15:59.445817+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:59.445817+0000","last_undegraded":"2024-09-21T03:15:59.445817+0000","last_fullsized":"2024-09-21T03:15:59.445817+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445217+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:59.445217+0000","last_peered":"2024-09-21T03:15:59.445217+0000","last_clean":"2024-09-21T03:15:59.445217+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:59.445217+0000","last_undegraded":"2024-09-21T03:15:59.445217+0000","last_fullsized":"2024-09-21T03:15:59.445217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457735+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:59.457735+0000","last_peered":"2024-09-21T03:15:59.457735+0000","last_clean":"2024-09-21T03:15:59.457735+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:59.457735+0000","last_undegraded":"2024-09-21T03:15:59.457735+0000","last_fullsized":"2024-09-21T03:15:59.457735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445603+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:59.445603+0000","last_peered":"2024-09-21T03:15:59.445603+0000","last_clean":"2024-09-21T03:15:59.445603+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:59.445603+0000","last_undegraded":"2024-09-21T03:15:59.445603+0000","last_fullsized":"2024-09-21T03:15:59.445603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448912+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:59.448912+0000","last_peered":"2024-09-21T03:15:59.448912+0000","last_clean":"2024-09-21T03:15:59.448912+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:59.448912+0000","last_undegraded":"2024-09-21T03:15:59.448912+0000","last_fullsized":"2024-09-21T03:15:59.448912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452258+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:59.452258+0000","last_peered":"2024-09-21T03:15:59.452258+0000","last_clean":"2024-09-21T03:15:59.452258+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:59.452258+0000","last_undegraded":"2024-09-21T03:15:59.452258+0000","last_fullsized":"2024-09-21T03:15:59.452258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436994+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:59.436994+0000","last_peered":"2024-09-21T03:15:59.436994+0000","last_clean":"2024-09-21T03:15:59.436994+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:59.436994+0000","last_undegraded":"2024-09-21T03:15:59.436994+0000","last_fullsized":"2024-09-21T03:15:59.436994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:37.920447+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:16:37.920447+0000","last_peered":"2024-09-21T03:16:37.920447+0000","last_clean":"2024-09-21T03:16:37.920447+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:16:37.920447+0000","last_undegraded":"2024-09-21T03:16:37.920447+0000","last_fullsized":"2024-09-21T03:16:37.920447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454550+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:59.454550+0000","last_peered":"2024-09-21T03:15:59.454550+0000","last_clean":"2024-09-21T03:15:59.454550+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:59.454550+0000","last_undegraded":"2024-09-21T03:15:59.454550+0000","last_fullsized":"2024-09-21T03:15:59.454550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450571+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:59.450571+0000","last_peered":"2024-09-21T03:15:59.450571+0000","last_clean":"2024-09-21T03:15:59.450571+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:59.450571+0000","last_undegraded":"2024-09-21T03:15:59.450571+0000","last_fullsized":"2024-09-21T03:15:59.450571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441638+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:59.441638+0000","last_peered":"2024-09-21T03:15:59.441638+0000","last_clean":"2024-09-21T03:15:59.441638+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:59.441638+0000","last_undegraded":"2024-09-21T03:15:59.441638+0000","last_fullsized":"2024-09-21T03:15:59.441638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":257,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925557+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:16:41.925557+0000","last_peered":"2024-09-21T03:16:41.925557+0000","last_clean":"2024-09-21T03:16:41.925557+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:16:41.925557+0000","last_undegraded":"2024-09-21T03:16:41.925557+0000","last_fullsized":"2024-09-21T03:16:41.925557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461606+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:59.461606+0000","last_peered":"2024-09-21T03:15:59.461606+0000","last_clean":"2024-09-21T03:15:59.461606+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:59.461606+0000","last_undegraded":"2024-09-21T03:15:59.461606+0000","last_fullsized":"2024-09-21T03:15:59.461606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454275+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:59.454275+0000","last_peered":"2024-09-21T03:15:59.454275+0000","last_clean":"2024-09-21T03:15:59.454275+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:59.454275+0000","last_undegraded":"2024-09-21T03:15:59.454275+0000","last_fullsized":"2024-09-21T03:15:59.454275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462517+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:59.462517+0000","last_peered":"2024-09-21T03:15:59.462517+0000","last_clean":"2024-09-21T03:15:59.462517+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:59.462517+0000","last_undegraded":"2024-09-21T03:15:59.462517+0000","last_fullsized":"2024-09-21T03:15:59.462517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":297,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925677+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:16:41.925677+0000","last_peered":"2024-09-21T03:16:41.925677+0000","last_clean":"2024-09-21T03:16:41.925677+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:16:41.925677+0000","last_undegraded":"2024-09-21T03:16:41.925677+0000","last_fullsized":"2024-09-21T03:16:41.925677+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65314,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":182,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447951+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:59.447951+0000","last_peered":"2024-09-21T03:15:59.447951+0000","last_clean":"2024-09-21T03:15:59.447951+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:59.447951+0000","last_undegraded":"2024-09-21T03:15:59.447951+0000","last_fullsized":"2024-09-21T03:15:59.447951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447254+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:59.447254+0000","last_peered":"2024-09-21T03:15:59.447254+0000","last_clean":"2024-09-21T03:15:59.447254+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:59.447254+0000","last_undegraded":"2024-09-21T03:15:59.447254+0000","last_fullsized":"2024-09-21T03:15:59.447254+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460287+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:59.460287+0000","last_peered":"2024-09-21T03:15:59.460287+0000","last_clean":"2024-09-21T03:15:59.460287+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:59.460287+0000","last_undegraded":"2024-09-21T03:15:59.460287+0000","last_fullsized":"2024-09-21T03:15:59.460287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":145,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.939765+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:16:41.939765+0000","last_peered":"2024-09-21T03:16:41.939765+0000","last_clean":"2024-09-21T03:16:41.939765+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:16:41.939765+0000","last_undegraded":"2024-09-21T03:16:41.939765+0000","last_fullsized":"2024-09-21T03:16:41.939765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450468+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:59.450468+0000","last_peered":"2024-09-21T03:15:59.450468+0000","last_clean":"2024-09-21T03:15:59.450468+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:59.450468+0000","last_undegraded":"2024-09-21T03:15:59.450468+0000","last_fullsized":"2024-09-21T03:15:59.450468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453704+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:59.453704+0000","last_peered":"2024-09-21T03:15:59.453704+0000","last_clean":"2024-09-21T03:15:59.453704+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:59.453704+0000","last_undegraded":"2024-09-21T03:15:59.453704+0000","last_fullsized":"2024-09-21T03:15:59.453704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452996+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:59.452996+0000","last_peered":"2024-09-21T03:15:59.452996+0000","last_clean":"2024-09-21T03:15:59.452996+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:59.452996+0000","last_undegraded":"2024-09-21T03:15:59.452996+0000","last_fullsized":"2024-09-21T03:15:59.452996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451990+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:59.451990+0000","last_peered":"2024-09-21T03:15:59.451990+0000","last_clean":"2024-09-21T03:15:59.451990+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:59.451990+0000","last_undegraded":"2024-09-21T03:15:59.451990+0000","last_fullsized":"2024-09-21T03:15:59.451990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450791+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:59.450791+0000","last_peered":"2024-09-21T03:15:59.450791+0000","last_clean":"2024-09-21T03:15:59.450791+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:59.450791+0000","last_undegraded":"2024-09-21T03:15:59.450791+0000","last_fullsized":"2024-09-21T03:15:59.450791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'8","reported_seq":150,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.924913+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:16:40.924913+0000","last_peered":"2024-09-21T03:16:40.924913+0000","last_clean":"2024-09-21T03:16:40.924913+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:16:40.924913+0000","last_undegraded":"2024-09-21T03:16:40.924913+0000","last_fullsized":"2024-09-21T03:16:40.924913+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460998+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:59.460998+0000","last_peered":"2024-09-21T03:15:59.460998+0000","last_clean":"2024-09-21T03:15:59.460998+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:59.460998+0000","last_undegraded":"2024-09-21T03:15:59.460998+0000","last_fullsized":"2024-09-21T03:15:59.460998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259828+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259828+0000","last_peered":"2024-09-21T03:15:00.259828+0000","last_clean":"2024-09-21T03:15:00.259828+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259828+0000","last_undegraded":"2024-09-21T03:15:00.259828+0000","last_fullsized":"2024-09-21T03:15:00.259828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264069+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264069+0000","last_peered":"2024-09-21T03:15:00.264069+0000","last_clean":"2024-09-21T03:15:00.264069+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264069+0000","last_undegraded":"2024-09-21T03:15:00.264069+0000","last_fullsized":"2024-09-21T03:15:00.264069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441320+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:59.441320+0000","last_peered":"2024-09-21T03:15:59.441320+0000","last_clean":"2024-09-21T03:15:59.441320+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:59.441320+0000","last_undegraded":"2024-09-21T03:15:59.441320+0000","last_fullsized":"2024-09-21T03:15:59.441320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259780+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259780+0000","last_peered":"2024-09-21T03:15:00.259780+0000","last_clean":"2024-09-21T03:15:00.259780+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259780+0000","last_undegraded":"2024-09-21T03:15:00.259780+0000","last_fullsized":"2024-09-21T03:15:00.259780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446838+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:59.446838+0000","last_peered":"2024-09-21T03:15:59.446838+0000","last_clean":"2024-09-21T03:15:59.446838+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:59.446838+0000","last_undegraded":"2024-09-21T03:15:59.446838+0000","last_fullsized":"2024-09-21T03:15:59.446838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.937742+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:16:40.937742+0000","last_peered":"2024-09-21T03:16:40.937742+0000","last_clean":"2024-09-21T03:16:40.937742+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:16:40.937742+0000","last_undegraded":"2024-09-21T03:16:40.937742+0000","last_fullsized":"2024-09-21T03:16:40.937742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455106+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:59.455106+0000","last_peered":"2024-09-21T03:15:59.455106+0000","last_clean":"2024-09-21T03:15:59.455106+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:59.455106+0000","last_undegraded":"2024-09-21T03:15:59.455106+0000","last_fullsized":"2024-09-21T03:15:59.455106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:39.922099+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:16:39.922099+0000","last_peered":"2024-09-21T03:16:39.922099+0000","last_clean":"2024-09-21T03:16:39.922099+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:16:39.922099+0000","last_undegraded":"2024-09-21T03:16:39.922099+0000","last_fullsized":"2024-09-21T03:16:39.922099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451490+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:59.451490+0000","last_peered":"2024-09-21T03:15:59.451490+0000","last_clean":"2024-09-21T03:15:59.451490+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:59.451490+0000","last_undegraded":"2024-09-21T03:15:59.451490+0000","last_fullsized":"2024-09-21T03:15:59.451490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265641+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265641+0000","last_peered":"2024-09-21T03:15:00.265641+0000","last_clean":"2024-09-21T03:15:00.265641+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265641+0000","last_undegraded":"2024-09-21T03:15:00.265641+0000","last_fullsized":"2024-09-21T03:15:00.265641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448466+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:59.448466+0000","last_peered":"2024-09-21T03:15:59.448466+0000","last_clean":"2024-09-21T03:15:59.448466+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:59.448466+0000","last_undegraded":"2024-09-21T03:15:59.448466+0000","last_fullsized":"2024-09-21T03:15:59.448466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261679+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261679+0000","last_peered":"2024-09-21T03:15:00.261679+0000","last_clean":"2024-09-21T03:15:00.261679+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261679+0000","last_undegraded":"2024-09-21T03:15:00.261679+0000","last_fullsized":"2024-09-21T03:15:00.261679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446331+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:59.446331+0000","last_peered":"2024-09-21T03:15:59.446331+0000","last_clean":"2024-09-21T03:15:59.446331+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:59.446331+0000","last_undegraded":"2024-09-21T03:15:59.446331+0000","last_fullsized":"2024-09-21T03:15:59.446331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.622195+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:59.622195+0000","last_peered":"2024-09-21T03:15:59.622195+0000","last_clean":"2024-09-21T03:15:59.622195+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:59.622195+0000","last_undegraded":"2024-09-21T03:15:59.622195+0000","last_fullsized":"2024-09-21T03:15:59.622195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.449080+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:59.449080+0000","last_peered":"2024-09-21T03:15:59.449080+0000","last_clean":"2024-09-21T03:15:59.449080+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:59.449080+0000","last_undegraded":"2024-09-21T03:15:59.449080+0000","last_fullsized":"2024-09-21T03:15:59.449080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457757+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:59.457757+0000","last_peered":"2024-09-21T03:15:59.457757+0000","last_clean":"2024-09-21T03:15:59.457757+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:59.457757+0000","last_undegraded":"2024-09-21T03:15:59.457757+0000","last_fullsized":"2024-09-21T03:15:59.457757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.408350+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:59.408350+0000","last_peered":"2024-09-21T03:15:59.408350+0000","last_clean":"2024-09-21T03:15:59.408350+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:59.408350+0000","last_undegraded":"2024-09-21T03:15:59.408350+0000","last_fullsized":"2024-09-21T03:15:59.408350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450927+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:59.450927+0000","last_peered":"2024-09-21T03:15:59.450927+0000","last_clean":"2024-09-21T03:15:59.450927+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:59.450927+0000","last_undegraded":"2024-09-21T03:15:59.450927+0000","last_fullsized":"2024-09-21T03:15:59.450927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265986+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265986+0000","last_peered":"2024-09-21T03:15:00.265986+0000","last_clean":"2024-09-21T03:15:00.265986+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265986+0000","last_undegraded":"2024-09-21T03:15:00.265986+0000","last_fullsized":"2024-09-21T03:15:00.265986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445834+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:59.445834+0000","last_peered":"2024-09-21T03:15:59.445834+0000","last_clean":"2024-09-21T03:15:59.445834+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:59.445834+0000","last_undegraded":"2024-09-21T03:15:59.445834+0000","last_fullsized":"2024-09-21T03:15:59.445834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458762+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:59.458762+0000","last_peered":"2024-09-21T03:15:59.458762+0000","last_clean":"2024-09-21T03:15:59.458762+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:59.458762+0000","last_undegraded":"2024-09-21T03:15:59.458762+0000","last_fullsized":"2024-09-21T03:15:59.458762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455714+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:59.455714+0000","last_peered":"2024-09-21T03:15:59.455714+0000","last_clean":"2024-09-21T03:15:59.455714+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:59.455714+0000","last_undegraded":"2024-09-21T03:15:59.455714+0000","last_fullsized":"2024-09-21T03:15:59.455714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461974+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:59.461974+0000","last_peered":"2024-09-21T03:15:59.461974+0000","last_clean":"2024-09-21T03:15:59.461974+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:59.461974+0000","last_undegraded":"2024-09-21T03:15:59.461974+0000","last_fullsized":"2024-09-21T03:15:59.461974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462659+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:59.462659+0000","last_peered":"2024-09-21T03:15:59.462659+0000","last_clean":"2024-09-21T03:15:59.462659+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:59.462659+0000","last_undegraded":"2024-09-21T03:15:59.462659+0000","last_fullsized":"2024-09-21T03:15:59.462659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259071+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259071+0000","last_peered":"2024-09-21T03:15:00.259071+0000","last_clean":"2024-09-21T03:15:00.259071+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259071+0000","last_undegraded":"2024-09-21T03:15:00.259071+0000","last_fullsized":"2024-09-21T03:15:00.259071+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452714+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:59.452714+0000","last_peered":"2024-09-21T03:15:59.452714+0000","last_clean":"2024-09-21T03:15:59.452714+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:59.452714+0000","last_undegraded":"2024-09-21T03:15:59.452714+0000","last_fullsized":"2024-09-21T03:15:59.452714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":152,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.924547+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:16:41.924547+0000","last_peered":"2024-09-21T03:16:41.924547+0000","last_clean":"2024-09-21T03:16:41.924547+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:16:41.924547+0000","last_undegraded":"2024-09-21T03:16:41.924547+0000","last_fullsized":"2024-09-21T03:16:41.924547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72576,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1179,"num_read_kb":670,"num_write":258,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":417205,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072640,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":483163,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":482171,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":418123,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:16:43.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:16:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:43 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3344264765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:43 smithi142 ceph-mon[34512]: pgmap v327: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T03:16:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- 172.21.15.83:0/1605239798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108c50 msgr2=0x7f2fb810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/1605239798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108c50 0x7f2fb810f480 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009ca0 tx=0x7f2fb002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- 172.21.15.83:0/1605239798 shutdown_connections 2024-09-21T03:16:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/1605239798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108c50 0x7f2fb810f480 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/1605239798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb8108300 0x7f2fb8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/1605239798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb81029e0 0x7f2fb8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- 172.21.15.83:0/1605239798 >> 172.21.15.83:0/1605239798 conn(0x7f2fb80786b0 msgr2=0x7f2fb8078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- 172.21.15.83:0/1605239798 shutdown_connections 2024-09-21T03:16:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- 172.21.15.83:0/1605239798 wait complete. 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 Processor -- start 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.746+0000 7f2fbe79b640 1 -- start start 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 0x7f2fb819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb8108c50 0x7f2fb819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112960 con 0x7f2fb8108300 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81127e0 con 0x7f2fb81029e0 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbe79b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112ae0 con 0x7f2fb8108c50 2024-09-21T03:16:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60504/0 (socket says 172.21.15.83:60504) 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 -- 172.21.15.83:0/2077974230 learned_addr learned my addr 172.21.15.83:0/2077974230 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb7fff640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 0x7f2fb819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 -- 172.21.15.83:0/2077974230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb8108c50 msgr2=0x7f2fb819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fbcd38640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb8108c50 0x7f2fb819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb8108c50 0x7f2fb819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 -- 172.21.15.83:0/2077974230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 msgr2=0x7f2fb819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 0x7f2fb819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 -- 172.21.15.83:0/2077974230 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb81a6390 con 0x7f2fb8108300 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb7fff640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 0x7f2fb819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb77fe640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00d740 tx=0x7f2fac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.747+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac017070 con 0x7f2fb8108300 2024-09-21T03:16:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.748+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fac00de50 con 0x7f2fb8108300 2024-09-21T03:16:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.748+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81a6680 con 0x7f2fb8108300 2024-09-21T03:16:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.748+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00f620 con 0x7f2fb8108300 2024-09-21T03:16:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.748+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81a6c20 con 0x7f2fb8108300 2024-09-21T03:16:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.749+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb8102df0 con 0x7f2fb8108300 2024-09-21T03:16:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.750+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2fac01a070 con 0x7f2fb8108300 2024-09-21T03:16:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.751+0000 7f2fb57fa640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 0x7f2fa00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.751+0000 7f2fb7fff640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 0x7f2fa00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.751+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2fac0e6e90 con 0x7f2fb8108300 2024-09-21T03:16:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.751+0000 7f2fb7fff640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 0x7f2fa00b8af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005fd0 tx=0x7f2fa8005f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.755+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fac0ad680 con 0x7f2fb8108300 2024-09-21T03:16:43.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.933+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fb8063800 con 0x7f2fa00b65e0 2024-09-21T03:16:43.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.937+0000 7f2fb57fa640 1 -- 172.21.15.83:0/2077974230 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259134 (secure 0 0 0) 0x7f2fb8063800 con 0x7f2fa00b65e0 2024-09-21T03:16:43.942 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:43.943 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 msgr2=0x7f2fa00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 0x7f2fa00b8af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005fd0 tx=0x7f2fa8005f40 comp rx=0 tx=0).stop 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 msgr2=0x7f2fb819b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00d740 tx=0x7f2fac00dc00 comp rx=0 tx=0).stop 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 shutdown_connections 2024-09-21T03:16:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2fa00b65e0 0x7f2fa00b8af0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fb8108c50 0x7f2fb819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb8108300 0x7f2fb819b6d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 --2- 172.21.15.83:0/2077974230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fb81029e0 0x7f2fb819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 >> 172.21.15.83:0/2077974230 conn(0x7f2fb80786b0 msgr2=0x7f2fb81062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 shutdown_connections 2024-09-21T03:16:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:43.941+0000 7f2fbe79b640 1 -- 172.21.15.83:0/2077974230 wait complete. 2024-09-21T03:16:43.955 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-21T03:16:43.202369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663178,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1260,"num_read_kb":739,"num_write":383,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351404,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590420,"statfs":{"total":1151940427776,"available":1151580590080,"internally_reserved":0,"allocated":10727424,"data_stored":6970794,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":77,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003256"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461448+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:59.461448+0000","last_peered":"2024-09-21T03:15:59.461448+0000","last_clean":"2024-09-21T03:15:59.461448+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:59.461448+0000","last_undegraded":"2024-09-21T03:15:59.461448+0000","last_fullsized":"2024-09-21T03:15:59.461448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454514+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:59.454514+0000","last_peered":"2024-09-21T03:15:59.454514+0000","last_clean":"2024-09-21T03:15:59.454514+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:59.454514+0000","last_undegraded":"2024-09-21T03:15:59.454514+0000","last_fullsized":"2024-09-21T03:15:59.454514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462578+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:59.462578+0000","last_peered":"2024-09-21T03:15:59.462578+0000","last_clean":"2024-09-21T03:15:59.462578+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:59.462578+0000","last_undegraded":"2024-09-21T03:15:59.462578+0000","last_fullsized":"2024-09-21T03:15:59.462578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459054+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:59.459054+0000","last_peered":"2024-09-21T03:15:59.459054+0000","last_clean":"2024-09-21T03:15:59.459054+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:59.459054+0000","last_undegraded":"2024-09-21T03:15:59.459054+0000","last_fullsized":"2024-09-21T03:15:59.459054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436242+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:59.436242+0000","last_peered":"2024-09-21T03:15:59.436242+0000","last_clean":"2024-09-21T03:15:59.436242+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:59.436242+0000","last_undegraded":"2024-09-21T03:15:59.436242+0000","last_fullsized":"2024-09-21T03:15:59.436242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458740+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:59.458740+0000","last_peered":"2024-09-21T03:15:59.458740+0000","last_clean":"2024-09-21T03:15:59.458740+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:59.458740+0000","last_undegraded":"2024-09-21T03:15:59.458740+0000","last_fullsized":"2024-09-21T03:15:59.458740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445973+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:59.445973+0000","last_peered":"2024-09-21T03:15:59.445973+0000","last_clean":"2024-09-21T03:15:59.445973+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:59.445973+0000","last_undegraded":"2024-09-21T03:15:59.445973+0000","last_fullsized":"2024-09-21T03:15:59.445973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452622+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:59.452622+0000","last_peered":"2024-09-21T03:15:59.452622+0000","last_clean":"2024-09-21T03:15:59.452622+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:59.452622+0000","last_undegraded":"2024-09-21T03:15:59.452622+0000","last_fullsized":"2024-09-21T03:15:59.452622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458000+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:59.458000+0000","last_peered":"2024-09-21T03:15:59.458000+0000","last_clean":"2024-09-21T03:15:59.458000+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:59.458000+0000","last_undegraded":"2024-09-21T03:15:59.458000+0000","last_fullsized":"2024-09-21T03:15:59.458000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459668+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:59.459668+0000","last_peered":"2024-09-21T03:15:59.459668+0000","last_clean":"2024-09-21T03:15:59.459668+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:59.459668+0000","last_undegraded":"2024-09-21T03:15:59.459668+0000","last_fullsized":"2024-09-21T03:15:59.459668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460855+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:59.460855+0000","last_peered":"2024-09-21T03:15:59.460855+0000","last_clean":"2024-09-21T03:15:59.460855+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:59.460855+0000","last_undegraded":"2024-09-21T03:15:59.460855+0000","last_fullsized":"2024-09-21T03:15:59.460855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.442827+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:59.442827+0000","last_peered":"2024-09-21T03:15:59.442827+0000","last_clean":"2024-09-21T03:15:59.442827+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:59.442827+0000","last_undegraded":"2024-09-21T03:15:59.442827+0000","last_fullsized":"2024-09-21T03:15:59.442827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452352+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:59.452352+0000","last_peered":"2024-09-21T03:15:59.452352+0000","last_clean":"2024-09-21T03:15:59.452352+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:59.452352+0000","last_undegraded":"2024-09-21T03:15:59.452352+0000","last_fullsized":"2024-09-21T03:15:59.452352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454556+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:59.454556+0000","last_peered":"2024-09-21T03:15:59.454556+0000","last_clean":"2024-09-21T03:15:59.454556+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:59.454556+0000","last_undegraded":"2024-09-21T03:15:59.454556+0000","last_fullsized":"2024-09-21T03:15:59.454556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453334+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:59.453334+0000","last_peered":"2024-09-21T03:15:59.453334+0000","last_clean":"2024-09-21T03:15:59.453334+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:59.453334+0000","last_undegraded":"2024-09-21T03:15:59.453334+0000","last_fullsized":"2024-09-21T03:15:59.453334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462531+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:59.462531+0000","last_peered":"2024-09-21T03:15:59.462531+0000","last_clean":"2024-09-21T03:15:59.462531+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:59.462531+0000","last_undegraded":"2024-09-21T03:15:59.462531+0000","last_fullsized":"2024-09-21T03:15:59.462531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461567+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:59.461567+0000","last_peered":"2024-09-21T03:15:59.461567+0000","last_clean":"2024-09-21T03:15:59.461567+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:59.461567+0000","last_undegraded":"2024-09-21T03:15:59.461567+0000","last_fullsized":"2024-09-21T03:15:59.461567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.437070+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:59.437070+0000","last_peered":"2024-09-21T03:15:59.437070+0000","last_clean":"2024-09-21T03:15:59.437070+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:59.437070+0000","last_undegraded":"2024-09-21T03:15:59.437070+0000","last_fullsized":"2024-09-21T03:15:59.437070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448112+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:59.448112+0000","last_peered":"2024-09-21T03:15:59.448112+0000","last_clean":"2024-09-21T03:15:59.448112+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:59.448112+0000","last_undegraded":"2024-09-21T03:15:59.448112+0000","last_fullsized":"2024-09-21T03:15:59.448112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452845+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:59.452845+0000","last_peered":"2024-09-21T03:15:59.452845+0000","last_clean":"2024-09-21T03:15:59.452845+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:59.452845+0000","last_undegraded":"2024-09-21T03:15:59.452845+0000","last_fullsized":"2024-09-21T03:15:59.452845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447478+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:59.447478+0000","last_peered":"2024-09-21T03:15:59.447478+0000","last_clean":"2024-09-21T03:15:59.447478+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:59.447478+0000","last_undegraded":"2024-09-21T03:15:59.447478+0000","last_fullsized":"2024-09-21T03:15:59.447478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462703+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:59.462703+0000","last_peered":"2024-09-21T03:15:59.462703+0000","last_clean":"2024-09-21T03:15:59.462703+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:59.462703+0000","last_undegraded":"2024-09-21T03:15:59.462703+0000","last_fullsized":"2024-09-21T03:15:59.462703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448989+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:59.448989+0000","last_peered":"2024-09-21T03:15:59.448989+0000","last_clean":"2024-09-21T03:15:59.448989+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:59.448989+0000","last_undegraded":"2024-09-21T03:15:59.448989+0000","last_fullsized":"2024-09-21T03:15:59.448989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451608+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:59.451608+0000","last_peered":"2024-09-21T03:15:59.451608+0000","last_clean":"2024-09-21T03:15:59.451608+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:59.451608+0000","last_undegraded":"2024-09-21T03:15:59.451608+0000","last_fullsized":"2024-09-21T03:15:59.451608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436971+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:59.436971+0000","last_peered":"2024-09-21T03:15:59.436971+0000","last_clean":"2024-09-21T03:15:59.436971+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:59.436971+0000","last_undegraded":"2024-09-21T03:15:59.436971+0000","last_fullsized":"2024-09-21T03:15:59.436971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.438447+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:59.438447+0000","last_peered":"2024-09-21T03:15:59.438447+0000","last_clean":"2024-09-21T03:15:59.438447+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:59.438447+0000","last_undegraded":"2024-09-21T03:15:59.438447+0000","last_fullsized":"2024-09-21T03:15:59.438447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445817+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:59.445817+0000","last_peered":"2024-09-21T03:15:59.445817+0000","last_clean":"2024-09-21T03:15:59.445817+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:59.445817+0000","last_undegraded":"2024-09-21T03:15:59.445817+0000","last_fullsized":"2024-09-21T03:15:59.445817+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445217+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:59.445217+0000","last_peered":"2024-09-21T03:15:59.445217+0000","last_clean":"2024-09-21T03:15:59.445217+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:59.445217+0000","last_undegraded":"2024-09-21T03:15:59.445217+0000","last_fullsized":"2024-09-21T03:15:59.445217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457735+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:59.457735+0000","last_peered":"2024-09-21T03:15:59.457735+0000","last_clean":"2024-09-21T03:15:59.457735+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:59.457735+0000","last_undegraded":"2024-09-21T03:15:59.457735+0000","last_fullsized":"2024-09-21T03:15:59.457735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445603+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:59.445603+0000","last_peered":"2024-09-21T03:15:59.445603+0000","last_clean":"2024-09-21T03:15:59.445603+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:59.445603+0000","last_undegraded":"2024-09-21T03:15:59.445603+0000","last_fullsized":"2024-09-21T03:15:59.445603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448912+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:59.448912+0000","last_peered":"2024-09-21T03:15:59.448912+0000","last_clean":"2024-09-21T03:15:59.448912+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:59.448912+0000","last_undegraded":"2024-09-21T03:15:59.448912+0000","last_fullsized":"2024-09-21T03:15:59.448912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452258+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:59.452258+0000","last_peered":"2024-09-21T03:15:59.452258+0000","last_clean":"2024-09-21T03:15:59.452258+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:59.452258+0000","last_undegraded":"2024-09-21T03:15:59.452258+0000","last_fullsized":"2024-09-21T03:15:59.452258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436994+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:59.436994+0000","last_peered":"2024-09-21T03:15:59.436994+0000","last_clean":"2024-09-21T03:15:59.436994+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:59.436994+0000","last_undegraded":"2024-09-21T03:15:59.436994+0000","last_fullsized":"2024-09-21T03:15:59.436994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:37.920447+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:16:37.920447+0000","last_peered":"2024-09-21T03:16:37.920447+0000","last_clean":"2024-09-21T03:16:37.920447+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:16:37.920447+0000","last_undegraded":"2024-09-21T03:16:37.920447+0000","last_fullsized":"2024-09-21T03:16:37.920447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454550+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:59.454550+0000","last_peered":"2024-09-21T03:15:59.454550+0000","last_clean":"2024-09-21T03:15:59.454550+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:59.454550+0000","last_undegraded":"2024-09-21T03:15:59.454550+0000","last_fullsized":"2024-09-21T03:15:59.454550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450571+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:59.450571+0000","last_peered":"2024-09-21T03:15:59.450571+0000","last_clean":"2024-09-21T03:15:59.450571+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:59.450571+0000","last_undegraded":"2024-09-21T03:15:59.450571+0000","last_fullsized":"2024-09-21T03:15:59.450571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441638+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:59.441638+0000","last_peered":"2024-09-21T03:15:59.441638+0000","last_clean":"2024-09-21T03:15:59.441638+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:59.441638+0000","last_undegraded":"2024-09-21T03:15:59.441638+0000","last_fullsized":"2024-09-21T03:15:59.441638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":257,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925557+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:16:41.925557+0000","last_peered":"2024-09-21T03:16:41.925557+0000","last_clean":"2024-09-21T03:16:41.925557+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:16:41.925557+0000","last_undegraded":"2024-09-21T03:16:41.925557+0000","last_fullsized":"2024-09-21T03:16:41.925557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461606+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:59.461606+0000","last_peered":"2024-09-21T03:15:59.461606+0000","last_clean":"2024-09-21T03:15:59.461606+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:59.461606+0000","last_undegraded":"2024-09-21T03:15:59.461606+0000","last_fullsized":"2024-09-21T03:15:59.461606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454275+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:59.454275+0000","last_peered":"2024-09-21T03:15:59.454275+0000","last_clean":"2024-09-21T03:15:59.454275+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:59.454275+0000","last_undegraded":"2024-09-21T03:15:59.454275+0000","last_fullsized":"2024-09-21T03:15:59.454275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462517+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:59.462517+0000","last_peered":"2024-09-21T03:15:59.462517+0000","last_clean":"2024-09-21T03:15:59.462517+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:59.462517+0000","last_undegraded":"2024-09-21T03:15:59.462517+0000","last_fullsized":"2024-09-21T03:15:59.462517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":297,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925677+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:16:41.925677+0000","last_peered":"2024-09-21T03:16:41.925677+0000","last_clean":"2024-09-21T03:16:41.925677+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:16:41.925677+0000","last_undegraded":"2024-09-21T03:16:41.925677+0000","last_fullsized":"2024-09-21T03:16:41.925677+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65314,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":182,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447951+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:59.447951+0000","last_peered":"2024-09-21T03:15:59.447951+0000","last_clean":"2024-09-21T03:15:59.447951+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:59.447951+0000","last_undegraded":"2024-09-21T03:15:59.447951+0000","last_fullsized":"2024-09-21T03:15:59.447951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447254+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:59.447254+0000","last_peered":"2024-09-21T03:15:59.447254+0000","last_clean":"2024-09-21T03:15:59.447254+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:59.447254+0000","last_undegraded":"2024-09-21T03:15:59.447254+0000","last_fullsized":"2024-09-21T03:15:59.447254+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460287+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:59.460287+0000","last_peered":"2024-09-21T03:15:59.460287+0000","last_clean":"2024-09-21T03:15:59.460287+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:59.460287+0000","last_undegraded":"2024-09-21T03:15:59.460287+0000","last_fullsized":"2024-09-21T03:15:59.460287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":145,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.939765+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:16:41.939765+0000","last_peered":"2024-09-21T03:16:41.939765+0000","last_clean":"2024-09-21T03:16:41.939765+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:16:41.939765+0000","last_undegraded":"2024-09-21T03:16:41.939765+0000","last_fullsized":"2024-09-21T03:16:41.939765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450468+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:59.450468+0000","last_peered":"2024-09-21T03:15:59.450468+0000","last_clean":"2024-09-21T03:15:59.450468+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:59.450468+0000","last_undegraded":"2024-09-21T03:15:59.450468+0000","last_fullsized":"2024-09-21T03:15:59.450468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453704+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:59.453704+0000","last_peered":"2024-09-21T03:15:59.453704+0000","last_clean":"2024-09-21T03:15:59.453704+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:59.453704+0000","last_undegraded":"2024-09-21T03:15:59.453704+0000","last_fullsized":"2024-09-21T03:15:59.453704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452996+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:59.452996+0000","last_peered":"2024-09-21T03:15:59.452996+0000","last_clean":"2024-09-21T03:15:59.452996+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:59.452996+0000","last_undegraded":"2024-09-21T03:15:59.452996+0000","last_fullsized":"2024-09-21T03:15:59.452996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451990+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:59.451990+0000","last_peered":"2024-09-21T03:15:59.451990+0000","last_clean":"2024-09-21T03:15:59.451990+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:59.451990+0000","last_undegraded":"2024-09-21T03:15:59.451990+0000","last_fullsized":"2024-09-21T03:15:59.451990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450791+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:59.450791+0000","last_peered":"2024-09-21T03:15:59.450791+0000","last_clean":"2024-09-21T03:15:59.450791+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:59.450791+0000","last_undegraded":"2024-09-21T03:15:59.450791+0000","last_fullsized":"2024-09-21T03:15:59.450791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'8","reported_seq":150,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.924913+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:16:40.924913+0000","last_peered":"2024-09-21T03:16:40.924913+0000","last_clean":"2024-09-21T03:16:40.924913+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:16:40.924913+0000","last_undegraded":"2024-09-21T03:16:40.924913+0000","last_fullsized":"2024-09-21T03:16:40.924913+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460998+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:59.460998+0000","last_peered":"2024-09-21T03:15:59.460998+0000","last_clean":"2024-09-21T03:15:59.460998+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:59.460998+0000","last_undegraded":"2024-09-21T03:15:59.460998+0000","last_fullsized":"2024-09-21T03:15:59.460998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259828+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259828+0000","last_peered":"2024-09-21T03:15:00.259828+0000","last_clean":"2024-09-21T03:15:00.259828+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259828+0000","last_undegraded":"2024-09-21T03:15:00.259828+0000","last_fullsized":"2024-09-21T03:15:00.259828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264069+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264069+0000","last_peered":"2024-09-21T03:15:00.264069+0000","last_clean":"2024-09-21T03:15:00.264069+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264069+0000","last_undegraded":"2024-09-21T03:15:00.264069+0000","last_fullsized":"2024-09-21T03:15:00.264069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441320+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:59.441320+0000","last_peered":"2024-09-21T03:15:59.441320+0000","last_clean":"2024-09-21T03:15:59.441320+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:59.441320+0000","last_undegraded":"2024-09-21T03:15:59.441320+0000","last_fullsized":"2024-09-21T03:15:59.441320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259780+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259780+0000","last_peered":"2024-09-21T03:15:00.259780+0000","last_clean":"2024-09-21T03:15:00.259780+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259780+0000","last_undegraded":"2024-09-21T03:15:00.259780+0000","last_fullsized":"2024-09-21T03:15:00.259780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446838+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:59.446838+0000","last_peered":"2024-09-21T03:15:59.446838+0000","last_clean":"2024-09-21T03:15:59.446838+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:59.446838+0000","last_undegraded":"2024-09-21T03:15:59.446838+0000","last_fullsized":"2024-09-21T03:15:59.446838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.937742+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:16:40.937742+0000","last_peered":"2024-09-21T03:16:40.937742+0000","last_clean":"2024-09-21T03:16:40.937742+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:16:40.937742+0000","last_undegraded":"2024-09-21T03:16:40.937742+0000","last_fullsized":"2024-09-21T03:16:40.937742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455106+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:59.455106+0000","last_peered":"2024-09-21T03:15:59.455106+0000","last_clean":"2024-09-21T03:15:59.455106+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:59.455106+0000","last_undegraded":"2024-09-21T03:15:59.455106+0000","last_fullsized":"2024-09-21T03:15:59.455106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:39.922099+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:16:39.922099+0000","last_peered":"2024-09-21T03:16:39.922099+0000","last_clean":"2024-09-21T03:16:39.922099+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:16:39.922099+0000","last_undegraded":"2024-09-21T03:16:39.922099+0000","last_fullsized":"2024-09-21T03:16:39.922099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451490+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:59.451490+0000","last_peered":"2024-09-21T03:15:59.451490+0000","last_clean":"2024-09-21T03:15:59.451490+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:59.451490+0000","last_undegraded":"2024-09-21T03:15:59.451490+0000","last_fullsized":"2024-09-21T03:15:59.451490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265641+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265641+0000","last_peered":"2024-09-21T03:15:00.265641+0000","last_clean":"2024-09-21T03:15:00.265641+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265641+0000","last_undegraded":"2024-09-21T03:15:00.265641+0000","last_fullsized":"2024-09-21T03:15:00.265641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448466+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:59.448466+0000","last_peered":"2024-09-21T03:15:59.448466+0000","last_clean":"2024-09-21T03:15:59.448466+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:59.448466+0000","last_undegraded":"2024-09-21T03:15:59.448466+0000","last_fullsized":"2024-09-21T03:15:59.448466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261679+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261679+0000","last_peered":"2024-09-21T03:15:00.261679+0000","last_clean":"2024-09-21T03:15:00.261679+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261679+0000","last_undegraded":"2024-09-21T03:15:00.261679+0000","last_fullsized":"2024-09-21T03:15:00.261679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446331+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:59.446331+0000","last_peered":"2024-09-21T03:15:59.446331+0000","last_clean":"2024-09-21T03:15:59.446331+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:59.446331+0000","last_undegraded":"2024-09-21T03:15:59.446331+0000","last_fullsized":"2024-09-21T03:15:59.446331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.622195+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:59.622195+0000","last_peered":"2024-09-21T03:15:59.622195+0000","last_clean":"2024-09-21T03:15:59.622195+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:59.622195+0000","last_undegraded":"2024-09-21T03:15:59.622195+0000","last_fullsized":"2024-09-21T03:15:59.622195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.449080+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:59.449080+0000","last_peered":"2024-09-21T03:15:59.449080+0000","last_clean":"2024-09-21T03:15:59.449080+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:59.449080+0000","last_undegraded":"2024-09-21T03:15:59.449080+0000","last_fullsized":"2024-09-21T03:15:59.449080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457757+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:59.457757+0000","last_peered":"2024-09-21T03:15:59.457757+0000","last_clean":"2024-09-21T03:15:59.457757+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:59.457757+0000","last_undegraded":"2024-09-21T03:15:59.457757+0000","last_fullsized":"2024-09-21T03:15:59.457757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.408350+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:59.408350+0000","last_peered":"2024-09-21T03:15:59.408350+0000","last_clean":"2024-09-21T03:15:59.408350+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:59.408350+0000","last_undegraded":"2024-09-21T03:15:59.408350+0000","last_fullsized":"2024-09-21T03:15:59.408350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450927+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:59.450927+0000","last_peered":"2024-09-21T03:15:59.450927+0000","last_clean":"2024-09-21T03:15:59.450927+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:59.450927+0000","last_undegraded":"2024-09-21T03:15:59.450927+0000","last_fullsized":"2024-09-21T03:15:59.450927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265986+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265986+0000","last_peered":"2024-09-21T03:15:00.265986+0000","last_clean":"2024-09-21T03:15:00.265986+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265986+0000","last_undegraded":"2024-09-21T03:15:00.265986+0000","last_fullsized":"2024-09-21T03:15:00.265986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445834+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:59.445834+0000","last_peered":"2024-09-21T03:15:59.445834+0000","last_clean":"2024-09-21T03:15:59.445834+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:59.445834+0000","last_undegraded":"2024-09-21T03:15:59.445834+0000","last_fullsized":"2024-09-21T03:15:59.445834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458762+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:59.458762+0000","last_peered":"2024-09-21T03:15:59.458762+0000","last_clean":"2024-09-21T03:15:59.458762+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:59.458762+0000","last_undegraded":"2024-09-21T03:15:59.458762+0000","last_fullsized":"2024-09-21T03:15:59.458762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455714+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:59.455714+0000","last_peered":"2024-09-21T03:15:59.455714+0000","last_clean":"2024-09-21T03:15:59.455714+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:59.455714+0000","last_undegraded":"2024-09-21T03:15:59.455714+0000","last_fullsized":"2024-09-21T03:15:59.455714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461974+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:59.461974+0000","last_peered":"2024-09-21T03:15:59.461974+0000","last_clean":"2024-09-21T03:15:59.461974+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:59.461974+0000","last_undegraded":"2024-09-21T03:15:59.461974+0000","last_fullsized":"2024-09-21T03:15:59.461974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462659+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:59.462659+0000","last_peered":"2024-09-21T03:15:59.462659+0000","last_clean":"2024-09-21T03:15:59.462659+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:59.462659+0000","last_undegraded":"2024-09-21T03:15:59.462659+0000","last_fullsized":"2024-09-21T03:15:59.462659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259071+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259071+0000","last_peered":"2024-09-21T03:15:00.259071+0000","last_clean":"2024-09-21T03:15:00.259071+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259071+0000","last_undegraded":"2024-09-21T03:15:00.259071+0000","last_fullsized":"2024-09-21T03:15:00.259071+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452714+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:59.452714+0000","last_peered":"2024-09-21T03:15:59.452714+0000","last_clean":"2024-09-21T03:15:59.452714+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:59.452714+0000","last_undegraded":"2024-09-21T03:15:59.452714+0000","last_fullsized":"2024-09-21T03:15:59.452714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":152,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.924547+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:16:41.924547+0000","last_peered":"2024-09-21T03:16:41.924547+0000","last_clean":"2024-09-21T03:16:41.924547+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:16:41.924547+0000","last_undegraded":"2024-09-21T03:16:41.924547+0000","last_fullsized":"2024-09-21T03:16:41.924547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72576,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1179,"num_read_kb":670,"num_write":258,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":417205,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072640,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":483163,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":482171,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":418123,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:16:43.959 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T03:16:43.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:16:44.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- 172.21.15.83:0/1832718835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218104930 msgr2=0x7f6218104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 --2- 172.21.15.83:0/1832718835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218104930 0x7f6218104d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802a410 comp rx=0 tx=0).stop 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- 172.21.15.83:0/1832718835 shutdown_connections 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 --2- 172.21.15.83:0/1832718835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621810dd50 0x7f6218110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 --2- 172.21.15.83:0/1832718835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6218105280 0x7f621810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 --2- 172.21.15.83:0/1832718835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218104930 0x7f6218104d40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- 172.21.15.83:0/1832718835 >> 172.21.15.83:0/1832718835 conn(0x7f62180fe380 msgr2=0x7f62181007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- 172.21.15.83:0/1832718835 shutdown_connections 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- 172.21.15.83:0/1832718835 wait complete. 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 Processor -- start 2024-09-21T03:16:44.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.103+0000 7f621e90b640 1 -- start start 2024-09-21T03:16:44.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 0x7f621819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:44.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:44.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621810dd50 0x7f621819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:44.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62181139c0 con 0x7f6218105280 2024-09-21T03:16:44.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6218113840 con 0x7f621810dd50 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6218113b40 con 0x7f6218104930 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60532/0 (socket says 172.21.15.83:60532) 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 -- 172.21.15.83:0/2651297122 learned_addr learned my addr 172.21.15.83:0/2651297122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621cea8640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621810dd50 0x7f621819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f6217fff640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 0x7f621819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 -- 172.21.15.83:0/2651297122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 msgr2=0x7f621819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 0x7f621819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 -- 172.21.15.83:0/2651297122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621810dd50 msgr2=0x7f621819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621810dd50 0x7f621819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 -- 172.21.15.83:0/2651297122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62181a02d0 con 0x7f6218105280 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f6217fff640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 0x7f621819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:44.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62177fe640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d740 tx=0x7f620c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c017070 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620c00de50 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.104+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62181a05c0 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.105+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f620 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.105+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a6d50 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.105+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218104d40 con 0x7f6218105280 2024-09-21T03:16:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.108+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f620c01a070 con 0x7f6218105280 2024-09-21T03:16:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.109+0000 7f62157fa640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 0x7f62000b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.109+0000 7f6217fff640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 0x7f62000b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.109+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f620c0e6eb0 con 0x7f6218105280 2024-09-21T03:16:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.109+0000 7f6217fff640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 0x7f62000b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62080096c0 tx=0x7f6208031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.112+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f620c0ad6a0 con 0x7f6218105280 2024-09-21T03:16:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.288+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6218107f20 con 0x7f62000b6560 2024-09-21T03:16:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.293+0000 7f62157fa640 1 -- 172.21.15.83:0/2651297122 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259134 (secure 0 0 0) 0x7f6218107f20 con 0x7f62000b6560 2024-09-21T03:16:44.298 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 msgr2=0x7f62000b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 0x7f62000b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62080096c0 tx=0x7f6208031400 comp rx=0 tx=0).stop 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 msgr2=0x7f621819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d740 tx=0x7f620c00dc00 comp rx=0 tx=0).stop 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 shutdown_connections 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62000b6560 0x7f62000b8a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621810dd50 0x7f621819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6218105280 0x7f621819a940 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 --2- 172.21.15.83:0/2651297122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6218104930 0x7f621819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 >> 172.21.15.83:0/2651297122 conn(0x7f62180fe380 msgr2=0x7f6218107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 shutdown_connections 2024-09-21T03:16:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:44.296+0000 7f621e90b640 1 -- 172.21.15.83:0/2651297122 wait complete. 2024-09-21T03:16:44.310 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-21T03:16:43.202369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663178,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1260,"num_read_kb":739,"num_write":383,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2952025,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351404,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124590420,"statfs":{"total":1151940427776,"available":1151580590080,"internally_reserved":0,"allocated":10727424,"data_stored":6970794,"data_compressed":176679,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":77,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003256"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461448+0000","last_change":"2024-09-21T03:14:36.613967+0000","last_active":"2024-09-21T03:15:59.461448+0000","last_peered":"2024-09-21T03:15:59.461448+0000","last_clean":"2024-09-21T03:15:59.461448+0000","last_became_active":"2024-09-21T03:14:36.613143+0000","last_became_peered":"2024-09-21T03:14:36.613143+0000","last_unstale":"2024-09-21T03:15:59.461448+0000","last_undegraded":"2024-09-21T03:15:59.461448+0000","last_fullsized":"2024-09-21T03:15:59.461448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:44.116502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454514+0000","last_change":"2024-09-21T03:14:36.626776+0000","last_active":"2024-09-21T03:15:59.454514+0000","last_peered":"2024-09-21T03:15:59.454514+0000","last_clean":"2024-09-21T03:15:59.454514+0000","last_became_active":"2024-09-21T03:14:36.626459+0000","last_became_peered":"2024-09-21T03:14:36.626459+0000","last_unstale":"2024-09-21T03:15:59.454514+0000","last_undegraded":"2024-09-21T03:15:59.454514+0000","last_fullsized":"2024-09-21T03:15:59.454514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:32.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462578+0000","last_change":"2024-09-21T03:14:36.620357+0000","last_active":"2024-09-21T03:15:59.462578+0000","last_peered":"2024-09-21T03:15:59.462578+0000","last_clean":"2024-09-21T03:15:59.462578+0000","last_became_active":"2024-09-21T03:14:36.620010+0000","last_became_peered":"2024-09-21T03:14:36.620010+0000","last_unstale":"2024-09-21T03:15:59.462578+0000","last_undegraded":"2024-09-21T03:15:59.462578+0000","last_fullsized":"2024-09-21T03:15:59.462578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:12.185864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459054+0000","last_change":"2024-09-21T03:14:36.621098+0000","last_active":"2024-09-21T03:15:59.459054+0000","last_peered":"2024-09-21T03:15:59.459054+0000","last_clean":"2024-09-21T03:15:59.459054+0000","last_became_active":"2024-09-21T03:14:36.620224+0000","last_became_peered":"2024-09-21T03:14:36.620224+0000","last_unstale":"2024-09-21T03:15:59.459054+0000","last_undegraded":"2024-09-21T03:15:59.459054+0000","last_fullsized":"2024-09-21T03:15:59.459054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:11:41.535706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436242+0000","last_change":"2024-09-21T03:14:36.609208+0000","last_active":"2024-09-21T03:15:59.436242+0000","last_peered":"2024-09-21T03:15:59.436242+0000","last_clean":"2024-09-21T03:15:59.436242+0000","last_became_active":"2024-09-21T03:14:36.607229+0000","last_became_peered":"2024-09-21T03:14:36.607229+0000","last_unstale":"2024-09-21T03:15:59.436242+0000","last_undegraded":"2024-09-21T03:15:59.436242+0000","last_fullsized":"2024-09-21T03:15:59.436242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:45.040152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458740+0000","last_change":"2024-09-21T03:14:36.618271+0000","last_active":"2024-09-21T03:15:59.458740+0000","last_peered":"2024-09-21T03:15:59.458740+0000","last_clean":"2024-09-21T03:15:59.458740+0000","last_became_active":"2024-09-21T03:14:36.617836+0000","last_became_peered":"2024-09-21T03:14:36.617836+0000","last_unstale":"2024-09-21T03:15:59.458740+0000","last_undegraded":"2024-09-21T03:15:59.458740+0000","last_fullsized":"2024-09-21T03:15:59.458740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:01.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445973+0000","last_change":"2024-09-21T03:14:36.621865+0000","last_active":"2024-09-21T03:15:59.445973+0000","last_peered":"2024-09-21T03:15:59.445973+0000","last_clean":"2024-09-21T03:15:59.445973+0000","last_became_active":"2024-09-21T03:14:36.620516+0000","last_became_peered":"2024-09-21T03:14:36.620516+0000","last_unstale":"2024-09-21T03:15:59.445973+0000","last_undegraded":"2024-09-21T03:15:59.445973+0000","last_fullsized":"2024-09-21T03:15:59.445973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:51.856381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452622+0000","last_change":"2024-09-21T03:14:36.618644+0000","last_active":"2024-09-21T03:15:59.452622+0000","last_peered":"2024-09-21T03:15:59.452622+0000","last_clean":"2024-09-21T03:15:59.452622+0000","last_became_active":"2024-09-21T03:14:36.618316+0000","last_became_peered":"2024-09-21T03:14:36.618316+0000","last_unstale":"2024-09-21T03:15:59.452622+0000","last_undegraded":"2024-09-21T03:15:59.452622+0000","last_fullsized":"2024-09-21T03:15:59.452622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:25.195358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458000+0000","last_change":"2024-09-21T03:14:36.616849+0000","last_active":"2024-09-21T03:15:59.458000+0000","last_peered":"2024-09-21T03:15:59.458000+0000","last_clean":"2024-09-21T03:15:59.458000+0000","last_became_active":"2024-09-21T03:14:36.615707+0000","last_became_peered":"2024-09-21T03:14:36.615707+0000","last_unstale":"2024-09-21T03:15:59.458000+0000","last_undegraded":"2024-09-21T03:15:59.458000+0000","last_fullsized":"2024-09-21T03:15:59.458000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:29.588991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.459668+0000","last_change":"2024-09-21T03:14:36.622819+0000","last_active":"2024-09-21T03:15:59.459668+0000","last_peered":"2024-09-21T03:15:59.459668+0000","last_clean":"2024-09-21T03:15:59.459668+0000","last_became_active":"2024-09-21T03:14:36.622443+0000","last_became_peered":"2024-09-21T03:14:36.622443+0000","last_unstale":"2024-09-21T03:15:59.459668+0000","last_undegraded":"2024-09-21T03:15:59.459668+0000","last_fullsized":"2024-09-21T03:15:59.459668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:50.033556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460855+0000","last_change":"2024-09-21T03:14:36.620925+0000","last_active":"2024-09-21T03:15:59.460855+0000","last_peered":"2024-09-21T03:15:59.460855+0000","last_clean":"2024-09-21T03:15:59.460855+0000","last_became_active":"2024-09-21T03:14:36.620054+0000","last_became_peered":"2024-09-21T03:14:36.620054+0000","last_unstale":"2024-09-21T03:15:59.460855+0000","last_undegraded":"2024-09-21T03:15:59.460855+0000","last_fullsized":"2024-09-21T03:15:59.460855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:00.222571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.442827+0000","last_change":"2024-09-21T03:14:36.626323+0000","last_active":"2024-09-21T03:15:59.442827+0000","last_peered":"2024-09-21T03:15:59.442827+0000","last_clean":"2024-09-21T03:15:59.442827+0000","last_became_active":"2024-09-21T03:14:36.625939+0000","last_became_peered":"2024-09-21T03:14:36.625939+0000","last_unstale":"2024-09-21T03:15:59.442827+0000","last_undegraded":"2024-09-21T03:15:59.442827+0000","last_fullsized":"2024-09-21T03:15:59.442827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:00.959802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452352+0000","last_change":"2024-09-21T03:14:36.626608+0000","last_active":"2024-09-21T03:15:59.452352+0000","last_peered":"2024-09-21T03:15:59.452352+0000","last_clean":"2024-09-21T03:15:59.452352+0000","last_became_active":"2024-09-21T03:14:36.626246+0000","last_became_peered":"2024-09-21T03:14:36.626246+0000","last_unstale":"2024-09-21T03:15:59.452352+0000","last_undegraded":"2024-09-21T03:15:59.452352+0000","last_fullsized":"2024-09-21T03:15:59.452352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:34.423839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454556+0000","last_change":"2024-09-21T03:14:36.627536+0000","last_active":"2024-09-21T03:15:59.454556+0000","last_peered":"2024-09-21T03:15:59.454556+0000","last_clean":"2024-09-21T03:15:59.454556+0000","last_became_active":"2024-09-21T03:14:36.627086+0000","last_became_peered":"2024-09-21T03:14:36.627086+0000","last_unstale":"2024-09-21T03:15:59.454556+0000","last_undegraded":"2024-09-21T03:15:59.454556+0000","last_fullsized":"2024-09-21T03:15:59.454556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:55.384429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453334+0000","last_change":"2024-09-21T03:14:36.622994+0000","last_active":"2024-09-21T03:15:59.453334+0000","last_peered":"2024-09-21T03:15:59.453334+0000","last_clean":"2024-09-21T03:15:59.453334+0000","last_became_active":"2024-09-21T03:14:36.622704+0000","last_became_peered":"2024-09-21T03:14:36.622704+0000","last_unstale":"2024-09-21T03:15:59.453334+0000","last_undegraded":"2024-09-21T03:15:59.453334+0000","last_fullsized":"2024-09-21T03:15:59.453334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:03.843488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462531+0000","last_change":"2024-09-21T03:14:36.617805+0000","last_active":"2024-09-21T03:15:59.462531+0000","last_peered":"2024-09-21T03:15:59.462531+0000","last_clean":"2024-09-21T03:15:59.462531+0000","last_became_active":"2024-09-21T03:14:36.617448+0000","last_became_peered":"2024-09-21T03:14:36.617448+0000","last_unstale":"2024-09-21T03:15:59.462531+0000","last_undegraded":"2024-09-21T03:15:59.462531+0000","last_fullsized":"2024-09-21T03:15:59.462531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:41.039263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461567+0000","last_change":"2024-09-21T03:14:36.614330+0000","last_active":"2024-09-21T03:15:59.461567+0000","last_peered":"2024-09-21T03:15:59.461567+0000","last_clean":"2024-09-21T03:15:59.461567+0000","last_became_active":"2024-09-21T03:14:36.613618+0000","last_became_peered":"2024-09-21T03:14:36.613618+0000","last_unstale":"2024-09-21T03:15:59.461567+0000","last_undegraded":"2024-09-21T03:15:59.461567+0000","last_fullsized":"2024-09-21T03:15:59.461567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.437070+0000","last_change":"2024-09-21T03:14:36.626027+0000","last_active":"2024-09-21T03:15:59.437070+0000","last_peered":"2024-09-21T03:15:59.437070+0000","last_clean":"2024-09-21T03:15:59.437070+0000","last_became_active":"2024-09-21T03:14:36.625653+0000","last_became_peered":"2024-09-21T03:14:36.625653+0000","last_unstale":"2024-09-21T03:15:59.437070+0000","last_undegraded":"2024-09-21T03:15:59.437070+0000","last_fullsized":"2024-09-21T03:15:59.437070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:34.414119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448112+0000","last_change":"2024-09-21T03:14:36.614013+0000","last_active":"2024-09-21T03:15:59.448112+0000","last_peered":"2024-09-21T03:15:59.448112+0000","last_clean":"2024-09-21T03:15:59.448112+0000","last_became_active":"2024-09-21T03:14:36.613553+0000","last_became_peered":"2024-09-21T03:14:36.613553+0000","last_unstale":"2024-09-21T03:15:59.448112+0000","last_undegraded":"2024-09-21T03:15:59.448112+0000","last_fullsized":"2024-09-21T03:15:59.448112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:32.915467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452845+0000","last_change":"2024-09-21T03:14:36.619485+0000","last_active":"2024-09-21T03:15:59.452845+0000","last_peered":"2024-09-21T03:15:59.452845+0000","last_clean":"2024-09-21T03:15:59.452845+0000","last_became_active":"2024-09-21T03:14:36.619107+0000","last_became_peered":"2024-09-21T03:14:36.619107+0000","last_unstale":"2024-09-21T03:15:59.452845+0000","last_undegraded":"2024-09-21T03:15:59.452845+0000","last_fullsized":"2024-09-21T03:15:59.452845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:27.198254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447478+0000","last_change":"2024-09-21T03:14:36.622816+0000","last_active":"2024-09-21T03:15:59.447478+0000","last_peered":"2024-09-21T03:15:59.447478+0000","last_clean":"2024-09-21T03:15:59.447478+0000","last_became_active":"2024-09-21T03:14:36.622025+0000","last_became_peered":"2024-09-21T03:14:36.622025+0000","last_unstale":"2024-09-21T03:15:59.447478+0000","last_undegraded":"2024-09-21T03:15:59.447478+0000","last_fullsized":"2024-09-21T03:15:59.447478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:56.148794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462703+0000","last_change":"2024-09-21T03:14:36.614884+0000","last_active":"2024-09-21T03:15:59.462703+0000","last_peered":"2024-09-21T03:15:59.462703+0000","last_clean":"2024-09-21T03:15:59.462703+0000","last_became_active":"2024-09-21T03:14:36.614403+0000","last_became_peered":"2024-09-21T03:14:36.614403+0000","last_unstale":"2024-09-21T03:15:59.462703+0000","last_undegraded":"2024-09-21T03:15:59.462703+0000","last_fullsized":"2024-09-21T03:15:59.462703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:46.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448989+0000","last_change":"2024-09-21T03:14:36.620324+0000","last_active":"2024-09-21T03:15:59.448989+0000","last_peered":"2024-09-21T03:15:59.448989+0000","last_clean":"2024-09-21T03:15:59.448989+0000","last_became_active":"2024-09-21T03:14:36.620029+0000","last_became_peered":"2024-09-21T03:14:36.620029+0000","last_unstale":"2024-09-21T03:15:59.448989+0000","last_undegraded":"2024-09-21T03:15:59.448989+0000","last_fullsized":"2024-09-21T03:15:59.448989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:28.237631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451608+0000","last_change":"2024-09-21T03:14:36.615231+0000","last_active":"2024-09-21T03:15:59.451608+0000","last_peered":"2024-09-21T03:15:59.451608+0000","last_clean":"2024-09-21T03:15:59.451608+0000","last_became_active":"2024-09-21T03:14:36.614757+0000","last_became_peered":"2024-09-21T03:14:36.614757+0000","last_unstale":"2024-09-21T03:15:59.451608+0000","last_undegraded":"2024-09-21T03:15:59.451608+0000","last_fullsized":"2024-09-21T03:15:59.451608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:46.909872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436971+0000","last_change":"2024-09-21T03:14:36.621080+0000","last_active":"2024-09-21T03:15:59.436971+0000","last_peered":"2024-09-21T03:15:59.436971+0000","last_clean":"2024-09-21T03:15:59.436971+0000","last_became_active":"2024-09-21T03:14:36.620769+0000","last_became_peered":"2024-09-21T03:14:36.620769+0000","last_unstale":"2024-09-21T03:15:59.436971+0000","last_undegraded":"2024-09-21T03:15:59.436971+0000","last_fullsized":"2024-09-21T03:15:59.436971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:34.354073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.438447+0000","last_change":"2024-09-21T03:14:36.620820+0000","last_active":"2024-09-21T03:15:59.438447+0000","last_peered":"2024-09-21T03:15:59.438447+0000","last_clean":"2024-09-21T03:15:59.438447+0000","last_became_active":"2024-09-21T03:14:36.620464+0000","last_became_peered":"2024-09-21T03:14:36.620464+0000","last_unstale":"2024-09-21T03:15:59.438447+0000","last_undegraded":"2024-09-21T03:15:59.438447+0000","last_fullsized":"2024-09-21T03:15:59.438447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:00.602988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445817+0000","last_change":"2024-09-21T03:14:36.617222+0000","last_active":"2024-09-21T03:15:59.445817+0000","last_peered":"2024-09-21T03:15:59.445817+0000","last_clean":"2024-09-21T03:15:59.445817+0000","last_became_active":"2024-09-21T03:14:36.616511+0000","last_became_peered":"2024-09-21T03:14:36.616511+0000","last_unstale":"2024-09-21T03:15:59.445817+0000","last_undegraded":"2024-09-21T03:15:59.445817+0000","last_fullsized":"2024-09-21T03:15:59.445817+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:48.816212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445217+0000","last_change":"2024-09-21T03:14:36.606198+0000","last_active":"2024-09-21T03:15:59.445217+0000","last_peered":"2024-09-21T03:15:59.445217+0000","last_clean":"2024-09-21T03:15:59.445217+0000","last_became_active":"2024-09-21T03:14:36.605174+0000","last_became_peered":"2024-09-21T03:14:36.605174+0000","last_unstale":"2024-09-21T03:15:59.445217+0000","last_undegraded":"2024-09-21T03:15:59.445217+0000","last_fullsized":"2024-09-21T03:15:59.445217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:54.627695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457735+0000","last_change":"2024-09-21T03:14:36.606379+0000","last_active":"2024-09-21T03:15:59.457735+0000","last_peered":"2024-09-21T03:15:59.457735+0000","last_clean":"2024-09-21T03:15:59.457735+0000","last_became_active":"2024-09-21T03:14:36.606021+0000","last_became_peered":"2024-09-21T03:14:36.606021+0000","last_unstale":"2024-09-21T03:15:59.457735+0000","last_undegraded":"2024-09-21T03:15:59.457735+0000","last_fullsized":"2024-09-21T03:15:59.457735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:11.565116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445603+0000","last_change":"2024-09-21T03:14:36.614006+0000","last_active":"2024-09-21T03:15:59.445603+0000","last_peered":"2024-09-21T03:15:59.445603+0000","last_clean":"2024-09-21T03:15:59.445603+0000","last_became_active":"2024-09-21T03:14:36.613124+0000","last_became_peered":"2024-09-21T03:14:36.613124+0000","last_unstale":"2024-09-21T03:15:59.445603+0000","last_undegraded":"2024-09-21T03:15:59.445603+0000","last_fullsized":"2024-09-21T03:15:59.445603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:12.353758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448912+0000","last_change":"2024-09-21T03:15:00.269259+0000","last_active":"2024-09-21T03:15:59.448912+0000","last_peered":"2024-09-21T03:15:59.448912+0000","last_clean":"2024-09-21T03:15:59.448912+0000","last_became_active":"2024-09-21T03:15:00.268797+0000","last_became_peered":"2024-09-21T03:15:00.268797+0000","last_unstale":"2024-09-21T03:15:59.448912+0000","last_undegraded":"2024-09-21T03:15:59.448912+0000","last_fullsized":"2024-09-21T03:15:59.448912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:08.202143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452258+0000","last_change":"2024-09-21T03:14:36.623657+0000","last_active":"2024-09-21T03:15:59.452258+0000","last_peered":"2024-09-21T03:15:59.452258+0000","last_clean":"2024-09-21T03:15:59.452258+0000","last_became_active":"2024-09-21T03:14:36.623283+0000","last_became_peered":"2024-09-21T03:14:36.623283+0000","last_unstale":"2024-09-21T03:15:59.452258+0000","last_undegraded":"2024-09-21T03:15:59.452258+0000","last_fullsized":"2024-09-21T03:15:59.452258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:08.159883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.436994+0000","last_change":"2024-09-21T03:14:36.612725+0000","last_active":"2024-09-21T03:15:59.436994+0000","last_peered":"2024-09-21T03:15:59.436994+0000","last_clean":"2024-09-21T03:15:59.436994+0000","last_became_active":"2024-09-21T03:14:36.612364+0000","last_became_peered":"2024-09-21T03:14:36.612364+0000","last_unstale":"2024-09-21T03:15:59.436994+0000","last_undegraded":"2024-09-21T03:15:59.436994+0000","last_fullsized":"2024-09-21T03:15:59.436994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:31.223650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:37.920447+0000","last_change":"2024-09-21T03:14:34.601183+0000","last_active":"2024-09-21T03:16:37.920447+0000","last_peered":"2024-09-21T03:16:37.920447+0000","last_clean":"2024-09-21T03:16:37.920447+0000","last_became_active":"2024-09-21T03:14:34.600353+0000","last_became_peered":"2024-09-21T03:14:34.600353+0000","last_unstale":"2024-09-21T03:16:37.920447+0000","last_undegraded":"2024-09-21T03:16:37.920447+0000","last_fullsized":"2024-09-21T03:16:37.920447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:57.135233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454550+0000","last_change":"2024-09-21T03:14:36.621857+0000","last_active":"2024-09-21T03:15:59.454550+0000","last_peered":"2024-09-21T03:15:59.454550+0000","last_clean":"2024-09-21T03:15:59.454550+0000","last_became_active":"2024-09-21T03:14:36.621487+0000","last_became_peered":"2024-09-21T03:14:36.621487+0000","last_unstale":"2024-09-21T03:15:59.454550+0000","last_undegraded":"2024-09-21T03:15:59.454550+0000","last_fullsized":"2024-09-21T03:15:59.454550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:38.783077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.273520+0000","last_change":"2024-09-21T03:15:00.273384+0000","last_active":"2024-09-21T03:15:00.273520+0000","last_peered":"2024-09-21T03:15:00.273520+0000","last_clean":"2024-09-21T03:15:00.273520+0000","last_became_active":"2024-09-21T03:15:00.266709+0000","last_became_peered":"2024-09-21T03:15:00.266709+0000","last_unstale":"2024-09-21T03:15:00.273520+0000","last_undegraded":"2024-09-21T03:15:00.273520+0000","last_fullsized":"2024-09-21T03:15:00.273520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:48.629161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265506+0000","last_change":"2024-09-21T03:14:34.596438+0000","last_active":"2024-09-21T03:15:00.265506+0000","last_peered":"2024-09-21T03:15:00.265506+0000","last_clean":"2024-09-21T03:15:00.265506+0000","last_became_active":"2024-09-21T03:14:34.596088+0000","last_became_peered":"2024-09-21T03:14:34.596088+0000","last_unstale":"2024-09-21T03:15:00.265506+0000","last_undegraded":"2024-09-21T03:15:00.265506+0000","last_fullsized":"2024-09-21T03:15:00.265506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:48.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450571+0000","last_change":"2024-09-21T03:14:36.606214+0000","last_active":"2024-09-21T03:15:59.450571+0000","last_peered":"2024-09-21T03:15:59.450571+0000","last_clean":"2024-09-21T03:15:59.450571+0000","last_became_active":"2024-09-21T03:14:36.605819+0000","last_became_peered":"2024-09-21T03:14:36.605819+0000","last_unstale":"2024-09-21T03:15:59.450571+0000","last_undegraded":"2024-09-21T03:15:59.450571+0000","last_fullsized":"2024-09-21T03:15:59.450571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:17.940371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264319+0000","last_change":"2024-09-21T03:14:34.599602+0000","last_active":"2024-09-21T03:15:00.264319+0000","last_peered":"2024-09-21T03:15:00.264319+0000","last_clean":"2024-09-21T03:15:00.264319+0000","last_became_active":"2024-09-21T03:14:34.599234+0000","last_became_peered":"2024-09-21T03:14:34.599234+0000","last_unstale":"2024-09-21T03:15:00.264319+0000","last_undegraded":"2024-09-21T03:15:00.264319+0000","last_fullsized":"2024-09-21T03:15:00.264319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:44.763384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441638+0000","last_change":"2024-09-21T03:14:36.626795+0000","last_active":"2024-09-21T03:15:59.441638+0000","last_peered":"2024-09-21T03:15:59.441638+0000","last_clean":"2024-09-21T03:15:59.441638+0000","last_became_active":"2024-09-21T03:14:36.626455+0000","last_became_peered":"2024-09-21T03:14:36.626455+0000","last_unstale":"2024-09-21T03:15:59.441638+0000","last_undegraded":"2024-09-21T03:15:59.441638+0000","last_fullsized":"2024-09-21T03:15:59.441638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:55.095679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":257,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925557+0000","last_change":"2024-09-21T03:14:34.597033+0000","last_active":"2024-09-21T03:16:41.925557+0000","last_peered":"2024-09-21T03:16:41.925557+0000","last_clean":"2024-09-21T03:16:41.925557+0000","last_became_active":"2024-09-21T03:14:34.596638+0000","last_became_peered":"2024-09-21T03:14:34.596638+0000","last_unstale":"2024-09-21T03:16:41.925557+0000","last_undegraded":"2024-09-21T03:16:41.925557+0000","last_fullsized":"2024-09-21T03:16:41.925557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:20.489259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461606+0000","last_change":"2024-09-21T03:14:36.627281+0000","last_active":"2024-09-21T03:15:59.461606+0000","last_peered":"2024-09-21T03:15:59.461606+0000","last_clean":"2024-09-21T03:15:59.461606+0000","last_became_active":"2024-09-21T03:14:36.626892+0000","last_became_peered":"2024-09-21T03:14:36.626892+0000","last_unstale":"2024-09-21T03:15:59.461606+0000","last_undegraded":"2024-09-21T03:15:59.461606+0000","last_fullsized":"2024-09-21T03:15:59.461606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:02.166855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260122+0000","last_change":"2024-09-21T03:14:34.600514+0000","last_active":"2024-09-21T03:15:00.260122+0000","last_peered":"2024-09-21T03:15:00.260122+0000","last_clean":"2024-09-21T03:15:00.260122+0000","last_became_active":"2024-09-21T03:14:34.600114+0000","last_became_peered":"2024-09-21T03:14:34.600114+0000","last_unstale":"2024-09-21T03:15:00.260122+0000","last_undegraded":"2024-09-21T03:15:00.260122+0000","last_fullsized":"2024-09-21T03:15:00.260122+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:51:24.740265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.454275+0000","last_change":"2024-09-21T03:14:36.616253+0000","last_active":"2024-09-21T03:15:59.454275+0000","last_peered":"2024-09-21T03:15:59.454275+0000","last_clean":"2024-09-21T03:15:59.454275+0000","last_became_active":"2024-09-21T03:14:36.615895+0000","last_became_peered":"2024-09-21T03:14:36.615895+0000","last_unstale":"2024-09-21T03:15:59.454275+0000","last_undegraded":"2024-09-21T03:15:59.454275+0000","last_fullsized":"2024-09-21T03:15:59.454275+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:57.408816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264107+0000","last_change":"2024-09-21T03:14:35.076180+0000","last_active":"2024-09-21T03:15:00.264107+0000","last_peered":"2024-09-21T03:15:00.264107+0000","last_clean":"2024-09-21T03:15:00.264107+0000","last_became_active":"2024-09-21T03:14:35.075497+0000","last_became_peered":"2024-09-21T03:14:35.075497+0000","last_unstale":"2024-09-21T03:15:00.264107+0000","last_undegraded":"2024-09-21T03:15:00.264107+0000","last_fullsized":"2024-09-21T03:15:00.264107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.901389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462517+0000","last_change":"2024-09-21T03:14:36.616857+0000","last_active":"2024-09-21T03:15:59.462517+0000","last_peered":"2024-09-21T03:15:59.462517+0000","last_clean":"2024-09-21T03:15:59.462517+0000","last_became_active":"2024-09-21T03:14:36.616452+0000","last_became_peered":"2024-09-21T03:14:36.616452+0000","last_unstale":"2024-09-21T03:15:59.462517+0000","last_undegraded":"2024-09-21T03:15:59.462517+0000","last_fullsized":"2024-09-21T03:15:59.462517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:56.150780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":297,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.925677+0000","last_change":"2024-09-21T03:14:34.594056+0000","last_active":"2024-09-21T03:16:41.925677+0000","last_peered":"2024-09-21T03:16:41.925677+0000","last_clean":"2024-09-21T03:16:41.925677+0000","last_became_active":"2024-09-21T03:14:34.593076+0000","last_became_peered":"2024-09-21T03:14:34.593076+0000","last_unstale":"2024-09-21T03:16:41.925677+0000","last_undegraded":"2024-09-21T03:16:41.925677+0000","last_fullsized":"2024-09-21T03:16:41.925677+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:55.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65314,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":182,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447951+0000","last_change":"2024-09-21T03:14:36.614355+0000","last_active":"2024-09-21T03:15:59.447951+0000","last_peered":"2024-09-21T03:15:59.447951+0000","last_clean":"2024-09-21T03:15:59.447951+0000","last_became_active":"2024-09-21T03:14:36.613773+0000","last_became_peered":"2024-09-21T03:14:36.613773+0000","last_unstale":"2024-09-21T03:15:59.447951+0000","last_undegraded":"2024-09-21T03:15:59.447951+0000","last_fullsized":"2024-09-21T03:15:59.447951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:36.924291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.277873+0000","last_change":"2024-09-21T03:15:00.277379+0000","last_active":"2024-09-21T03:15:00.277873+0000","last_peered":"2024-09-21T03:15:00.277873+0000","last_clean":"2024-09-21T03:15:00.277873+0000","last_became_active":"2024-09-21T03:15:00.265530+0000","last_became_peered":"2024-09-21T03:15:00.265530+0000","last_unstale":"2024-09-21T03:15:00.277873+0000","last_undegraded":"2024-09-21T03:15:00.277873+0000","last_fullsized":"2024-09-21T03:15:00.277873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:56.379283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.447254+0000","last_change":"2024-09-21T03:14:36.622637+0000","last_active":"2024-09-21T03:15:59.447254+0000","last_peered":"2024-09-21T03:15:59.447254+0000","last_clean":"2024-09-21T03:15:59.447254+0000","last_became_active":"2024-09-21T03:14:36.622323+0000","last_became_peered":"2024-09-21T03:14:36.622323+0000","last_unstale":"2024-09-21T03:15:59.447254+0000","last_undegraded":"2024-09-21T03:15:59.447254+0000","last_fullsized":"2024-09-21T03:15:59.447254+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.627032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263653+0000","last_change":"2024-09-21T03:14:34.602075+0000","last_active":"2024-09-21T03:15:00.263653+0000","last_peered":"2024-09-21T03:15:00.263653+0000","last_clean":"2024-09-21T03:15:00.263653+0000","last_became_active":"2024-09-21T03:14:34.601661+0000","last_became_peered":"2024-09-21T03:14:34.601661+0000","last_unstale":"2024-09-21T03:15:00.263653+0000","last_undegraded":"2024-09-21T03:15:00.263653+0000","last_fullsized":"2024-09-21T03:15:00.263653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:17.366418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460287+0000","last_change":"2024-09-21T03:14:36.618256+0000","last_active":"2024-09-21T03:15:59.460287+0000","last_peered":"2024-09-21T03:15:59.460287+0000","last_clean":"2024-09-21T03:15:59.460287+0000","last_became_active":"2024-09-21T03:14:36.617865+0000","last_became_peered":"2024-09-21T03:14:36.617865+0000","last_unstale":"2024-09-21T03:15:59.460287+0000","last_undegraded":"2024-09-21T03:15:59.460287+0000","last_fullsized":"2024-09-21T03:15:59.460287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:55.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":145,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.939765+0000","last_change":"2024-09-21T03:14:34.601960+0000","last_active":"2024-09-21T03:16:41.939765+0000","last_peered":"2024-09-21T03:16:41.939765+0000","last_clean":"2024-09-21T03:16:41.939765+0000","last_became_active":"2024-09-21T03:14:34.601596+0000","last_became_peered":"2024-09-21T03:14:34.601596+0000","last_unstale":"2024-09-21T03:16:41.939765+0000","last_undegraded":"2024-09-21T03:16:41.939765+0000","last_fullsized":"2024-09-21T03:16:41.939765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:59.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450468+0000","last_change":"2024-09-21T03:14:36.626556+0000","last_active":"2024-09-21T03:15:59.450468+0000","last_peered":"2024-09-21T03:15:59.450468+0000","last_clean":"2024-09-21T03:15:59.450468+0000","last_became_active":"2024-09-21T03:14:36.625873+0000","last_became_peered":"2024-09-21T03:14:36.625873+0000","last_unstale":"2024-09-21T03:15:59.450468+0000","last_undegraded":"2024-09-21T03:15:59.450468+0000","last_fullsized":"2024-09-21T03:15:59.450468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:46.086691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260405+0000","last_change":"2024-09-21T03:14:35.075574+0000","last_active":"2024-09-21T03:15:00.260405+0000","last_peered":"2024-09-21T03:15:00.260405+0000","last_clean":"2024-09-21T03:15:00.260405+0000","last_became_active":"2024-09-21T03:14:35.075072+0000","last_became_peered":"2024-09-21T03:14:35.075072+0000","last_unstale":"2024-09-21T03:15:00.260405+0000","last_undegraded":"2024-09-21T03:15:00.260405+0000","last_fullsized":"2024-09-21T03:15:00.260405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:16.361429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.453704+0000","last_change":"2024-09-21T03:14:36.617121+0000","last_active":"2024-09-21T03:15:59.453704+0000","last_peered":"2024-09-21T03:15:59.453704+0000","last_clean":"2024-09-21T03:15:59.453704+0000","last_became_active":"2024-09-21T03:14:36.616760+0000","last_became_peered":"2024-09-21T03:14:36.616760+0000","last_unstale":"2024-09-21T03:15:59.453704+0000","last_undegraded":"2024-09-21T03:15:59.453704+0000","last_fullsized":"2024-09-21T03:15:59.453704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:32.505461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260949+0000","last_change":"2024-09-21T03:14:34.599371+0000","last_active":"2024-09-21T03:15:00.260949+0000","last_peered":"2024-09-21T03:15:00.260949+0000","last_clean":"2024-09-21T03:15:00.260949+0000","last_became_active":"2024-09-21T03:14:34.598852+0000","last_became_peered":"2024-09-21T03:14:34.598852+0000","last_unstale":"2024-09-21T03:15:00.260949+0000","last_undegraded":"2024-09-21T03:15:00.260949+0000","last_fullsized":"2024-09-21T03:15:00.260949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.347630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452996+0000","last_change":"2024-09-21T03:14:36.619811+0000","last_active":"2024-09-21T03:15:59.452996+0000","last_peered":"2024-09-21T03:15:59.452996+0000","last_clean":"2024-09-21T03:15:59.452996+0000","last_became_active":"2024-09-21T03:14:36.619143+0000","last_became_peered":"2024-09-21T03:14:36.619143+0000","last_unstale":"2024-09-21T03:15:59.452996+0000","last_undegraded":"2024-09-21T03:15:59.452996+0000","last_fullsized":"2024-09-21T03:15:59.452996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:54.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265203+0000","last_change":"2024-09-21T03:14:34.592729+0000","last_active":"2024-09-21T03:15:00.265203+0000","last_peered":"2024-09-21T03:15:00.265203+0000","last_clean":"2024-09-21T03:15:00.265203+0000","last_became_active":"2024-09-21T03:14:34.592393+0000","last_became_peered":"2024-09-21T03:14:34.592393+0000","last_unstale":"2024-09-21T03:15:00.265203+0000","last_undegraded":"2024-09-21T03:15:00.265203+0000","last_fullsized":"2024-09-21T03:15:00.265203+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:57.600218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451990+0000","last_change":"2024-09-21T03:14:36.612200+0000","last_active":"2024-09-21T03:15:59.451990+0000","last_peered":"2024-09-21T03:15:59.451990+0000","last_clean":"2024-09-21T03:15:59.451990+0000","last_became_active":"2024-09-21T03:14:36.611756+0000","last_became_peered":"2024-09-21T03:14:36.611756+0000","last_unstale":"2024-09-21T03:15:59.451990+0000","last_undegraded":"2024-09-21T03:15:59.451990+0000","last_fullsized":"2024-09-21T03:15:59.451990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:49.156826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264584+0000","last_change":"2024-09-21T03:14:34.593670+0000","last_active":"2024-09-21T03:15:00.264584+0000","last_peered":"2024-09-21T03:15:00.264584+0000","last_clean":"2024-09-21T03:15:00.264584+0000","last_became_active":"2024-09-21T03:14:34.593267+0000","last_became_peered":"2024-09-21T03:14:34.593267+0000","last_unstale":"2024-09-21T03:15:00.264584+0000","last_undegraded":"2024-09-21T03:15:00.264584+0000","last_fullsized":"2024-09-21T03:15:00.264584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:39.784606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450791+0000","last_change":"2024-09-21T03:15:00.270911+0000","last_active":"2024-09-21T03:15:59.450791+0000","last_peered":"2024-09-21T03:15:59.450791+0000","last_clean":"2024-09-21T03:15:59.450791+0000","last_became_active":"2024-09-21T03:15:00.266891+0000","last_became_peered":"2024-09-21T03:15:00.266891+0000","last_unstale":"2024-09-21T03:15:59.450791+0000","last_undegraded":"2024-09-21T03:15:59.450791+0000","last_fullsized":"2024-09-21T03:15:59.450791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.690233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'8","reported_seq":150,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.924913+0000","last_change":"2024-09-21T03:14:35.550372+0000","last_active":"2024-09-21T03:16:40.924913+0000","last_peered":"2024-09-21T03:16:40.924913+0000","last_clean":"2024-09-21T03:16:40.924913+0000","last_became_active":"2024-09-21T03:14:35.549902+0000","last_became_peered":"2024-09-21T03:14:35.549902+0000","last_unstale":"2024-09-21T03:16:40.924913+0000","last_undegraded":"2024-09-21T03:16:40.924913+0000","last_fullsized":"2024-09-21T03:16:40.924913+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:47.024734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.460998+0000","last_change":"2024-09-21T03:14:36.614059+0000","last_active":"2024-09-21T03:15:59.460998+0000","last_peered":"2024-09-21T03:15:59.460998+0000","last_clean":"2024-09-21T03:15:59.460998+0000","last_became_active":"2024-09-21T03:14:36.613683+0000","last_became_peered":"2024-09-21T03:14:36.613683+0000","last_unstale":"2024-09-21T03:15:59.460998+0000","last_undegraded":"2024-09-21T03:15:59.460998+0000","last_fullsized":"2024-09-21T03:15:59.460998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:20.358338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259828+0000","last_change":"2024-09-21T03:13:23.279399+0000","last_active":"2024-09-21T03:15:00.259828+0000","last_peered":"2024-09-21T03:15:00.259828+0000","last_clean":"2024-09-21T03:15:00.259828+0000","last_became_active":"2024-09-21T03:13:23.273524+0000","last_became_peered":"2024-09-21T03:13:23.273524+0000","last_unstale":"2024-09-21T03:15:00.259828+0000","last_undegraded":"2024-09-21T03:15:00.259828+0000","last_fullsized":"2024-09-21T03:15:00.259828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:09:58.239093+0000","last_clean_scrub_stamp":"2024-09-21T03:09:58.239093+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.887887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264069+0000","last_change":"2024-09-21T03:14:34.601086+0000","last_active":"2024-09-21T03:15:00.264069+0000","last_peered":"2024-09-21T03:15:00.264069+0000","last_clean":"2024-09-21T03:15:00.264069+0000","last_became_active":"2024-09-21T03:14:34.600688+0000","last_became_peered":"2024-09-21T03:14:34.600688+0000","last_unstale":"2024-09-21T03:15:00.264069+0000","last_undegraded":"2024-09-21T03:15:00.264069+0000","last_fullsized":"2024-09-21T03:15:00.264069+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:59.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.441320+0000","last_change":"2024-09-21T03:14:36.611683+0000","last_active":"2024-09-21T03:15:59.441320+0000","last_peered":"2024-09-21T03:15:59.441320+0000","last_clean":"2024-09-21T03:15:59.441320+0000","last_became_active":"2024-09-21T03:14:36.611397+0000","last_became_peered":"2024-09-21T03:14:36.611397+0000","last_unstale":"2024-09-21T03:15:59.441320+0000","last_undegraded":"2024-09-21T03:15:59.441320+0000","last_fullsized":"2024-09-21T03:15:59.441320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.155414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259780+0000","last_change":"2024-09-21T03:14:34.597580+0000","last_active":"2024-09-21T03:15:00.259780+0000","last_peered":"2024-09-21T03:15:00.259780+0000","last_clean":"2024-09-21T03:15:00.259780+0000","last_became_active":"2024-09-21T03:14:34.597162+0000","last_became_peered":"2024-09-21T03:14:34.597162+0000","last_unstale":"2024-09-21T03:15:00.259780+0000","last_undegraded":"2024-09-21T03:15:00.259780+0000","last_fullsized":"2024-09-21T03:15:00.259780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:24:24.171416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446838+0000","last_change":"2024-09-21T03:14:36.623800+0000","last_active":"2024-09-21T03:15:59.446838+0000","last_peered":"2024-09-21T03:15:59.446838+0000","last_clean":"2024-09-21T03:15:59.446838+0000","last_became_active":"2024-09-21T03:14:36.623490+0000","last_became_peered":"2024-09-21T03:14:36.623490+0000","last_unstale":"2024-09-21T03:15:59.446838+0000","last_undegraded":"2024-09-21T03:15:59.446838+0000","last_fullsized":"2024-09-21T03:15:59.446838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:02.072577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:40.937742+0000","last_change":"2024-09-21T03:14:35.075339+0000","last_active":"2024-09-21T03:16:40.937742+0000","last_peered":"2024-09-21T03:16:40.937742+0000","last_clean":"2024-09-21T03:16:40.937742+0000","last_became_active":"2024-09-21T03:14:35.074774+0000","last_became_peered":"2024-09-21T03:14:35.074774+0000","last_unstale":"2024-09-21T03:16:40.937742+0000","last_undegraded":"2024-09-21T03:16:40.937742+0000","last_fullsized":"2024-09-21T03:16:40.937742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:19.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455106+0000","last_change":"2024-09-21T03:14:36.618131+0000","last_active":"2024-09-21T03:15:59.455106+0000","last_peered":"2024-09-21T03:15:59.455106+0000","last_clean":"2024-09-21T03:15:59.455106+0000","last_became_active":"2024-09-21T03:14:36.617762+0000","last_became_peered":"2024-09-21T03:14:36.617762+0000","last_unstale":"2024-09-21T03:15:59.455106+0000","last_undegraded":"2024-09-21T03:15:59.455106+0000","last_fullsized":"2024-09-21T03:15:59.455106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:11.587848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:39.922099+0000","last_change":"2024-09-21T03:14:34.601047+0000","last_active":"2024-09-21T03:16:39.922099+0000","last_peered":"2024-09-21T03:16:39.922099+0000","last_clean":"2024-09-21T03:16:39.922099+0000","last_became_active":"2024-09-21T03:14:34.600676+0000","last_became_peered":"2024-09-21T03:14:34.600676+0000","last_unstale":"2024-09-21T03:16:39.922099+0000","last_undegraded":"2024-09-21T03:16:39.922099+0000","last_fullsized":"2024-09-21T03:16:39.922099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:02.775558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.451490+0000","last_change":"2024-09-21T03:15:00.266362+0000","last_active":"2024-09-21T03:15:59.451490+0000","last_peered":"2024-09-21T03:15:59.451490+0000","last_clean":"2024-09-21T03:15:59.451490+0000","last_became_active":"2024-09-21T03:15:00.265986+0000","last_became_peered":"2024-09-21T03:15:00.265986+0000","last_unstale":"2024-09-21T03:15:59.451490+0000","last_undegraded":"2024-09-21T03:15:59.451490+0000","last_fullsized":"2024-09-21T03:15:59.451490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:40:32.904814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265641+0000","last_change":"2024-09-21T03:15:00.264969+0000","last_active":"2024-09-21T03:15:00.265641+0000","last_peered":"2024-09-21T03:15:00.265641+0000","last_clean":"2024-09-21T03:15:00.265641+0000","last_became_active":"2024-09-21T03:15:00.264484+0000","last_became_peered":"2024-09-21T03:15:00.264484+0000","last_unstale":"2024-09-21T03:15:00.265641+0000","last_undegraded":"2024-09-21T03:15:00.265641+0000","last_fullsized":"2024-09-21T03:15:00.265641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:13.471146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.448466+0000","last_change":"2024-09-21T03:14:36.621887+0000","last_active":"2024-09-21T03:15:59.448466+0000","last_peered":"2024-09-21T03:15:59.448466+0000","last_clean":"2024-09-21T03:15:59.448466+0000","last_became_active":"2024-09-21T03:14:36.621512+0000","last_became_peered":"2024-09-21T03:14:36.621512+0000","last_unstale":"2024-09-21T03:15:59.448466+0000","last_undegraded":"2024-09-21T03:15:59.448466+0000","last_fullsized":"2024-09-21T03:15:59.448466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:09.281539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261679+0000","last_change":"2024-09-21T03:14:35.076302+0000","last_active":"2024-09-21T03:15:00.261679+0000","last_peered":"2024-09-21T03:15:00.261679+0000","last_clean":"2024-09-21T03:15:00.261679+0000","last_became_active":"2024-09-21T03:14:35.074995+0000","last_became_peered":"2024-09-21T03:14:35.074995+0000","last_unstale":"2024-09-21T03:15:00.261679+0000","last_undegraded":"2024-09-21T03:15:00.261679+0000","last_fullsized":"2024-09-21T03:15:00.261679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:09.138029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.446331+0000","last_change":"2024-09-21T03:14:36.624994+0000","last_active":"2024-09-21T03:15:59.446331+0000","last_peered":"2024-09-21T03:15:59.446331+0000","last_clean":"2024-09-21T03:15:59.446331+0000","last_became_active":"2024-09-21T03:14:36.624593+0000","last_became_peered":"2024-09-21T03:14:36.624593+0000","last_unstale":"2024-09-21T03:15:59.446331+0000","last_undegraded":"2024-09-21T03:15:59.446331+0000","last_fullsized":"2024-09-21T03:15:59.446331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:08.746678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.622195+0000","last_change":"2024-09-21T03:14:34.593470+0000","last_active":"2024-09-21T03:15:59.622195+0000","last_peered":"2024-09-21T03:15:59.622195+0000","last_clean":"2024-09-21T03:15:59.622195+0000","last_became_active":"2024-09-21T03:14:34.593101+0000","last_became_peered":"2024-09-21T03:14:34.593101+0000","last_unstale":"2024-09-21T03:15:59.622195+0000","last_undegraded":"2024-09-21T03:15:59.622195+0000","last_fullsized":"2024-09-21T03:15:59.622195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:56.264437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.449080+0000","last_change":"2024-09-21T03:14:36.622502+0000","last_active":"2024-09-21T03:15:59.449080+0000","last_peered":"2024-09-21T03:15:59.449080+0000","last_clean":"2024-09-21T03:15:59.449080+0000","last_became_active":"2024-09-21T03:14:36.622182+0000","last_became_peered":"2024-09-21T03:14:36.622182+0000","last_unstale":"2024-09-21T03:15:59.449080+0000","last_undegraded":"2024-09-21T03:15:59.449080+0000","last_fullsized":"2024-09-21T03:15:59.449080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:27.755761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.263227+0000","last_change":"2024-09-21T03:14:35.550182+0000","last_active":"2024-09-21T03:15:00.263227+0000","last_peered":"2024-09-21T03:15:00.263227+0000","last_clean":"2024-09-21T03:15:00.263227+0000","last_became_active":"2024-09-21T03:14:35.549791+0000","last_became_peered":"2024-09-21T03:14:35.549791+0000","last_unstale":"2024-09-21T03:15:00.263227+0000","last_undegraded":"2024-09-21T03:15:00.263227+0000","last_fullsized":"2024-09-21T03:15:00.263227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.254771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.457757+0000","last_change":"2024-09-21T03:14:36.616189+0000","last_active":"2024-09-21T03:15:59.457757+0000","last_peered":"2024-09-21T03:15:59.457757+0000","last_clean":"2024-09-21T03:15:59.457757+0000","last_became_active":"2024-09-21T03:14:36.615844+0000","last_became_peered":"2024-09-21T03:14:36.615844+0000","last_unstale":"2024-09-21T03:15:59.457757+0000","last_undegraded":"2024-09-21T03:15:59.457757+0000","last_fullsized":"2024-09-21T03:15:59.457757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:29.397119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.264980+0000","last_change":"2024-09-21T03:14:35.075833+0000","last_active":"2024-09-21T03:15:00.264980+0000","last_peered":"2024-09-21T03:15:00.264980+0000","last_clean":"2024-09-21T03:15:00.264980+0000","last_became_active":"2024-09-21T03:14:35.075446+0000","last_became_peered":"2024-09-21T03:14:35.075446+0000","last_unstale":"2024-09-21T03:15:00.264980+0000","last_undegraded":"2024-09-21T03:15:00.264980+0000","last_fullsized":"2024-09-21T03:15:00.264980+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:04.432684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.408350+0000","last_change":"2024-09-21T03:14:36.624039+0000","last_active":"2024-09-21T03:15:59.408350+0000","last_peered":"2024-09-21T03:15:59.408350+0000","last_clean":"2024-09-21T03:15:59.408350+0000","last_became_active":"2024-09-21T03:14:36.623583+0000","last_became_peered":"2024-09-21T03:14:36.623583+0000","last_unstale":"2024-09-21T03:15:59.408350+0000","last_undegraded":"2024-09-21T03:15:59.408350+0000","last_fullsized":"2024-09-21T03:15:59.408350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:12.025580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.258562+0000","last_change":"2024-09-21T03:14:34.591973+0000","last_active":"2024-09-21T03:15:00.258562+0000","last_peered":"2024-09-21T03:15:00.258562+0000","last_clean":"2024-09-21T03:15:00.258562+0000","last_became_active":"2024-09-21T03:14:34.591629+0000","last_became_peered":"2024-09-21T03:14:34.591629+0000","last_unstale":"2024-09-21T03:15:00.258562+0000","last_undegraded":"2024-09-21T03:15:00.258562+0000","last_fullsized":"2024-09-21T03:15:00.258562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:55.146766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.450927+0000","last_change":"2024-09-21T03:14:36.621458+0000","last_active":"2024-09-21T03:15:59.450927+0000","last_peered":"2024-09-21T03:15:59.450927+0000","last_clean":"2024-09-21T03:15:59.450927+0000","last_became_active":"2024-09-21T03:14:36.621099+0000","last_became_peered":"2024-09-21T03:14:36.621099+0000","last_unstale":"2024-09-21T03:15:59.450927+0000","last_undegraded":"2024-09-21T03:15:59.450927+0000","last_fullsized":"2024-09-21T03:15:59.450927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:13.703423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.265986+0000","last_change":"2024-09-21T03:14:34.593810+0000","last_active":"2024-09-21T03:15:00.265986+0000","last_peered":"2024-09-21T03:15:00.265986+0000","last_clean":"2024-09-21T03:15:00.265986+0000","last_became_active":"2024-09-21T03:14:34.593380+0000","last_became_peered":"2024-09-21T03:14:34.593380+0000","last_unstale":"2024-09-21T03:15:00.265986+0000","last_undegraded":"2024-09-21T03:15:00.265986+0000","last_fullsized":"2024-09-21T03:15:00.265986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:11.727862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.445834+0000","last_change":"2024-09-21T03:14:36.626053+0000","last_active":"2024-09-21T03:15:59.445834+0000","last_peered":"2024-09-21T03:15:59.445834+0000","last_clean":"2024-09-21T03:15:59.445834+0000","last_became_active":"2024-09-21T03:14:36.625453+0000","last_became_peered":"2024-09-21T03:14:36.625453+0000","last_unstale":"2024-09-21T03:15:59.445834+0000","last_undegraded":"2024-09-21T03:15:59.445834+0000","last_fullsized":"2024-09-21T03:15:59.445834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:48.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260681+0000","last_change":"2024-09-21T03:14:34.597569+0000","last_active":"2024-09-21T03:15:00.260681+0000","last_peered":"2024-09-21T03:15:00.260681+0000","last_clean":"2024-09-21T03:15:00.260681+0000","last_became_active":"2024-09-21T03:14:34.597118+0000","last_became_peered":"2024-09-21T03:14:34.597118+0000","last_unstale":"2024-09-21T03:15:00.260681+0000","last_undegraded":"2024-09-21T03:15:00.260681+0000","last_fullsized":"2024-09-21T03:15:00.260681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:38.574885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.458762+0000","last_change":"2024-09-21T03:14:36.609747+0000","last_active":"2024-09-21T03:15:59.458762+0000","last_peered":"2024-09-21T03:15:59.458762+0000","last_clean":"2024-09-21T03:15:59.458762+0000","last_became_active":"2024-09-21T03:14:36.608943+0000","last_became_peered":"2024-09-21T03:14:36.608943+0000","last_unstale":"2024-09-21T03:15:59.458762+0000","last_undegraded":"2024-09-21T03:15:59.458762+0000","last_fullsized":"2024-09-21T03:15:59.458762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:37.197734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.455714+0000","last_change":"2024-09-21T03:15:00.267709+0000","last_active":"2024-09-21T03:15:59.455714+0000","last_peered":"2024-09-21T03:15:59.455714+0000","last_clean":"2024-09-21T03:15:59.455714+0000","last_became_active":"2024-09-21T03:15:00.267239+0000","last_became_peered":"2024-09-21T03:15:00.267239+0000","last_unstale":"2024-09-21T03:15:59.455714+0000","last_undegraded":"2024-09-21T03:15:59.455714+0000","last_fullsized":"2024-09-21T03:15:59.455714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:57.734609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.260788+0000","last_change":"2024-09-21T03:14:34.598594+0000","last_active":"2024-09-21T03:15:00.260788+0000","last_peered":"2024-09-21T03:15:00.260788+0000","last_clean":"2024-09-21T03:15:00.260788+0000","last_became_active":"2024-09-21T03:14:34.598170+0000","last_became_peered":"2024-09-21T03:14:34.598170+0000","last_unstale":"2024-09-21T03:15:00.260788+0000","last_undegraded":"2024-09-21T03:15:00.260788+0000","last_fullsized":"2024-09-21T03:15:00.260788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:59.904104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.461974+0000","last_change":"2024-09-21T03:14:36.626283+0000","last_active":"2024-09-21T03:15:59.461974+0000","last_peered":"2024-09-21T03:15:59.461974+0000","last_clean":"2024-09-21T03:15:59.461974+0000","last_became_active":"2024-09-21T03:14:36.625930+0000","last_became_peered":"2024-09-21T03:14:36.625930+0000","last_unstale":"2024-09-21T03:15:59.461974+0000","last_undegraded":"2024-09-21T03:15:59.461974+0000","last_fullsized":"2024-09-21T03:15:59.461974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:48.285888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.261480+0000","last_change":"2024-09-21T03:14:34.600454+0000","last_active":"2024-09-21T03:15:00.261480+0000","last_peered":"2024-09-21T03:15:00.261480+0000","last_clean":"2024-09-21T03:15:00.261480+0000","last_became_active":"2024-09-21T03:14:34.600097+0000","last_became_peered":"2024-09-21T03:14:34.600097+0000","last_unstale":"2024-09-21T03:15:00.261480+0000","last_undegraded":"2024-09-21T03:15:00.261480+0000","last_fullsized":"2024-09-21T03:15:00.261480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:51.837284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.462659+0000","last_change":"2024-09-21T03:14:36.613181+0000","last_active":"2024-09-21T03:15:59.462659+0000","last_peered":"2024-09-21T03:15:59.462659+0000","last_clean":"2024-09-21T03:15:59.462659+0000","last_became_active":"2024-09-21T03:14:36.612851+0000","last_became_peered":"2024-09-21T03:14:36.612851+0000","last_unstale":"2024-09-21T03:15:59.462659+0000","last_undegraded":"2024-09-21T03:15:59.462659+0000","last_fullsized":"2024-09-21T03:15:59.462659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:40.546534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:00.259071+0000","last_change":"2024-09-21T03:14:35.075398+0000","last_active":"2024-09-21T03:15:00.259071+0000","last_peered":"2024-09-21T03:15:00.259071+0000","last_clean":"2024-09-21T03:15:00.259071+0000","last_became_active":"2024-09-21T03:14:35.074921+0000","last_became_peered":"2024-09-21T03:14:35.074921+0000","last_unstale":"2024-09-21T03:15:00.259071+0000","last_undegraded":"2024-09-21T03:15:00.259071+0000","last_fullsized":"2024-09-21T03:15:00.259071+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:05.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:15:59.452714+0000","last_change":"2024-09-21T03:14:36.619411+0000","last_active":"2024-09-21T03:15:59.452714+0000","last_peered":"2024-09-21T03:15:59.452714+0000","last_clean":"2024-09-21T03:15:59.452714+0000","last_became_active":"2024-09-21T03:14:36.619109+0000","last_became_peered":"2024-09-21T03:14:36.619109+0000","last_unstale":"2024-09-21T03:15:59.452714+0000","last_undegraded":"2024-09-21T03:15:59.452714+0000","last_fullsized":"2024-09-21T03:15:59.452714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:35.582522+0000","last_clean_scrub_stamp":"2024-09-21T03:14:35.582522+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:51.048876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":152,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T03:16:41.924547+0000","last_change":"2024-09-21T03:14:34.599135+0000","last_active":"2024-09-21T03:16:41.924547+0000","last_peered":"2024-09-21T03:16:41.924547+0000","last_clean":"2024-09-21T03:16:41.924547+0000","last_became_active":"2024-09-21T03:14:34.598851+0000","last_became_peered":"2024-09-21T03:14:34.598851+0000","last_unstale":"2024-09-21T03:16:41.924547+0000","last_undegraded":"2024-09-21T03:16:41.924547+0000","last_fullsized":"2024-09-21T03:16:41.924547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:14:33.568665+0000","last_clean_scrub_stamp":"2024-09-21T03:14:33.568665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:59.757402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72576,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1179,"num_read_kb":670,"num_write":258,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282060,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":417205,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939692,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1072640,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":416344,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":416605,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1008732,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":415944,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1008410,"data_compressed":41622,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415805,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":415652,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":483163,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":482171,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":418123,"data_compressed":5757,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34245,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35865,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:16:44.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:16:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:44 smithi193 ceph-mon[35224]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:44 smithi193 ceph-mon[35224]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:44.577 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:44 smithi142 ceph-mon[34512]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:44.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:44 smithi142 ceph-mon[34512]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:44 smithi083 ceph-mon[31219]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:44 smithi083 ceph-mon[31219]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:45.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 -- 172.21.15.83:0/1887737669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0110140 msgr2=0x7f77b0112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 --2- 172.21.15.83:0/1887737669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0110140 0x7f77b0112560 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009ca0 tx=0x7f77a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 -- 172.21.15.83:0/1887737669 shutdown_connections 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 --2- 172.21.15.83:0/1887737669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0110140 0x7f77b0112560 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 --2- 172.21.15.83:0/1887737669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0076080 0x7f77b010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 --2- 172.21.15.83:0/1887737669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b0075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 -- 172.21.15.83:0/1887737669 >> 172.21.15.83:0/1887737669 conn(0x7f77b007b850 msgr2=0x7f77b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 -- 172.21.15.83:0/1887737669 shutdown_connections 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 -- 172.21.15.83:0/1887737669 wait complete. 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.153+0000 7f77b60fb640 1 Processor -- start 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 -- start start 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0110140 0x7f77b01a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0115ca0 con 0x7f77b0076080 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0115b20 con 0x7f77b0075730 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77b60fb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0115e20 con 0x7f77b0110140 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60538/0 (socket says 172.21.15.83:60538) 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 -- 172.21.15.83:0/200221384 learned_addr learned my addr 172.21.15.83:0/200221384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77af7fe640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 -- 172.21.15.83:0/200221384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0110140 msgr2=0x7f77b01a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0110140 0x7f77b01a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 -- 172.21.15.83:0/200221384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 msgr2=0x7f77b019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77aeffd640 1 -- 172.21.15.83:0/200221384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b01a2500 con 0x7f77b0076080 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.154+0000 7f77af7fe640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:45.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77aeffd640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77a400d750 tx=0x7f77a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4017070 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b01a27f0 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a40054d0 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004c30 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.155+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b01a9010 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.156+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b0076c40 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.157+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f77a4004dd0 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.158+0000 7f77acff9640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 0x7f77980b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.158+0000 7f77af7fe640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 0x7f77980b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.158+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f77a40e7270 con 0x7f77b0076080 2024-09-21T03:16:45.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.158+0000 7f77af7fe640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 0x7f77980b8aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77a000baa0 tx=0x7f77a0005d90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:45.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.161+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77a40ada60 con 0x7f77b0076080 2024-09-21T03:16:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:45 smithi083 ceph-mon[31219]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:45 smithi083 ceph-mon[31219]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:45.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.378+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b0075b40 con 0x7f77b0076080 2024-09-21T03:16:45.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.378+0000 7f77acff9640 1 -- 172.21.15.83:0/200221384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f77a40b1e40 con 0x7f77b0076080 2024-09-21T03:16:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:45.381 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"btime":"2024-09-21T03:16:40:269796+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:16:40.040340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:16:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-09-21T03:16:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 msgr2=0x7f77980b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 0x7f77980b8aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77a000baa0 tx=0x7f77a0005d90 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 msgr2=0x7f77b019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77a400d750 tx=0x7f77a400dc10 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 shutdown_connections 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f77980b6590 0x7f77980b8aa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0110140 0x7f77b01a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b0076080 0x7f77b019cbf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 --2- 172.21.15.83:0/200221384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77b0075730 0x7f77b019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 >> 172.21.15.83:0/200221384 conn(0x7f77b007b850 msgr2=0x7f77b0109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 shutdown_connections 2024-09-21T03:16:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:45.381+0000 7f77b60fb640 1 -- 172.21.15.83:0/200221384 wait complete. 2024-09-21T03:16:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:45 smithi193 ceph-mon[35224]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:45 smithi193 ceph-mon[35224]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:45 smithi142 ceph-mon[34512]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:16:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:45 smithi142 ceph-mon[34512]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:46.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:16:46.371 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:46.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:46 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/200221384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:46 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/200221384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:46 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/200221384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 -- 172.21.15.83:0/1853960386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 msgr2=0x7f3fe01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/1853960386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe01087d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402a410 comp rx=0 tx=0).stop 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 -- 172.21.15.83:0/1853960386 shutdown_connections 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/1853960386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 0x7f3fe010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/1853960386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe01087d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/1853960386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 0x7f3fe0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 -- 172.21.15.83:0/1853960386 >> 172.21.15.83:0/1853960386 conn(0x7f3fe0079740 msgr2=0x7f3fe0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 -- 172.21.15.83:0/1853960386 shutdown_connections 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.869+0000 7f3fe5e98640 1 -- 172.21.15.83:0/1853960386 wait complete. 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 Processor -- start 2024-09-21T03:16:46.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 -- start start 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 0x7f3fe019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 0x7f3fe019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113aa0 con 0x7f3fe01083c0 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113920 con 0x7f3fe0108d10 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fe5e98640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113c20 con 0x7f3fe0102a80 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.870+0000 7f3fdeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60562/0 (socket says 172.21.15.83:60562) 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 -- 172.21.15.83:0/572790233 learned_addr learned my addr 172.21.15.83:0/572790233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdf7fe640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 0x7f3fe019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdffff640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 0x7f3fe019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 -- 172.21.15.83:0/572790233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 msgr2=0x7f3fe019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 0x7f3fe019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 -- 172.21.15.83:0/572790233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 msgr2=0x7f3fe019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 0x7f3fe019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 -- 172.21.15.83:0/572790233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a0480 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdeffd640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4032fd0 tx=0x7f3fd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd403f030 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd4004490 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a0710 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4035070 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.871+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a0bf0 con 0x7f3fe01083c0 2024-09-21T03:16:46.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.873+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe0103d00 con 0x7f3fe01083c0 2024-09-21T03:16:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.874+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f3fd4042030 con 0x7f3fe01083c0 2024-09-21T03:16:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.875+0000 7f3fdcff9640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.875+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3fd4107d00 con 0x7f3fe01083c0 2024-09-21T03:16:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.875+0000 7f3fdf7fe640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.876+0000 7f3fdf7fe640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000abe0 tx=0x7f3fd000a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:46.878+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fd40ce4f0 con 0x7f3fe01083c0 2024-09-21T03:16:47.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.058+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fe010ab00 con 0x7f3fe01083c0 2024-09-21T03:16:47.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.059+0000 7f3fdcff9640 1 -- 172.21.15.83:0/572790233 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22947 (secure 0 0 0) 0x7f3fd40d28d0 con 0x7f3fe01083c0 2024-09-21T03:16:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:47.062 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:15:00.250430+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.061+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 msgr2=0x7f3fc80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.061+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000abe0 tx=0x7f3fd000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.061+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 msgr2=0x7f3fe019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.061+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4032fd0 tx=0x7f3fd4004270 comp rx=0 tx=0).stop 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.061+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 shutdown_connections 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fe0108d10 0x7f3fe019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fe01083c0 0x7f3fe019ab70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 --2- 172.21.15.83:0/572790233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe0102a80 0x7f3fe019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 >> 172.21.15.83:0/572790233 conn(0x7f3fe0079740 msgr2=0x7f3fe0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:47.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 shutdown_connections 2024-09-21T03:16:47.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.062+0000 7f3fe5e98640 1 -- 172.21.15.83:0/572790233 wait complete. 2024-09-21T03:16:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:47 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/572790233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:47 smithi083 ceph-mon[31219]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:47 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/572790233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:47 smithi193 ceph-mon[35224]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:47 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/572790233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:16:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:47 smithi142 ceph-mon[34512]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T03:16:47.692 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T03:16:47.692 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:16:47.702 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:16:47.702 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:16:47.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T03:16:47.702 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T03:16:47.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:16:47.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 -- 172.21.15.83:0/3257162113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01082e0 msgr2=0x7fcee01086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3257162113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01082e0 0x7fcee01086f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009900 tx=0x7fcedc02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 -- 172.21.15.83:0/3257162113 shutdown_connections 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3257162113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3257162113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01082e0 0x7fcee01086f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3257162113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01029b0 0x7fcee0102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 -- 172.21.15.83:0/3257162113 >> 172.21.15.83:0/3257162113 conn(0x7fcee007a740 msgr2=0x7fcee007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:47.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 -- 172.21.15.83:0/3257162113 shutdown_connections 2024-09-21T03:16:47.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.813+0000 7fcee7b13640 1 -- 172.21.15.83:0/3257162113 wait complete. 2024-09-21T03:16:47.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 Processor -- start 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 -- start start 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 0x7fcee019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01128f0 con 0x7fcee01029b0 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0112770 con 0x7fcee01082e0 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee7b13640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0112a70 con 0x7fcee0108c30 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee58af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee58af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60572/0 (socket says 172.21.15.83:60572) 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.814+0000 7fcee58af640 1 -- 172.21.15.83:0/3970225786 learned_addr learned my addr 172.21.15.83:0/3970225786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee50ae640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 0x7fcee019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee60b0640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 -- 172.21.15.83:0/3970225786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 msgr2=0x7fcee019bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee019bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 -- 172.21.15.83:0/3970225786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 msgr2=0x7fcee019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 0x7fcee019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 -- 172.21.15.83:0/3970225786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee01a6530 con 0x7fcee01029b0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee60b0640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee019bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee50ae640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 0x7fcee019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee58af640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fced400e9f0 tx=0x7fced400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced400cd80 con 0x7fcee01029b0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced40108e0 con 0x7fcee01029b0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee01a6820 con 0x7fcee01029b0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4018980 con 0x7fcee01029b0 2024-09-21T03:16:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.815+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01a6d40 con 0x7fcee01029b0 2024-09-21T03:16:47.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.816+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee0102dc0 con 0x7fcee01029b0 2024-09-21T03:16:47.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.819+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fced4010430 con 0x7fcee01029b0 2024-09-21T03:16:47.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.819+0000 7fced2ffd640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 0x7fcecc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:47.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.820+0000 7fcee50ae640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 0x7fcecc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:47.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.820+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fced40e7d40 con 0x7fcee01029b0 2024-09-21T03:16:47.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.820+0000 7fcee50ae640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 0x7fcecc0b8a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009230 tx=0x7fcedc000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:47.823+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fced40ae530 con 0x7fcee01029b0 2024-09-21T03:16:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:48.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.035+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fcee0063800 con 0x7fcecc0b6560 2024-09-21T03:16:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.199+0000 7fced2ffd640 1 -- 172.21.15.83:0/3970225786 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcee0063800 con 0x7fcecc0b6560 2024-09-21T03:16:48.202 INFO:teuthology.orchestra.run.smithi083.stdout:Schedule set for path / 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 msgr2=0x7fcecc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 0x7fcecc0b8a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009230 tx=0x7fcedc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 msgr2=0x7fcee019b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fced400e9f0 tx=0x7fced400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 shutdown_connections 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcecc0b6560 0x7fcecc0b8a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcee0108c30 0x7fcee019bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee01082e0 0x7fcee019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 --2- 172.21.15.83:0/3970225786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee01029b0 0x7fcee019b220 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 >> 172.21.15.83:0/3970225786 conn(0x7fcee007a740 msgr2=0x7fcee0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 shutdown_connections 2024-09-21T03:16:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.202+0000 7fcee7b13640 1 -- 172.21.15.83:0/3970225786 wait complete. 2024-09-21T03:16:48.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T03:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 -- 172.21.15.83:0/2151291585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 msgr2=0x7f03b4102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 --2- 172.21.15.83:0/2151291585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4102e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009900 tx=0x7f03a802a410 comp rx=0 tx=0).stop 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 -- 172.21.15.83:0/2151291585 shutdown_connections 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 --2- 172.21.15.83:0/2151291585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 0x7f03b410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 --2- 172.21.15.83:0/2151291585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 0x7f03b4108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 --2- 172.21.15.83:0/2151291585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4102e20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 -- 172.21.15.83:0/2151291585 >> 172.21.15.83:0/2151291585 conn(0x7f03b40fe410 msgr2=0x7f03b4100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 -- 172.21.15.83:0/2151291585 shutdown_connections 2024-09-21T03:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 -- 172.21.15.83:0/2151291585 wait complete. 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.359+0000 7f03b9790640 1 Processor -- start 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 -- start start 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 0x7f03b4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 0x7f03b419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4112950 con 0x7f03b4102a10 2024-09-21T03:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41127d0 con 0x7f03b4108340 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b9790640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4112ad0 con 0x7f03b4108c90 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60600/0 (socket says 172.21.15.83:60600) 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 -- 172.21.15.83:0/1901186351 learned_addr learned my addr 172.21.15.83:0/1901186351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b27fc640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 0x7f03b4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b37fe640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 0x7f03b419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 -- 172.21.15.83:0/1901186351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 msgr2=0x7f03b419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 0x7f03b419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 -- 172.21.15.83:0/1901186351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 msgr2=0x7f03b4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 0x7f03b4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.360+0000 7f03b2ffd640 1 -- 172.21.15.83:0/1901186351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b419f2c0 con 0x7f03b4102a10 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f03b2ffd640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005940 tx=0x7f03a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a803f030 con 0x7f03b4102a10 2024-09-21T03:16:48.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a80044c0 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b419f550 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8035070 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.361+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a5d40 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.362+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b4102e20 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.364+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f03a8042030 con 0x7f03b4102a10 2024-09-21T03:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.364+0000 7f039bfff640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 0x7f039c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.364+0000 7f03b27fc640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 0x7f039c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.364+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f03a8107b30 con 0x7f03b4102a10 2024-09-21T03:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.365+0000 7f03b27fc640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 0x7f039c0b8a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03b4101cf0 tx=0x7f03a4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.368+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03a80ce320 con 0x7f03b4102a10 2024-09-21T03:16:48.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:48 smithi083 ceph-mon[31219]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 -- 172.21.15.83:0/3237044380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe068108d10 msgr2=0x7fe06810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 --2- 172.21.15.83:0/3237044380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe068108d10 0x7fe06810f540 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe060009ca0 tx=0x7fe06002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 -- 172.21.15.83:0/3237044380 shutdown_connections 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 --2- 172.21.15.83:0/3237044380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe068108d10 0x7fe06810f540 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 --2- 172.21.15.83:0/3237044380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0681083c0 0x7fe0681087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 --2- 172.21.15.83:0/3237044380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 -- 172.21.15.83:0/3237044380 >> 172.21.15.83:0/3237044380 conn(0x7fe068079740 msgr2=0x7fe068079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 -- 172.21.15.83:0/3237044380 shutdown_connections 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.470+0000 7fe06fefe640 1 -- 172.21.15.83:0/3237044380 wait complete. 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 Processor -- start 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 -- start start 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe068108d10 0x7fe068199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe068112980 con 0x7fe0681083c0 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe068112800 con 0x7fe068108d10 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06fefe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe068112b00 con 0x7fe068102a80 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06d499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.471+0000 7fe06d499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60614/0 (socket says 172.21.15.83:60614) 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 -- 172.21.15.83:0/1331088575 learned_addr learned my addr 172.21.15.83:0/1331088575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06e49b640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe068108d10 0x7fe068199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 -- 172.21.15.83:0/1331088575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 msgr2=0x7fe068198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 -- 172.21.15.83:0/1331088575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe068108d10 msgr2=0x7fe068199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe068108d10 0x7fe068199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 -- 172.21.15.83:0/1331088575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06819e260 con 0x7fe0681083c0 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06d499640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe064005fd0 tx=0x7fe06400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe05effd640 1 -- 172.21.15.83:0/1331088575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064017070 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06819e550 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe05effd640 1 -- 172.21.15.83:0/1331088575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe064009c30 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe05effd640 1 -- 172.21.15.83:0/1331088575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064003d30 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.472+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06819eae0 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.475+0000 7fe05effd640 1 -- 172.21.15.83:0/1331088575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fe064003ed0 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe05effd640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 0x7fe0540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 0x7fe0540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe05effd640 1 -- 172.21.15.83:0/1331088575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe0640e6fd0 con 0x7fe0681083c0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe076472740 1 Processor -- start 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe076472740 1 -- start start 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.476+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 0x7fe0540b8a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe06806c3e0 tx=0x7fe05800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594bd928e60 con 0x5594bd5ea670 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594bd918290 con 0x5594bd5e2490 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe076472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594bd91c0a0 con 0x5594bd60b350 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60630/0 (socket says 172.21.15.83:60630) 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 -- 172.21.15.83:0/3942653992 learned_addr learned my addr 172.21.15.83:0/3942653992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06d499640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 -- 172.21.15.83:0/3942653992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 msgr2=0x5594bd9b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 -- 172.21.15.83:0/3942653992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 msgr2=0x5594bd9b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 -- 172.21.15.83:0/3942653992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594bd676930 con 0x5594bd5ea670 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe06e49b640 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe060035000 tx=0x7fe0600094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bfeafbf4b99a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.477+0000 7fe053fff640 1 -- 172.21.15.83:0/3942653992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060031db0 con 0x5594bd5ea670 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe053fff640 1 -- 172.21.15.83:0/3942653992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060036030 con 0x5594bd5ea670 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe053fff640 1 -- 172.21.15.83:0/3942653992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600042a0 con 0x5594bd5ea670 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 -- 172.21.15.83:0/3942653992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 msgr2=0x5594bd9b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe060035000 tx=0x7fe0600094a0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 -- 172.21.15.83:0/3942653992 shutdown_connections 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5ea670 0x5594bd9b2eb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 --2- 172.21.15.83:0/3942653992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.478+0000 7fe076472740 1 -- 172.21.15.83:0/3942653992 >> 172.21.15.83:0/3942653992 conn(0x5594bd5ff740 msgr2=0x5594bd9b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 -- 172.21.15.83:0/3942653992 shutdown_connections 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 -- 172.21.15.83:0/3942653992 wait complete. 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 -- start start 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 Processor -- start 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.479+0000 7fe076472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bd9c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe076472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594bd779890 con 0x5594bd60b350 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe076472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594bd8b2d40 con 0x5594bd5ea670 2024-09-21T03:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe076472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594bd9c0a30 con 0x5594bd5e2490 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe076472740 10 client.0 _ll_register_callbacks cb 0x5594bd916c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bd9c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06dc9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06dc9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:48114/0 (socket says 172.21.15.83:48114) 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06dc9a640 1 -- 172.21.15.83:0/1373222820 learned_addr learned my addr 172.21.15.83:0/1373222820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06d499640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 -- 172.21.15.83:0/1373222820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 msgr2=0x5594bd9be010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 -- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 msgr2=0x5594bd9be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 -- 172.21.15.83:0/1373222820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594bd9ca310 con 0x5594bd60b350 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe06e49b640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bd9c9d20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe060005e30 tx=0x7fe0600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d85d637054e70d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe0517fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06004a040 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe060031880 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.480+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe060036030 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe060003970 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 10 client.15564 fetch_fsmap learned FSMap version 27 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594bd9ccc40 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060036030 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe06004d030 con 0x5594bd60b350 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 20 client.15564 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd5c9cd0 0x5594bd9cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 4 client.15564 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594bd92adb0 con 0x5594bd5c9cd0 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.481+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd5c9cd0 0x5594bd9cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.482+0000 7fe0517fa640 10 client.15564 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.482+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd5c9cd0 0x5594bd9cffa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39be75dd27912a9 server_cookie=50c1ee5c15095407 in_seq=0 out_seq=0 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe0517fa640 1 -- 172.21.15.83:0/1373222820 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594bd92adb0 con 0x5594bd5c9cd0 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe0517fa640 10 client.15564 handle_command_reply: tid=0 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 client.15564 shutdown 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 msgr2=0x5594bd9c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bd9c9d20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe060005e30 tx=0x7fe0600095c0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 shutdown_connections 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd5c9cd0 0x5594bd9cffa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bd9c9d20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 --2- 172.21.15.83:0/1373222820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5e2490 0x5594bd9be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe0517fa640 0 client.15564 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 >> 172.21.15.83:0/1373222820 conn(0x5594bd5ff740 msgr2=0x5594bd9b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 shutdown_connections 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 1 -- 172.21.15.83:0/1373222820 wait complete. 2024-09-21T03:16:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.483+0000 7fe076472740 20 client.15564 trim_cache size 0 max 16384 2024-09-21T03:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.500+0000 7fe076472740 1 Processor -- start 2024-09-21T03:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.500+0000 7fe076472740 1 -- start start 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b5a70 con 0x5594bd5e2490 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b75b0 con 0x5594bd5ea670 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe076472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b6140 con 0x5594bd60b350 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60656/0 (socket says 172.21.15.83:60656) 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 -- 172.21.15.83:0/508469233 learned_addr learned my addr 172.21.15.83:0/508469233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06d499640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 -- 172.21.15.83:0/508469233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 msgr2=0x5594bd9bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06e49b640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 -- 172.21.15.83:0/508469233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 msgr2=0x5594bd9b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 -- 172.21.15.83:0/508469233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594bd9be380 con 0x5594bd5e2490 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06d499640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06e49b640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.501+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5594bd918350 tx=0x7fe058013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa3a327e1f6dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.502+0000 7fe0537fe640 1 -- 172.21.15.83:0/508469233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058021db0 con 0x5594bd5e2490 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.502+0000 7fe0537fe640 1 -- 172.21.15.83:0/508469233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580210d0 con 0x5594bd5e2490 2024-09-21T03:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.502+0000 7fe0537fe640 1 -- 172.21.15.83:0/508469233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580038e0 con 0x5594bd5e2490 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- 172.21.15.83:0/508469233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 msgr2=0x5594bd9c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5594bd918350 tx=0x7fe058013000 comp rx=0 tx=0).stop 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- 172.21.15.83:0/508469233 shutdown_connections 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd60b350 0x5594bd9bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5ea670 0x5594bd9b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 --2- 172.21.15.83:0/508469233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd5e2490 0x5594bd9c9c90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- 172.21.15.83:0/508469233 >> 172.21.15.83:0/508469233 conn(0x5594bd5ff740 msgr2=0x5594bd9bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- 172.21.15.83:0/508469233 shutdown_connections 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- 172.21.15.83:0/508469233 wait complete. 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 -- start start 2024-09-21T03:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.503+0000 7fe076472740 1 Processor -- start 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5ea670 0x5594bd9d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b6ee0 con 0x5594bd60b350 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b6810 con 0x5594bd5e2490 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b3a50 con 0x5594bd5ea670 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 10 client.0 _ll_register_callbacks cb 0x7fe060009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60664/0 (socket says 172.21.15.83:60664) 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 -- 172.21.15.83:0/152843288 learned_addr learned my addr 172.21.15.83:0/152843288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06d499640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5ea670 0x5594bd9d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 -- 172.21.15.83:0/152843288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5ea670 msgr2=0x5594bd9d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5ea670 0x5594bd9d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 -- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 msgr2=0x5594bd9d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 -- 172.21.15.83:0/152843288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594bda39c60 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe06e49b640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe060009ca0 tx=0x7fe060030ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802be38d8248ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe0517fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060031660 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe060031800 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.504+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580210d0 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe06002f660 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 10 client.15570 fetch_fsmap learned FSMap version 27 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594bda3ad90 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580210d0 con 0x5594bd60b350 2024-09-21T03:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe06004d040 con 0x5594bd60b350 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 10 client.15570 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 20 client.15570 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd59f080 0x5594bda3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 4 client.15570 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.505+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5594bd926860 con 0x5594bd59f080 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.506+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd59f080 0x5594bda3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.506+0000 7fe0517fa640 10 client.15570 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.506+0000 7fe06dc9a640 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd59f080 0x5594bda3d580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa885eac3415c6d server_cookie=a2c18ac015f50fa3 in_seq=0 out_seq=0 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe0517fa640 1 -- 172.21.15.83:0/152843288 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5594bd926860 con 0x5594bd59f080 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe0517fa640 10 client.15570 handle_command_reply: tid=0 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 client.15570 shutdown 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 msgr2=0x5594bda39720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe060009ca0 tx=0x7fe060030ea0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 shutdown_connections 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594bd59f080 0x5594bda3d580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bd60b350 0x5594bda39720 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594bd5ea670 0x5594bd9d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 --2- 172.21.15.83:0/152843288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594bd5e2490 0x5594bd9d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe0517fa640 0 client.15570 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 >> 172.21.15.83:0/152843288 conn(0x5594bd5ff740 msgr2=0x5594bd9b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 shutdown_connections 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 1 -- 172.21.15.83:0/152843288 wait complete. 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.508+0000 7fe076472740 20 client.15570 trim_cache size 0 max 16384 2024-09-21T03:16:48.514 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1d819a88-2c9a-4459-a4f4-ef203dd15338", 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 msgr2=0x7fe0540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 0x7fe0540b8a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe06806c3e0 tx=0x7fe05800b390 comp rx=0 tx=0).stop 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 msgr2=0x7fe0681994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe064005fd0 tx=0x7fe06400dde0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 shutdown_connections 2024-09-21T03:16:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe0540b6560 0x7fe0540b8a70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe068108d10 0x7fe068199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0681083c0 0x7fe0681994c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 --2- 172.21.15.83:0/1331088575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe068102a80 0x7fe068198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 >> 172.21.15.83:0/1331088575 conn(0x7fe068079740 msgr2=0x7fe068104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 shutdown_connections 2024-09-21T03:16:48.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.509+0000 7fe06fefe640 1 -- 172.21.15.83:0/1331088575 wait complete. 2024-09-21T03:16:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:48 smithi193 ceph-mon[35224]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:48.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.586+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f03b4063800 con 0x7f039c0b6560 2024-09-21T03:16:48.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.654+0000 7f039bfff640 1 -- 172.21.15.83:0/1901186351 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f03b4063800 con 0x7f039c0b6560 2024-09-21T03:16:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:Retention added to path / 2024-09-21T03:16:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.656+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 msgr2=0x7f039c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.656+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 0x7f039c0b8a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03b4101cf0 tx=0x7f03a4005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.656+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 msgr2=0x7f03b4199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.656+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005940 tx=0x7f03a80040c0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 shutdown_connections 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f039c0b6560 0x7f039c0b8a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b4108c90 0x7f03b419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03b4108340 0x7f03b4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 --2- 172.21.15.83:0/1901186351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03b4102a10 0x7f03b4199390 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 >> 172.21.15.83:0/1901186351 conn(0x7f03b40fe410 msgr2=0x7f03b4107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 shutdown_connections 2024-09-21T03:16:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.657+0000 7f03b9790640 1 -- 172.21.15.83:0/1901186351 wait complete. 2024-09-21T03:16:48.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:16:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:48 smithi142 ceph-mon[34512]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:48.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 -- 172.21.15.83:0/2843242206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0109d60 msgr2=0x7fd6e0110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 --2- 172.21.15.83:0/2843242206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0109d60 0x7fd6e0110590 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009ca0 tx=0x7fd6d802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 -- 172.21.15.83:0/2843242206 shutdown_connections 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 --2- 172.21.15.83:0/2843242206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0109d60 0x7fd6e0110590 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 --2- 172.21.15.83:0/2843242206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109410 0x7fd6e0109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 --2- 172.21.15.83:0/2843242206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0103ae0 0x7fd6e0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 -- 172.21.15.83:0/2843242206 >> 172.21.15.83:0/2843242206 conn(0x7fd6e00ff520 msgr2=0x7fd6e0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 -- 172.21.15.83:0/2843242206 shutdown_connections 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.781+0000 7fd6e729b640 1 -- 172.21.15.83:0/2843242206 wait complete. 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 Processor -- start 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 -- start start 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0109410 0x7fd6e019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 0x7fd6e019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0113a20 con 0x7fd6e0103ae0 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01138a0 con 0x7fd6e0109d60 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e729b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0113ba0 con 0x7fd6e0109410 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60682/0 (socket says 172.21.15.83:60682) 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 -- 172.21.15.83:0/3456407369 learned_addr learned my addr 172.21.15.83:0/3456407369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5838640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 0x7fd6e019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e4836640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0109410 0x7fd6e019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 -- 172.21.15.83:0/3456407369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0109410 msgr2=0x7fd6e019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0109410 0x7fd6e019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 -- 172.21.15.83:0/3456407369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 msgr2=0x7fd6e019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 0x7fd6e019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5037640 1 -- 172.21.15.83:0/3456407369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01a0370 con 0x7fd6e0103ae0 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.782+0000 7fd6e5838640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 0x7fd6e019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6e5037640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d770 tx=0x7fd6dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc017070 con 0x7fd6e0103ae0 2024-09-21T03:16:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6dc0054d0 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01a0660 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc004c30 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.783+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a6df0 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.784+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e0103ef0 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.786+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fd6dc004dd0 con 0x7fd6e0103ae0 2024-09-21T03:16:48.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.787+0000 7fd6d67fc640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 0x7fd6cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:48.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.787+0000 7fd6e4836640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 0x7fd6cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:48.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.787+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd6dc0e7310 con 0x7fd6e0103ae0 2024-09-21T03:16:48.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.787+0000 7fd6e4836640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 0x7fd6cc0b8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0106d10 tx=0x7fd6d000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.790+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6dc0adb00 con 0x7fd6e0103ae0 2024-09-21T03:16:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.992+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd6e0063800 con 0x7fd6cc0b6560 2024-09-21T03:16:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:48.999+0000 7fd6d67fc640 1 -- 172.21.15.83:0/3456407369 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fd6e0063800 con 0x7fd6cc0b6560 2024-09-21T03:16:49.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:16:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T03:16:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 msgr2=0x7fd6cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 0x7fd6cc0b8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0106d10 tx=0x7fd6d000b390 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 msgr2=0x7fd6e019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d770 tx=0x7fd6dc00dc30 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 shutdown_connections 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd6cc0b6560 0x7fd6cc0b8a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0109d60 0x7fd6e019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e0109410 0x7fd6e019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 --2- 172.21.15.83:0/3456407369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e0103ae0 0x7fd6e019a440 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 >> 172.21.15.83:0/3456407369 conn(0x7fd6e00ff520 msgr2=0x7fd6e0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 shutdown_connections 2024-09-21T03:16:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.002+0000 7fd6e729b640 1 -- 172.21.15.83:0/3456407369 wait complete. 2024-09-21T03:16:49.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/3848617059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f010dd80 msgr2=0x7fd0f0110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/3848617059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f010dd80 0x7fd0f0110260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009ca0 tx=0x7fd0e802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/3848617059 shutdown_connections 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/3848617059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f010dd80 0x7fd0f0110260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/3848617059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 0x7fd0f010d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/3848617059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f0104960 0x7fd0f0104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/3848617059 >> 172.21.15.83:0/3848617059 conn(0x7fd0f007a740 msgr2=0x7fd0f007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/3848617059 shutdown_connections 2024-09-21T03:16:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.128+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/3848617059 wait complete. 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 Processor -- start 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 -- start start 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 0x7fd0f0102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f010dd80 0x7fd0f0103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f01139f0 con 0x7fd0f0104960 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0113870 con 0x7fd0f01052b0 2024-09-21T03:16:49.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f7f4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0113b70 con 0x7fd0f010dd80 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f5ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f5ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60706/0 (socket says 172.21.15.83:60706) 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.129+0000 7fd0f5ce6640 1 -- 172.21.15.83:0/1774151411 learned_addr learned my addr 172.21.15.83:0/1774151411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f64e7640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f010dd80 0x7fd0f0103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 -- 172.21.15.83:0/1774151411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f010dd80 msgr2=0x7fd0f0103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f54e5640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 0x7fd0f0102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f010dd80 0x7fd0f0103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 -- 172.21.15.83:0/1774151411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 msgr2=0x7fd0f0102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 0x7fd0f0102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 -- 172.21.15.83:0/1774151411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f01a86a0 con 0x7fd0f0104960 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f5ce6640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000e9f0 tx=0x7fd0e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000cd80 con 0x7fd0f0104960 2024-09-21T03:16:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f01a8990 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e00054d0 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0010690 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.130+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f01a8e90 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.131+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f0104d70 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.134+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fd0e0004e50 con 0x7fd0f0104960 2024-09-21T03:16:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.134+0000 7fd0e6ffd640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 0x7fd0dc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:49.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.134+0000 7fd0f54e5640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 0x7fd0dc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:49.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.134+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd0e00e7920 con 0x7fd0f0104960 2024-09-21T03:16:49.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.135+0000 7fd0f54e5640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 0x7fd0dc0b8aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd0f010fb80 tx=0x7fd0ec005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:49.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.140+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0e00ae110 con 0x7fd0f0104960 2024-09-21T03:16:49.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:16:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.336+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fd0e00a9c80 con 0x7fd0f0104960 2024-09-21T03:16:49.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.362+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd0f0106eb0 con 0x7fd0dc0b6590 2024-09-21T03:16:49.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.370+0000 7fd0e6ffd640 1 -- 172.21.15.83:0/1774151411 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd0f0106eb0 con 0x7fd0dc0b6590 2024-09-21T03:16:49.372 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 msgr2=0x7fd0dc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 0x7fd0dc0b8aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd0f010fb80 tx=0x7fd0ec005f90 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 msgr2=0x7fd0f0102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000e9f0 tx=0x7fd0e000edf0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 shutdown_connections 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd0dc0b6590 0x7fd0dc0b8aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0f010dd80 0x7fd0f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f01052b0 0x7fd0f0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 --2- 172.21.15.83:0/1774151411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0f0104960 0x7fd0f0102690 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 >> 172.21.15.83:0/1774151411 conn(0x7fd0f007a740 msgr2=0x7fd0f0105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 shutdown_connections 2024-09-21T03:16:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:49.372+0000 7fd0f7f4a640 1 -- 172.21.15.83:0/1774151411 wait complete. 2024-09-21T03:16:49.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T03:16:49.456 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T03:16:49.466 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:16:49.490 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:16:49.490 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:16:49.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-21T03:16:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:16:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:16:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:49 smithi083 ceph-mon[31219]: pgmap v330: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:16:49.534 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:49.551 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-21T03:16:49.551 INFO:teuthology.orchestra.run.smithi083.stdout:0@5108497e-77c6-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/5a50105c-b77d-4a93-a145-d0128cc748da 340G 0 340G 0% /tmp/cephfs 2024-09-21T03:16:49.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-21T03:16:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:16:49.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-21T03:16:49.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:49 smithi142 ceph-mon[34512]: pgmap v330: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:49 smithi193 ceph-mon[35224]: pgmap v330: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 -- 172.21.15.83:0/3263764418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 msgr2=0x7f86901098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 --2- 172.21.15.83:0/3263764418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f86901098c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f867c009900 tx=0x7f867c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 -- 172.21.15.83:0/3263764418 shutdown_connections 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 --2- 172.21.15.83:0/3263764418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690109e00 0x7f8690110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 --2- 172.21.15.83:0/3263764418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f86901098c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 --2- 172.21.15.83:0/3263764418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690103bc0 0x7f8690103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 -- 172.21.15.83:0/3263764418 >> 172.21.15.83:0/3263764418 conn(0x7f869007b910 msgr2=0x7f869007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 -- 172.21.15.83:0/3263764418 shutdown_connections 2024-09-21T03:16:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.123+0000 7f8696d59640 1 -- 172.21.15.83:0/3263764418 wait complete. 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 Processor -- start 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 -- start start 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690103bc0 0x7f869019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690109e00 0x7f869019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8690113ac0 con 0x7f86901094b0 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8690113940 con 0x7f8690103bc0 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.124+0000 7f8696d59640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8690113c40 con 0x7f8690109e00 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48080/0 (socket says 172.21.15.83:48080) 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 -- 172.21.15.83:0/2930797931 learned_addr learned my addr 172.21.15.83:0/2930797931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f86952f6640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690109e00 0x7f869019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f8694af5640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690103bc0 0x7f869019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 -- 172.21.15.83:0/2930797931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690109e00 msgr2=0x7f869019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690109e00 0x7f869019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 -- 172.21.15.83:0/2930797931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690103bc0 msgr2=0x7f869019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690103bc0 0x7f869019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 -- 172.21.15.83:0/2930797931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86901a0340 con 0x7f86901094b0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868ffff640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f867c02a930 tx=0x7f867c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.125+0000 7f868dffb640 1 -- 172.21.15.83:0/2930797931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c03f030 con 0x7f86901094b0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.126+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901a05d0 con 0x7f86901094b0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.126+0000 7f868dffb640 1 -- 172.21.15.83:0/2930797931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c003cb0 con 0x7f86901094b0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.126+0000 7f868dffb640 1 -- 172.21.15.83:0/2930797931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c035070 con 0x7f86901094b0 2024-09-21T03:16:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.126+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a0b00 con 0x7f86901094b0 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.128+0000 7f869d2cd740 1 Processor -- start 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 -- start start 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f868dffb640 1 -- 172.21.15.83:0/2930797931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f867c042030 con 0x7f86901094b0 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd98850e50 con 0x55cd98512390 2024-09-21T03:16:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd98840280 con 0x55cd9850a1b0 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f869d2cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd98844090 con 0x55cd98533070 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f8694af5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f8694af5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:56506/0 (socket says 172.21.15.83:56506) 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f8694af5640 1 -- 172.21.15.83:0/3644644009 learned_addr learned my addr 172.21.15.83:0/3644644009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.129+0000 7f86952f6640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 -- 172.21.15.83:0/3644644009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 msgr2=0x55cd988da630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988da630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f868ffff640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 -- 172.21.15.83:0/3644644009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 msgr2=0x55cd988db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 -- 172.21.15.83:0/3644644009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd9859e650 con 0x55cd9850a1b0 2024-09-21T03:16:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f86952f6640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f868ffff640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f868dffb640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 0x7f86800b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f8694af5640 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f868400ac20 tx=0x7f868400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490771676b300fed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f86952f6640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 0x7f86800b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f868dffb640 1 -- 172.21.15.83:0/2930797931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f867c107b90 con 0x7f86901094b0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f867affd640 1 -- 172.21.15.83:0/3644644009 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401d070 con 0x55cd9850a1b0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.130+0000 7f867affd640 1 -- 172.21.15.83:0/3644644009 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684018af0 con 0x55cd9850a1b0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.131+0000 7f867affd640 1 -- 172.21.15.83:0/3644644009 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684010b30 con 0x55cd9850a1b0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- 172.21.15.83:0/3644644009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 msgr2=0x55cd988dab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f86952f6640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 0x7f86800b8bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8688007f90 tx=0x7f8688007520 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f868400ac20 tx=0x7f868400f790 comp rx=0 tx=0).stop 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- 172.21.15.83:0/3644644009 shutdown_connections 2024-09-21T03:16:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988dab70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 --2- 172.21.15.83:0/3644644009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- 172.21.15.83:0/3644644009 >> 172.21.15.83:0/3644644009 conn(0x55cd98527460 msgr2=0x55cd988d8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- 172.21.15.83:0/3644644009 shutdown_connections 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- 172.21.15.83:0/3644644009 wait complete. 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 -- start start 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.132+0000 7f869d2cd740 1 Processor -- start 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd986a1700 con 0x55cd98512390 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd987db110 con 0x55cd9850a1b0 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd988ed1e0 con 0x55cd98533070 2024-09-21T03:16:50.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f869d2cd740 10 client.0 _ll_register_callbacks cb 0x55cd987f4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48090/0 (socket says 172.21.15.83:48090) 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f868ffff640 1 -- 172.21.15.83:0/848221762 learned_addr learned my addr 172.21.15.83:0/848221762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f8694af5640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.133+0000 7f86952f6640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 -- 172.21.15.83:0/848221762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 msgr2=0x55cd988eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 -- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 msgr2=0x55cd988e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988e9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 -- 172.21.15.83:0/848221762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd98808c40 con 0x55cd98512390 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f868ffff640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f867c006010 tx=0x7f867c1058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275ac1d8d16e7dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f866ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c049070 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f867c03bbe0 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8684018af0 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f867c0cd2e0 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f869d2cd740 10 client.15591 fetch_fsmap learned FSMap version 27 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.134+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd988f8d70 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684018af0 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f867c0d7050 con 0x55cd98512390 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 20 client.15591 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984f19f0 0x55cd988fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 4 client.15591 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:16:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd98852da0 con 0x55cd984f19f0 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.135+0000 7f8694af5640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984f19f0 0x55cd988fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.136+0000 7f866ffff640 10 client.15591 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.136+0000 7f8694af5640 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984f19f0 0x55cd988fc1b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dafb6649f288a3 server_cookie=ec6104acf973a59c in_seq=0 out_seq=0 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.136+0000 7f866ffff640 1 -- 172.21.15.83:0/848221762 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd98852da0 con 0x55cd984f19f0 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.136+0000 7f866ffff640 10 client.15591 handle_command_reply: tid=0 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 client.15591 shutdown 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 msgr2=0x55cd988ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f867c006010 tx=0x7f867c1058c0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 shutdown_connections 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984f19f0 0x55cd988fc1b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f866ffff640 0 client.15591 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988ea4c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 --2- 172.21.15.83:0/848221762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 >> 172.21.15.83:0/848221762 conn(0x55cd98527460 msgr2=0x55cd988dd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 shutdown_connections 2024-09-21T03:16:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 1 -- 172.21.15.83:0/848221762 wait complete. 2024-09-21T03:16:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.137+0000 7f869d2cd740 20 client.15591 trim_cache size 0 max 16384 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.171+0000 7f869d2cd740 1 Processor -- start 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.171+0000 7f869d2cd740 1 -- start start 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.171+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f869d2cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ad370 con 0x55cd98512390 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f869d2cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86901aeeb0 con 0x55cd9850a1b0 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f869d2cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ada40 con 0x55cd98533070 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-09-21T03:16:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 -- 172.21.15.83:0/2700757370 learned_addr learned my addr 172.21.15.83:0/2700757370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f8694af5640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f86952f6640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 -- 172.21.15.83:0/2700757370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 msgr2=0x55cd988e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 -- 172.21.15.83:0/2700757370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 msgr2=0x55cd988ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f868ffff640 1 -- 172.21.15.83:0/2700757370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd988de6f0 con 0x55cd98512390 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.172+0000 7f8694af5640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:16:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.173+0000 7f868ffff640 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f867c0d2040 tx=0x7f867c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b19ed5c9a0dac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.173+0000 7f867a7fc640 1 -- 172.21.15.83:0/2700757370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c049070 con 0x55cd98512390 2024-09-21T03:16:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.173+0000 7f867a7fc640 1 -- 172.21.15.83:0/2700757370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c0cd680 con 0x55cd98512390 2024-09-21T03:16:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.173+0000 7f867a7fc640 1 -- 172.21.15.83:0/2700757370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0cd940 con 0x55cd98512390 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 -- 172.21.15.83:0/2700757370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 msgr2=0x55cd988e7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f867c0d2040 tx=0x7f867c02a410 comp rx=0 tx=0).stop 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 -- 172.21.15.83:0/2700757370 shutdown_connections 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd988e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988e7170 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 --2- 172.21.15.83:0/2700757370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 -- 172.21.15.83:0/2700757370 >> 172.21.15.83:0/2700757370 conn(0x55cd98527460 msgr2=0x55cd988dd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 -- 172.21.15.83:0/2700757370 shutdown_connections 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.174+0000 7f869d2cd740 1 -- 172.21.15.83:0/2700757370 wait complete. 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 -- start start 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 Processor -- start 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd98965b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ae7e0 con 0x55cd98512390 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ae110 con 0x55cd9850a1b0 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86901ab350 con 0x55cd98533070 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.175+0000 7f869d2cd740 10 client.0 _ll_register_callbacks cb 0x7f867c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48100/0 (socket says 172.21.15.83:48100) 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 -- 172.21.15.83:0/849073664 learned_addr learned my addr 172.21.15.83:0/849073664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f8694af5640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f86952f6640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd98965b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 -- 172.21.15.83:0/849073664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 msgr2=0x55cd98965b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd98965b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 -- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 msgr2=0x55cd988ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:16:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 -- 172.21.15.83:0/849073664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd98966080 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f86952f6640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd98965b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f868ffff640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f867c0d2010 tx=0x7f867c003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35c559ef883d1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f866ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.176+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c049070 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f867c003800 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867c0cd680 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f867c0cd940 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd98967240 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c0cd680 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f867c0ccf60 con 0x55cd98512390 2024-09-21T03:16:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 20 client.15597 populate_metadata read hostname 'smithi083' 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984c6da0 0x55cd98969830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 4 client.15597 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.177+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd9884e850 con 0x55cd984c6da0 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f8694af5640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984c6da0 0x55cd98969830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f866ffff640 10 client.15597 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f8694af5640 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984c6da0 0x55cd98969830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a1a7ed03aa9370 server_cookie=31b9d6a9f891c172 in_seq=0 out_seq=0 2024-09-21T03:16:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f866ffff640 1 -- 172.21.15.83:0/849073664 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55cd9884e850 con 0x55cd984c6da0 2024-09-21T03:16:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f866ffff640 10 client.15597 handle_command_reply: tid=0 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.178+0000 7f869d2cd740 1 client.15597 shutdown 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 msgr2=0x55cd988eeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f867c0d2010 tx=0x7f867c003e60 comp rx=0 tx=0).stop 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 shutdown_connections 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cd984c6da0 0x55cd98969830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd98533070 0x55cd98965b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd98512390 0x55cd988eeef0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 --2- 172.21.15.83:0/849073664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd9850a1b0 0x55cd988ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 >> 172.21.15.83:0/849073664 conn(0x55cd98527460 msgr2=0x55cd988d8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f866ffff640 0 client.15597 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 shutdown_connections 2024-09-21T03:16:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 1 -- 172.21.15.83:0/849073664 wait complete. 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.179+0000 7f869d2cd740 20 client.15597 trim_cache size 0 max 16384 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1d819a88-2c9a-4459-a4f4-ef203dd15338", 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:16:50.186 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:16:50.187 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 msgr2=0x7f86800b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 0x7f86800b8bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8688007f90 tx=0x7f8688007520 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 msgr2=0x7f869019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f867c02a930 tx=0x7f867c004270 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 shutdown_connections 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86800b66a0 0x7f86800b8bb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8690109e00 0x7f869019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86901094b0 0x7f869019aa30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 --2- 172.21.15.83:0/2930797931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8690103bc0 0x7f869019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 >> 172.21.15.83:0/2930797931 conn(0x7f869007b910 msgr2=0x7f8690105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:16:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 shutdown_connections 2024-09-21T03:16:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:16:50.180+0000 7f8696d59640 1 -- 172.21.15.83:0/2930797931 wait complete. 2024-09-21T03:16:50.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:50 smithi083 ceph-mon[31219]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:16:50.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:50 smithi083 ceph-mon[31219]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:50 smithi142 ceph-mon[34512]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:16:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:50 smithi142 ceph-mon[34512]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:50.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:50 smithi193 ceph-mon[35224]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:16:50.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:50 smithi193 ceph-mon[35224]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:16:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:51 smithi083 ceph-mon[31219]: pgmap v331: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 74 KiB/s wr, 18 op/s 2024-09-21T03:16:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:51 smithi142 ceph-mon[34512]: pgmap v331: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 74 KiB/s wr, 18 op/s 2024-09-21T03:16:51.735 INFO:tasks.cephfs.filesystem:scrub status for tag:1d819a88-2c9a-4459-a4f4-ef203dd15338 - {'path': '/', 'tag': '1d819a88-2c9a-4459-a4f4-ef203dd15338', 'options': 'recursive,force'} 2024-09-21T03:16:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:51 smithi193 ceph-mon[35224]: pgmap v331: 97 pgs: 97 active+clean; 1.0 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 74 KiB/s wr, 18 op/s 2024-09-21T03:16:51.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-21T03:16:51.996 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-21T03:16:51.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-21T03:16:52.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 data 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:52.154 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-09-21T03:16:52.155 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:52.155 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:52.155 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:16:52.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-21T03:16:52.243 INFO:teuthology.orchestra.run.smithi083.stdout:[Service] 2024-09-21T03:16:52.243 INFO:teuthology.orchestra.run.smithi083.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-21T03:16:52.243 INFO:teuthology.orchestra.run.smithi083.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-21T03:16:52.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-21T03:16:52.379 INFO:teuthology.orchestra.run.smithi083.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-21T03:16:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:54 smithi193 ceph-mon[35224]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:16:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:16:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:54 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:54 smithi083 ceph-mon[31219]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:16:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:16:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:54 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:54 smithi142 ceph-mon[34512]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 176 KiB/s wr, 60 op/s 2024-09-21T03:16:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:16:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:54 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 178 KiB/s wr, 57 op/s 2024-09-21T03:16:56.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:16:56.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:16:56.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:55 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 178 KiB/s wr, 57 op/s 2024-09-21T03:16:56.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:16:56.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:16:56.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:55 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:56.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 178 KiB/s wr, 57 op/s 2024-09-21T03:16:56.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:16:56.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:16:56.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:55 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:16:58.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:58 smithi193 ceph-mon[35224]: pgmap v334: 97 pgs: 97 active+clean; 5.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 490 KiB/s wr, 68 op/s 2024-09-21T03:16:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:58 smithi083 ceph-mon[31219]: pgmap v334: 97 pgs: 97 active+clean; 5.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 490 KiB/s wr, 68 op/s 2024-09-21T03:16:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:58 smithi142 ceph-mon[34512]: pgmap v334: 97 pgs: 97 active+clean; 5.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 490 KiB/s wr, 68 op/s 2024-09-21T03:16:58.945 INFO:teuthology.orchestra.run.smithi083.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-21T03:16:58.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 data 2024-09-21T03:16:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:total 35 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 . 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 base 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 03:16 global 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 log 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_commit_ts 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_dynshmem 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 03:16 pg_hba.conf 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 03:16 pg_ident.conf 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 pg_logical 2024-09-21T03:16:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 pg_multixact 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_notify 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_replslot 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_serial 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_snapshots 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_stat 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_stat_tmp 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 pg_subtrans 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_tblspc 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_twophase 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 pg_wal 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 pg_xact 2024-09-21T03:16:58.993 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 03:16 postgresql.auto.conf 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 03:16 postgresql.conf 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 . 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 1 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 13493 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 13494 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 . 2024-09-21T03:16:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 .. 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 1247 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1247_vm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:16 1249 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1249_fsm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1249_vm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:16 1259 2024-09-21T03:16:58.995 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1259_vm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:16:58.996 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:16:58.997 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2579 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:16:58.998 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606_vm 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:16:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:16 2608 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2608_vm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2610 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2610_vm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:16:59.000 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:16:59.001 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 03:16 2619 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2619_fsm 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2619_vm 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:16:59.002 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:16 2658 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 2659 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2662 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2663 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2664 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2665 2024-09-21T03:16:59.003 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2666 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2667 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:16 2673 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:16 2674 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2678 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2679 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:16:59.004 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2696 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:16:59.005 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2703 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2704 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:16:59.006 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:16:59.007 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:16:59.008 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:16:59.009 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:16:59.010 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3455 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:16:59.012 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:16:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:16:59.014 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:16:59.015 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:16:59.016 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:16:59.016 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:16:59.016 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:16:59.018 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:16:59.018 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:16:59.018 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:16:59.019 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-09-21T03:16:59.020 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 . 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 .. 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 1247 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1247_vm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:16 1249 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1249_fsm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1249_vm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:16 1259 2024-09-21T03:16:59.021 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1259_vm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:16:59.022 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:16:59.023 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2579 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:16:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606_vm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:16 2608 2024-09-21T03:16:59.025 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2608_vm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2610 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2610_vm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:16:59.026 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 03:16 2619 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2619_fsm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2619_vm 2024-09-21T03:16:59.027 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:16 2658 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 2659 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2662 2024-09-21T03:16:59.028 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2663 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2664 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2665 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2666 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2667 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:16 2673 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:16 2674 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2678 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2679 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:16:59.029 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:16:59.030 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2696 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2703 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2704 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:16:59.031 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:16:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:16:59.033 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3455 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:16:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:16:59.037 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:16:59.038 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:16:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:16:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T03:16:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 . 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 .. 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 1247 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1247_vm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:16 1249 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1249_fsm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1249_vm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:16 1259 2024-09-21T03:16:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1259_vm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:16:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:16:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2579 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:16:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606_vm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:16:59.046 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:16 2608 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2608_vm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2610 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2610_vm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:16:59.047 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 03:16 2619 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2619_fsm 2024-09-21T03:16:59.048 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2619_vm 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:16 2658 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 2659 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2662 2024-09-21T03:16:59.049 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2663 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2664 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2665 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2666 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2667 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:16 2673 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:16 2674 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2678 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2679 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:16:59.050 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:16:59.051 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2696 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2703 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2704 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:16:59.052 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:16:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:16:59.054 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:16:59.055 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3455 2024-09-21T03:16:59.056 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:16:59.057 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:16:59.058 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:16:59.059 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:16:59.060 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:16:59.061 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:16:59.062 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:16:59.063 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:total 537 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 03:16 . 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1213 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1213_fsm 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1213_vm 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1214 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1214_fsm 2024-09-21T03:16:59.064 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1214_vm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 1232 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 1233 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1260 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1260_fsm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1260_vm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1261 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1261_fsm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1261_vm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1262 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1262_fsm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1262_vm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2396 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2396_fsm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2396_vm 2024-09-21T03:16:59.065 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2397 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2671 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2672 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2676 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2677 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2694 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2695 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2697 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2698 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2846 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2847 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2964 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2965 2024-09-21T03:16:59.066 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2966 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2967 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3592 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3593 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4060 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4061 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4175 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4176 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4177 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4178 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4181 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4182 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4183 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4184 2024-09-21T03:16:59.067 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4185 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4186 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6000 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6001 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6002 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6100 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6114 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6115 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 pg_control 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.068 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:total 1 2024-09-21T03:16:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 . 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 mappings 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 03:16 replorigin_checkpoint 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 snapshots 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 .. 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.070 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 .. 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 . 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 members 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 offsets 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 .. 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 0000 2024-09-21T03:16:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 .. 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 0000 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.072 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.073 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 0000 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.074 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:total 16384 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 . 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.075 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:16 000000010000000000000001 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 archive_status 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 .. 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 03:16 .. 2024-09-21T03:16:59.076 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 0000 2024-09-21T03:16:59.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-21T03:16:59.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:dropping old tables... 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-21T03:16:59.261 INFO:teuthology.orchestra.run.smithi083.stderr:creating tables... 2024-09-21T03:16:59.272 INFO:teuthology.orchestra.run.smithi083.stderr:generating data (client-side)... 2024-09-21T03:16:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.73 s) 2024-09-21T03:16:59.491 INFO:teuthology.orchestra.run.smithi083.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.20 s, remaining 3.06 s) 2024-09-21T03:16:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:16:59 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:16:59 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.37 s, remaining 3.53 s) 2024-09-21T03:16:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:16:59 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:16:59.727 INFO:teuthology.orchestra.run.smithi083.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.44 s, remaining 3.08 s) 2024-09-21T03:16:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.61 s, remaining 3.27 s) 2024-09-21T03:17:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.75 s, remaining 3.26 s) 2024-09-21T03:17:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.84 s, remaining 2.99 s) 2024-09-21T03:17:00.257 INFO:teuthology.orchestra.run.smithi083.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.97 s, remaining 2.91 s) 2024-09-21T03:17:00.356 INFO:teuthology.orchestra.run.smithi083.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.07 s, remaining 2.73 s) 2024-09-21T03:17:00.494 INFO:teuthology.orchestra.run.smithi083.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.21 s, remaining 2.66 s) 2024-09-21T03:17:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:00 smithi193 ceph-mon[35224]: pgmap v335: 97 pgs: 97 active+clean; 32 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 2.8 MiB/s wr, 105 op/s 2024-09-21T03:17:00.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-mon[31219]: pgmap v335: 97 pgs: 97 active+clean; 32 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 2.8 MiB/s wr, 105 op/s 2024-09-21T03:17:00.649 INFO:teuthology.orchestra.run.smithi083.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.36 s, remaining 2.60 s) 2024-09-21T03:17:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:00 smithi142 ceph-mon[34512]: pgmap v335: 97 pgs: 97 active+clean; 32 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 2.8 MiB/s wr, 105 op/s 2024-09-21T03:17:01.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:17:00.666+0000 7f4692b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:17:01.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:17:00.666+0000 7f4692b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:17:01.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:17:00.666+0000 7f4692b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:17:01.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:17:00.666+0000 7f4692b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:17:01.122 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:17:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x[31446]: 2024-09-21T03:17:00.666+0000 7f4692b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:17:02.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:02 smithi193 ceph-mon[35224]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.2 MiB/s wr, 99 op/s 2024-09-21T03:17:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:02 smithi083 ceph-mon[31219]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.2 MiB/s wr, 99 op/s 2024-09-21T03:17:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:02 smithi142 ceph-mon[34512]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.2 MiB/s wr, 99 op/s 2024-09-21T03:17:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.60 s, remaining 6.00 s) 2024-09-21T03:17:03.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:03 smithi193 ceph-mon[35224]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:17:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:03 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:03 smithi083 ceph-mon[31219]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:17:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:03 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:03 smithi142 ceph-mon[34512]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:17:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:03 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:04.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:04 smithi193 ceph-mon[35224]: pgmap v337: 97 pgs: 97 active+clean; 128 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 18 MiB/s wr, 258 op/s 2024-09-21T03:17:04.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:04 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:04 smithi083 ceph-mon[31219]: pgmap v337: 97 pgs: 97 active+clean; 128 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 18 MiB/s wr, 258 op/s 2024-09-21T03:17:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:04 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:04 smithi142 ceph-mon[34512]: pgmap v337: 97 pgs: 97 active+clean; 128 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 18 MiB/s wr, 258 op/s 2024-09-21T03:17:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:04 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:1300000 of 3200000 tuples (40%) done (elapsed 7.09 s, remaining 10.36 s) 2024-09-21T03:17:06.519 INFO:teuthology.orchestra.run.smithi083.stderr:1400000 of 3200000 tuples (43%) done (elapsed 7.23 s, remaining 9.30 s) 2024-09-21T03:17:06.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:06 smithi193 ceph-mon[35224]: pgmap v338: 97 pgs: 97 active+clean; 168 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 23 MiB/s wr, 308 op/s 2024-09-21T03:17:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:1500000 of 3200000 tuples (46%) done (elapsed 7.31 s, remaining 8.28 s) 2024-09-21T03:17:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:06 smithi083 ceph-mon[31219]: pgmap v338: 97 pgs: 97 active+clean; 168 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 23 MiB/s wr, 308 op/s 2024-09-21T03:17:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:06 smithi142 ceph-mon[34512]: pgmap v338: 97 pgs: 97 active+clean; 168 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 23 MiB/s wr, 308 op/s 2024-09-21T03:17:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:1600000 of 3200000 tuples (50%) done (elapsed 7.45 s, remaining 7.45 s) 2024-09-21T03:17:06.818 INFO:teuthology.orchestra.run.smithi083.stderr:1700000 of 3200000 tuples (53%) done (elapsed 7.53 s, remaining 6.65 s) 2024-09-21T03:17:06.984 INFO:teuthology.orchestra.run.smithi083.stderr:1800000 of 3200000 tuples (56%) done (elapsed 7.70 s, remaining 5.99 s) 2024-09-21T03:17:07.112 INFO:teuthology.orchestra.run.smithi083.stderr:1900000 of 3200000 tuples (59%) done (elapsed 7.83 s, remaining 5.35 s) 2024-09-21T03:17:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2000000 of 3200000 tuples (62%) done (elapsed 7.91 s, remaining 4.75 s) 2024-09-21T03:17:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2100000 of 3200000 tuples (65%) done (elapsed 8.05 s, remaining 4.22 s) 2024-09-21T03:17:07.490 INFO:teuthology.orchestra.run.smithi083.stderr:2200000 of 3200000 tuples (68%) done (elapsed 8.20 s, remaining 3.73 s) 2024-09-21T03:17:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2300000 of 3200000 tuples (71%) done (elapsed 8.29 s, remaining 3.24 s) 2024-09-21T03:17:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2400000 of 3200000 tuples (75%) done (elapsed 8.44 s, remaining 2.81 s) 2024-09-21T03:17:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2500000 of 3200000 tuples (78%) done (elapsed 8.59 s, remaining 2.40 s) 2024-09-21T03:17:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2600000 of 3200000 tuples (81%) done (elapsed 8.67 s, remaining 2.00 s) 2024-09-21T03:17:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2700000 of 3200000 tuples (84%) done (elapsed 8.80 s, remaining 1.63 s) 2024-09-21T03:17:08.182 INFO:teuthology.orchestra.run.smithi083.stderr:2800000 of 3200000 tuples (87%) done (elapsed 8.89 s, remaining 1.27 s) 2024-09-21T03:17:08.341 INFO:teuthology.orchestra.run.smithi083.stderr:2900000 of 3200000 tuples (90%) done (elapsed 9.05 s, remaining 0.94 s) 2024-09-21T03:17:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:3000000 of 3200000 tuples (93%) done (elapsed 9.17 s, remaining 0.61 s) 2024-09-21T03:17:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:08 smithi193 ceph-mon[35224]: pgmap v339: 97 pgs: 97 active+clean; 176 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 24 MiB/s wr, 308 op/s 2024-09-21T03:17:08.572 INFO:teuthology.orchestra.run.smithi083.stderr:3100000 of 3200000 tuples (96%) done (elapsed 9.29 s, remaining 0.30 s) 2024-09-21T03:17:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:08 smithi083 ceph-mon[31219]: pgmap v339: 97 pgs: 97 active+clean; 176 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 24 MiB/s wr, 308 op/s 2024-09-21T03:17:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:08 smithi142 ceph-mon[34512]: pgmap v339: 97 pgs: 97 active+clean; 176 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 24 MiB/s wr, 308 op/s 2024-09-21T03:17:08.746 INFO:teuthology.orchestra.run.smithi083.stderr:3200000 of 3200000 tuples (100%) done (elapsed 9.46 s, remaining 0.00 s) 2024-09-21T03:17:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:09 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:09 smithi193 ceph-mon[35224]: pgmap v340: 97 pgs: 97 active+clean; 273 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 39 MiB/s wr, 493 op/s 2024-09-21T03:17:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:09 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:09 smithi083 ceph-mon[31219]: pgmap v340: 97 pgs: 97 active+clean; 273 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 39 MiB/s wr, 493 op/s 2024-09-21T03:17:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:09 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:09 smithi142 ceph-mon[34512]: pgmap v340: 97 pgs: 97 active+clean; 273 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 39 MiB/s wr, 493 op/s 2024-09-21T03:17:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:12 smithi193 ceph-mon[35224]: pgmap v341: 97 pgs: 97 active+clean; 309 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 42 MiB/s wr, 458 op/s 2024-09-21T03:17:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:12 smithi083 ceph-mon[31219]: pgmap v341: 97 pgs: 97 active+clean; 309 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 42 MiB/s wr, 458 op/s 2024-09-21T03:17:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:12 smithi142 ceph-mon[34512]: pgmap v341: 97 pgs: 97 active+clean; 309 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 42 MiB/s wr, 458 op/s 2024-09-21T03:17:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:14 smithi193 ceph-mon[35224]: pgmap v342: 97 pgs: 97 active+clean; 393 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 55 MiB/s wr, 462 op/s 2024-09-21T03:17:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:14 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:14 smithi083 ceph-mon[31219]: pgmap v342: 97 pgs: 97 active+clean; 393 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 55 MiB/s wr, 462 op/s 2024-09-21T03:17:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:14 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:14 smithi142 ceph-mon[34512]: pgmap v342: 97 pgs: 97 active+clean; 393 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 55 MiB/s wr, 462 op/s 2024-09-21T03:17:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:14 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:16 smithi193 ceph-mon[35224]: pgmap v343: 97 pgs: 97 active+clean; 393 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 303 op/s 2024-09-21T03:17:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:16 smithi083 ceph-mon[31219]: pgmap v343: 97 pgs: 97 active+clean; 393 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 303 op/s 2024-09-21T03:17:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:16 smithi142 ceph-mon[34512]: pgmap v343: 97 pgs: 97 active+clean; 393 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 303 op/s 2024-09-21T03:17:18.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:18 smithi193 ceph-mon[35224]: pgmap v344: 97 pgs: 97 active+clean; 393 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 212 op/s 2024-09-21T03:17:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:18 smithi083 ceph-mon[31219]: pgmap v344: 97 pgs: 97 active+clean; 393 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 212 op/s 2024-09-21T03:17:18.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:18 smithi142 ceph-mon[34512]: pgmap v344: 97 pgs: 97 active+clean; 393 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 212 op/s 2024-09-21T03:17:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:vacuuming... 2024-09-21T03:17:19.658 INFO:teuthology.orchestra.run.smithi083.stderr:creating primary keys... 2024-09-21T03:17:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:20 smithi193 ceph-mon[35224]: pgmap v345: 97 pgs: 97 active+clean; 439 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 39 MiB/s wr, 218 op/s 2024-09-21T03:17:20.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:20 smithi083 ceph-mon[31219]: pgmap v345: 97 pgs: 97 active+clean; 439 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 39 MiB/s wr, 218 op/s 2024-09-21T03:17:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:20 smithi142 ceph-mon[34512]: pgmap v345: 97 pgs: 97 active+clean; 439 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 39 MiB/s wr, 218 op/s 2024-09-21T03:17:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:done in 21.60 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 19.52 s, vacuum 0.87 s, primary keys 1.20 s). 2024-09-21T03:17:20.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-21T03:17:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:starting vacuum...end. 2024-09-21T03:17:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:21 smithi083 ceph-mon[31219]: pgmap v346: 97 pgs: 97 active+clean; 507 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-21T03:17:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:21 smithi142 ceph-mon[34512]: pgmap v346: 97 pgs: 97 active+clean; 507 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-21T03:17:21.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:17:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:21 smithi193 ceph-mon[35224]: pgmap v346: 97 pgs: 97 active+clean; 507 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-21T03:17:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 -- 172.21.15.83:0/1296887396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec1083b0 msgr2=0x7fa2ec1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 --2- 172.21.15.83:0/1296887396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec1087c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009900 tx=0x7fa2dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 -- 172.21.15.83:0/1296887396 shutdown_connections 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 --2- 172.21.15.83:0/1296887396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec108d00 0x7fa2ec10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 --2- 172.21.15.83:0/1296887396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec1087c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 --2- 172.21.15.83:0/1296887396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 0x7fa2ec103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 -- 172.21.15.83:0/1296887396 >> 172.21.15.83:0/1296887396 conn(0x7fa2ec077c60 msgr2=0x7fa2ec0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 -- 172.21.15.83:0/1296887396 shutdown_connections 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.852+0000 7fa2f4238640 1 -- 172.21.15.83:0/1296887396 wait complete. 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 Processor -- start 2024-09-21T03:17:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 -- start start 2024-09-21T03:17:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 0x7fa2ec1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec112a10 con 0x7fa2ec108d00 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec112890 con 0x7fa2ec103140 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.853+0000 7fa2f4238640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec112b90 con 0x7fa2ec1083b0 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55830/0 (socket says 172.21.15.83:55830) 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 -- 172.21.15.83:0/368563417 learned_addr learned my addr 172.21.15.83:0/368563417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 0x7fa2ec1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 -- 172.21.15.83:0/368563417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1083b0 msgr2=0x7fa2ec199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 -- 172.21.15.83:0/368563417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 msgr2=0x7fa2ec1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 0x7fa2ec1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 -- 172.21.15.83:0/368563417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19f310 con 0x7fa2ec108d00 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2f27d5640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400eaa0 tx=0x7fa2e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.854+0000 7fa2e2ffd640 1 -- 172.21.15.83:0/368563417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400cdb0 con 0x7fa2ec108d00 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.855+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec19f600 con 0x7fa2ec108d00 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.855+0000 7fa2e2ffd640 1 -- 172.21.15.83:0/368563417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e4005020 con 0x7fa2ec108d00 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.855+0000 7fa2e2ffd640 1 -- 172.21.15.83:0/368563417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40116d0 con 0x7fa2ec108d00 2024-09-21T03:17:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.855+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec101da0 con 0x7fa2ec108d00 2024-09-21T03:17:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.858+0000 7fa2fa7ac740 1 Processor -- start 2024-09-21T03:17:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.859+0000 7fa2e2ffd640 1 -- 172.21.15.83:0/368563417 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa2e40054a0 con 0x7fa2ec108d00 2024-09-21T03:17:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.859+0000 7fa2fa7ac740 1 -- start start 2024-09-21T03:17:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b922070 0x55587bcc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b901390 0x55587bcca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55587bc3fe50 con 0x55587b922070 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55587bc2f280 con 0x55587b8f91b0 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55587bc33090 con 0x55587b901390 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53762/0 (socket says 172.21.15.83:53762) 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2e2ffd640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 0x7fa2d80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/821998037 learned_addr learned my addr 172.21.15.83:0/821998037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b922070 0x55587bcc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f27d5640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 0x7fa2d80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b901390 0x55587bcca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/821998037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b901390 msgr2=0x55587bcca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b901390 0x55587bcca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/821998037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b922070 msgr2=0x55587bcc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b922070 0x55587bcc9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/821998037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587b98d650 con 0x55587b8f91b0 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.860+0000 7fa2e2ffd640 1 -- 172.21.15.83:0/368563417 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2e40e7780 con 0x7fa2ec108d00 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.861+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800bd10 tx=0x7fa2e80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93eb841846e88d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.861+0000 7fa2d7fff640 1 -- 172.21.15.83:0/821998037 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8004030 con 0x55587b8f91b0 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.861+0000 7fa2f27d5640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 0x7fa2d80b8bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40e5550 tx=0x7fa2e4005840 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.861+0000 7fa2d7fff640 1 -- 172.21.15.83:0/821998037 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e80035f0 con 0x55587b8f91b0 2024-09-21T03:17:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.861+0000 7fa2d7fff640 1 -- 172.21.15.83:0/821998037 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80116a0 con 0x55587b8f91b0 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.862+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/821998037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 msgr2=0x55587bcc9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.862+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800bd10 tx=0x7fa2e80107a0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/821998037 shutdown_connections 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b901390 0x55587bcca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8f91b0 0x55587bcc9b90 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800bd10 tx=0x7fa2e80107a0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/821998037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b922070 0x55587bcc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/821998037 >> 172.21.15.83:0/821998037 conn(0x55587b916460 msgr2=0x55587bcc7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.863+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/821998037 shutdown_connections 2024-09-21T03:17:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.864+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/821998037 wait complete. 2024-09-21T03:17:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.864+0000 7fa2fa7ac740 1 -- start start 2024-09-21T03:17:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.864+0000 7fa2fa7ac740 1 Processor -- start 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bcd9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bce5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55587ba90700 con 0x55587b901390 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55587bbca110 con 0x55587b8e09f0 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55587bcdc130 con 0x55587b922070 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2fa7ac740 10 client.0 _ll_register_callbacks cb 0x55587bbe3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:22.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f17d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bcd9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40352/0 (socket says 172.21.15.83:40352) 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/2716956773 learned_addr learned my addr 172.21.15.83:0/2716956773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f17d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bcd9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55850/0 (socket says 172.21.15.83:55850) 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.865+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bce5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 msgr2=0x55587bce5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bce5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/2716956773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 msgr2=0x55587bcd9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bcd9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/2716956773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587bbf7c40 con 0x55587b922070 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bce5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8000c00 tx=0x7fa2e800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa8a35ea05b510f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8029060 con 0x55587b922070 2024-09-21T03:17:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e80035f0 con 0x55587b922070 2024-09-21T03:17:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2e8003b30 con 0x55587b922070 2024-09-21T03:17:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2e8003f60 con 0x55587b922070 2024-09-21T03:17:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.866+0000 7fa2fa7ac740 10 client.25102 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.867+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55587bcd11f0 con 0x55587b922070 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.867+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e80035f0 con 0x55587b922070 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.867+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa2e80046c0 con 0x55587b922070 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 10 client.25102 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 10 client.25102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 20 client.25102 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587b8b5da0 0x55587bcd25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 4 client.25102 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55587bc41da0 con 0x55587b8b5da0 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.868+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587b8b5da0 0x55587bcd25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587b8b5da0 0x55587bcd25a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d5b7946cbad622 server_cookie=e5bc5a89caca4608 in_seq=0 out_seq=0 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2d57fa640 10 client.25102 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2d57fa640 1 -- 172.21.15.83:0/2716956773 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55587bc41da0 con 0x55587b8b5da0 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2d57fa640 10 client.25102 handle_command_reply: tid=0 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2fa7ac740 1 client.25102 shutdown 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 msgr2=0x55587bcd9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.869+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8000c00 tx=0x7fa2e800bfd0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 shutdown_connections 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587b8b5da0 0x55587bcd25a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bce5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcd9790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2d57fa640 0 client.25102 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/2716956773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bcd9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 >> 172.21.15.83:0/2716956773 conn(0x55587b916460 msgr2=0x55587bcc6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 shutdown_connections 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/2716956773 wait complete. 2024-09-21T03:17:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.870+0000 7fa2fa7ac740 20 client.25102 trim_cache size 0 max 16384 2024-09-21T03:17:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 Processor -- start 2024-09-21T03:17:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 -- start start 2024-09-21T03:17:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bcd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcc6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1ad440 con 0x55587b901390 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1af040 con 0x55587b8e09f0 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.895+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1adb10 con 0x55587b922070 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55860/0 (socket says 172.21.15.83:55860) 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/1700420037 learned_addr learned my addr 172.21.15.83:0/1700420037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bcd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 -- 172.21.15.83:0/1700420037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 msgr2=0x55587bcc6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcc6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 -- 172.21.15.83:0/1700420037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 msgr2=0x55587bce53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f17d3640 1 -- 172.21.15.83:0/1700420037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587bcd81b0 con 0x55587b8e09f0 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.896+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.897+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bcd9710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009230 tx=0x7fa2dc009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b2c9191098aeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.897+0000 7fa2d77fe640 1 -- 172.21.15.83:0/1700420037 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc047040 con 0x55587b8e09f0 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.897+0000 7fa2d77fe640 1 -- 172.21.15.83:0/1700420037 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc004740 con 0x55587b8e09f0 2024-09-21T03:17:22.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.897+0000 7fa2d77fe640 1 -- 172.21.15.83:0/1700420037 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc035040 con 0x55587b8e09f0 2024-09-21T03:17:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.898+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1700420037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 msgr2=0x55587bcd9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.898+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bcd9710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009230 tx=0x7fa2dc009600 comp rx=0 tx=0).stop 2024-09-21T03:17:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1700420037 shutdown_connections 2024-09-21T03:17:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bcc6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b901390 0x55587bce53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1700420037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b8e09f0 0x55587bcd9710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1700420037 >> 172.21.15.83:0/1700420037 conn(0x55587b916460 msgr2=0x55587bcd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1700420037 shutdown_connections 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.899+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1700420037 wait complete. 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.900+0000 7fa2fa7ac740 1 -- start start 2024-09-21T03:17:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.900+0000 7fa2fa7ac740 1 Processor -- start 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b8e09f0 0x55587bcf17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bd59170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1ae8b0 con 0x55587b8e09f0 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1ae1e0 con 0x55587b901390 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1ab440 con 0x55587b922070 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53802/0 (socket says 172.21.15.83:53802) 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/1411883222 learned_addr learned my addr 172.21.15.83:0/1411883222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2fa7ac740 10 client.0 _ll_register_callbacks cb 0x7fa2dc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bd59170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/1411883222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 msgr2=0x55587bd59170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bd59170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/1411883222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b8e09f0 msgr2=0x55587bcf17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b8e09f0 0x55587bcf17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.901+0000 7fa2f1fd4640 1 -- 172.21.15.83:0/1411883222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587bd597e0 con 0x55587b901390 2024-09-21T03:17:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.902+0000 7fa2f1fd4640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80107e0 tx=0x7fa2e8004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8e8d49e7826872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.904+0000 7fa2d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.904+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8029070 con 0x55587b901390 2024-09-21T03:17:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.904+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2e8003d40 con 0x55587b901390 2024-09-21T03:17:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.904+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc004740 con 0x55587b901390 2024-09-21T03:17:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.904+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2e800be00 con 0x55587b901390 2024-09-21T03:17:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 10 client.25142 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55587bd5aa80 con 0x55587b901390 2024-09-21T03:17:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e8005490 con 0x55587b901390 2024-09-21T03:17:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa2e80045b0 con 0x55587b901390 2024-09-21T03:17:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 10 client.25142 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 10 client.25142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 20 client.25142 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587bd5b2c0 0x55587bd5d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 4 client.25142 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa2dc004740 con 0x55587bd5b2c0 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.905+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587bd5b2c0 0x55587bd5d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2d57fa640 10 client.25142 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2f17d3640 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587bd5b2c0 0x55587bd5d6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fea9a9ea6f22c7b server_cookie=9a26812d433e0951 in_seq=0 out_seq=0 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2d57fa640 1 -- 172.21.15.83:0/1411883222 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7fa2dc004740 con 0x55587bd5b2c0 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2d57fa640 10 client.25142 handle_command_reply: tid=0 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 client.25142 shutdown 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 msgr2=0x55587bcf1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80107e0 tx=0x7fa2e8004f70 comp rx=0 tx=0).stop 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 shutdown_connections 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55587bd5b2c0 0x55587bd5d6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55587b922070 0x55587bd59170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55587b901390 0x55587bcf1d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 --2- 172.21.15.83:0/1411883222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55587b8e09f0 0x55587bcf17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 >> 172.21.15.83:0/1411883222 conn(0x55587b916460 msgr2=0x55587bcc6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.906+0000 7fa2d57fa640 0 client.25142 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.908+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 shutdown_connections 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.908+0000 7fa2fa7ac740 1 -- 172.21.15.83:0/1411883222 wait complete. 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.909+0000 7fa2fa7ac740 20 client.25142 trim_cache size 0 max 16384 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:17:22.911 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:17:22.912 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 msgr2=0x7fa2d80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 0x7fa2d80b8bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40e5550 tx=0x7fa2e4005840 comp rx=0 tx=0).stop 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 msgr2=0x7fa2ec19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400eaa0 tx=0x7fa2e400ef60 comp rx=0 tx=0).stop 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 shutdown_connections 2024-09-21T03:17:22.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2d80b66b0 0x7fa2d80b8bc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2ec108d00 0x7fa2ec19ebe0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ec1083b0 0x7fa2ec199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 --2- 172.21.15.83:0/368563417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec103140 0x7fa2ec1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.910+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 >> 172.21.15.83:0/368563417 conn(0x7fa2ec077c60 msgr2=0x7fa2ec104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.911+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 shutdown_connections 2024-09-21T03:17:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:22.911+0000 7fa2f4238640 1 -- 172.21.15.83:0/368563417 wait complete. 2024-09-21T03:17:23.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:17:23.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:17:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:24.420 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:24 smithi083 ceph-mon[31219]: pgmap v347: 97 pgs: 97 active+clean; 826 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 54 MiB/s wr, 75 op/s 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/1444576457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0acaa0 msgr2=0x7f6bcc0aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/1444576457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0aceb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009890 tx=0x7f6bc402a380 comp rx=0 tx=0).stop 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/1444576457 shutdown_connections 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/1444576457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/1444576457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0aceb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/1444576457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/1444576457 >> 172.21.15.83:0/1444576457 conn(0x7f6bcc01cc20 msgr2=0x7f6bcc01d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/1444576457 shutdown_connections 2024-09-21T03:17:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.532+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/1444576457 wait complete. 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 Processor -- start 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 -- start start 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0b7010 con 0x7f6bcc0ad3f0 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0b6e90 con 0x7f6bcc0acaa0 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.533+0000 7f6bdaa2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0b7190 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40388/0 (socket says 172.21.15.83:40388) 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 -- 172.21.15.83:0/4160980009 learned_addr learned my addr 172.21.15.83:0/4160980009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9227640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bda229640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 -- 172.21.15.83:0/4160980009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0acaa0 msgr2=0x7f6bcc0a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 -- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 msgr2=0x7f6bcc0a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0a7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 -- 172.21.15.83:0/4160980009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc14bda0 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bda229640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:24.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.534+0000 7f6bd9a28640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4051d10 tx=0x7f6bd406bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.535+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4075e30 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.535+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd40778e0 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.535+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd407f960 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.535+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc14c090 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.535+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc14c550 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.536+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0a54e0 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.539+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6bd4077430 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.540+0000 7f6bcaffd640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 0x7f6bc00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:24.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.541+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6bd414d470 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.541+0000 7f6bd9227640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 0x7f6bc00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.542+0000 7f6bd9227640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 0x7f6bc00b8c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bc402a970 tx=0x7f6bc4030340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:24.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.544+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6bd407b030 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:24 smithi193 ceph-mon[35224]: pgmap v347: 97 pgs: 97 active+clean; 826 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 54 MiB/s wr, 75 op/s 2024-09-21T03:17:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:24 smithi142 ceph-mon[34512]: pgmap v347: 97 pgs: 97 active+clean; 826 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 54 MiB/s wr, 75 op/s 2024-09-21T03:17:24.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.883+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bcc0a7c00 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.883+0000 7f6bcaffd640 1 -- 172.21.15.83:0/4160980009 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11989 (secure 0 0 0) 0x7f6bd41139e0 con 0x7f6bcc0a50d0 2024-09-21T03:17:24.886 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:17:24.886 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"btime":"2024-09-21T03:17:13:263958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:17:13.239520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:17:24.887 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-09-21T03:17:24.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 msgr2=0x7f6bc00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 0x7f6bc00b8c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bc402a970 tx=0x7f6bc4030340 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 msgr2=0x7f6bcc0a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4051d10 tx=0x7f6bd406bea0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 shutdown_connections 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6bc00b6780 0x7f6bc00b8c90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bcc0ad3f0 0x7f6bcc0a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bcc0acaa0 0x7f6bcc0a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 --2- 172.21.15.83:0/4160980009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0a50d0 0x7f6bcc0a6900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 >> 172.21.15.83:0/4160980009 conn(0x7f6bcc01cc20 msgr2=0x7f6bcc0a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 shutdown_connections 2024-09-21T03:17:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:24.888+0000 7f6bdaa2a640 1 -- 172.21.15.83:0/4160980009 wait complete. 2024-09-21T03:17:25.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4160980009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:17:25.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:25 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4160980009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:17:25.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:17:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:25 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4160980009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:17:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 5.0 s, 3443.2 tps, lat 28.540 ms stddev 28.106 2024-09-21T03:17:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:26.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:26 smithi083 ceph-mon[31219]: pgmap v348: 97 pgs: 97 active+clean; 918 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 78 op/s 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 -- 172.21.15.83:0/240700145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66701083d0 msgr2=0x7f66701087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 --2- 172.21.15.83:0/240700145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66701083d0 0x7f66701087e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6660009900 tx=0x7f666002a410 comp rx=0 tx=0).stop 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 -- 172.21.15.83:0/240700145 shutdown_connections 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 --2- 172.21.15.83:0/240700145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 --2- 172.21.15.83:0/240700145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66701083d0 0x7f66701087e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 --2- 172.21.15.83:0/240700145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6670102ae0 0x7f6670102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 -- 172.21.15.83:0/240700145 >> 172.21.15.83:0/240700145 conn(0x7f6670079760 msgr2=0x7f6670079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 -- 172.21.15.83:0/240700145 shutdown_connections 2024-09-21T03:17:26.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.545+0000 7f6678386640 1 -- 172.21.15.83:0/240700145 wait complete. 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 Processor -- start 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 -- start start 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 0x7f667019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 0x7f667019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112990 con 0x7f6670102ae0 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676122640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 0x7f667019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.546+0000 7f6678386640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112810 con 0x7f66701083d0 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6678386640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112b10 con 0x7f6670108d20 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676122640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 0x7f667019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55896/0 (socket says 172.21.15.83:55896) 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6675921640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 0x7f667019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6675921640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 0x7f667019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53850/0 (socket says 172.21.15.83:53850) 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6675921640 1 -- 172.21.15.83:0/3993269244 learned_addr learned my addr 172.21.15.83:0/3993269244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 -- 172.21.15.83:0/3993269244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 msgr2=0x7f667019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 0x7f667019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 -- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 msgr2=0x7f667019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 0x7f667019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 -- 172.21.15.83:0/3993269244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701a0400 con 0x7f6670108d20 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.547+0000 7f6676923640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667019fcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6668009fd0 tx=0x7f666800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.548+0000 7f66677fe640 1 -- 172.21.15.83:0/3993269244 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800ce20 con 0x7f6670108d20 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.548+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66701a06f0 con 0x7f6670108d20 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.548+0000 7f66677fe640 1 -- 172.21.15.83:0/3993269244 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668005020 con 0x7f6670108d20 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.548+0000 7f66677fe640 1 -- 172.21.15.83:0/3993269244 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668010690 con 0x7f6670108d20 2024-09-21T03:17:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.548+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66700ff550 con 0x7f6670108d20 2024-09-21T03:17:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.551+0000 7f667e8fa740 1 Processor -- start 2024-09-21T03:17:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.551+0000 7f66677fe640 1 -- 172.21.15.83:0/3993269244 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f66680054d0 con 0x7f6670108d20 2024-09-21T03:17:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 -- start start 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf27d070 0x55e7bf6242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf624d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf59aea0 con 0x55e7bf27d070 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf58a2d0 con 0x55e7bf2541b0 2024-09-21T03:17:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.552+0000 7f667e8fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf58e0e0 con 0x55e7bf25c390 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53864/0 (socket says 172.21.15.83:53864) 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 -- 172.21.15.83:0/3646275141 learned_addr learned my addr 172.21.15.83:0/3646275141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6675921640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf27d070 0x55e7bf6242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f66677fe640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 0x7f665c0bd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6675921640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf624d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 -- 172.21.15.83:0/3646275141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 msgr2=0x55e7bf624d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676923640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 0x7f665c0bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf624d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 -- 172.21.15.83:0/3646275141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf27d070 msgr2=0x55e7bf6242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf27d070 0x55e7bf6242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6676122640 1 -- 172.21.15.83:0/3646275141 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7bf2e8650 con 0x55e7bf2541b0 2024-09-21T03:17:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.553+0000 7f6675921640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf624d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:17:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.554+0000 7f66677fe640 1 -- 172.21.15.83:0/3993269244 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f66680e7d50 con 0x7f6670108d20 2024-09-21T03:17:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.554+0000 7f6676122640 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f666c00bd10 tx=0x7f666c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea245fce4b8dddda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.554+0000 7f6664d9a640 1 -- 172.21.15.83:0/3646275141 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c004030 con 0x55e7bf2541b0 2024-09-21T03:17:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.554+0000 7f6676923640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 0x7f665c0bd020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66680e55d0 tx=0x7f66680a9b60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.555+0000 7f6664d9a640 1 -- 172.21.15.83:0/3646275141 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666c0035f0 con 0x55e7bf2541b0 2024-09-21T03:17:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f6664d9a640 1 -- 172.21.15.83:0/3646275141 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c004030 con 0x55e7bf2541b0 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 -- 172.21.15.83:0/3646275141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 msgr2=0x55e7bf624800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f666c00bd10 tx=0x7f666c0107a0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 -- 172.21.15.83:0/3646275141 shutdown_connections 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf624d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf2541b0 0x55e7bf624800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.556+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3646275141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf27d070 0x55e7bf6242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 -- 172.21.15.83:0/3646275141 >> 172.21.15.83:0/3646275141 conn(0x55e7bf271460 msgr2=0x55e7bf6221b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 -- 172.21.15.83:0/3646275141 shutdown_connections 2024-09-21T03:17:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 -- 172.21.15.83:0/3646275141 wait complete. 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 -- start start 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 Processor -- start 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.557+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf25c390 0x55e7bf62fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf27d070 0x55e7bf6303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf3eb700 con 0x55e7bf2541b0 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf525160 con 0x55e7bf25c390 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e7bf632af0 con 0x55e7bf27d070 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 10 client.0 _ll_register_callbacks cb 0x55e7bf53e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55914/0 (socket says 172.21.15.83:55914) 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 -- 172.21.15.83:0/1368720298 learned_addr learned my addr 172.21.15.83:0/1368720298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf27d070 0x55e7bf6303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 -- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf27d070 msgr2=0x55e7bf6303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf27d070 0x55e7bf6303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 -- 172.21.15.83:0/1368720298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf25c390 msgr2=0x55e7bf62fe60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf25c390 0x55e7bf62fe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 -- 172.21.15.83:0/1368720298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7bf552c90 con 0x55e7bf2541b0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6675921640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66600315c0 tx=0x7f666002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb83063ba0140369 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660047040 con 0x55e7bf2541b0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.558+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666c0035f0 con 0x55e7bf2541b0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6660004970 con 0x55e7bf2541b0 2024-09-21T03:17:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6660053050 con 0x55e7bf2541b0 2024-09-21T03:17:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f667e8fa740 10 client.15636 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e7bf63e730 con 0x55e7bf2541b0 2024-09-21T03:17:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66600316d0 con 0x55e7bf2541b0 2024-09-21T03:17:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f666003a040 con 0x55e7bf2541b0 2024-09-21T03:17:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f667e8fa740 10 client.15636 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f667e8fa740 10 client.15636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:17:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.559+0000 7f667e8fa740 20 client.15636 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf23b9f0 0x55e7bf641b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f667e8fa740 4 client.15636 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f666c0035f0 con 0x55e7bf23b9f0 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f6676122640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf23b9f0 0x55e7bf641b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f6676122640 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf23b9f0 0x55e7bf641b90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa53185141793650 server_cookie=50998b4e5eacc977 in_seq=0 out_seq=0 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.560+0000 7f6659ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.561+0000 7f6659ffb640 1 -- 172.21.15.83:0/1368720298 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f666c0035f0 con 0x55e7bf23b9f0 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.561+0000 7f6659ffb640 10 client.15636 handle_command_reply: tid=0 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.561+0000 7f667e8fa740 1 client.15636 shutdown 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 msgr2=0x55e7bf62f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f66600315c0 tx=0x7f666002a410 comp rx=0 tx=0).stop 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 shutdown_connections 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf23b9f0 0x55e7bf641b90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf27d070 0x55e7bf6303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf25c390 0x55e7bf62fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f6659ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 --2- 172.21.15.83:0/1368720298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf62f920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 >> 172.21.15.83:0/1368720298 conn(0x55e7bf271460 msgr2=0x55e7bf6252f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 shutdown_connections 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 1 -- 172.21.15.83:0/1368720298 wait complete. 2024-09-21T03:17:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.562+0000 7f667e8fa740 20 client.15636 trim_cache size 0 max 16384 2024-09-21T03:17:26.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:26 smithi193 ceph-mon[35224]: pgmap v348: 97 pgs: 97 active+clean; 918 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 78 op/s 2024-09-21T03:17:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.579+0000 7f667e8fa740 1 Processor -- start 2024-09-21T03:17:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.579+0000 7f667e8fa740 1 -- start start 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf25c390 0x55e7bf6257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf62cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66701ad430 con 0x55e7bf25c390 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66701af030 con 0x55e7bf27d070 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f667e8fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66701adb00 con 0x55e7bf2541b0 2024-09-21T03:17:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40414/0 (socket says 172.21.15.83:40414) 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 -- 172.21.15.83:0/3217694852 learned_addr learned my addr 172.21.15.83:0/3217694852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf62cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 -- 172.21.15.83:0/3217694852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 msgr2=0x55e7bf62cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf62cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 -- 172.21.15.83:0/3217694852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf25c390 msgr2=0x55e7bf6257b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf25c390 0x55e7bf6257b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.580+0000 7f6675921640 1 -- 172.21.15.83:0/3217694852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7bf6265b0 con 0x55e7bf2541b0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.581+0000 7f6675921640 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66600044e0 tx=0x7f6660009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f568be0689880f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.581+0000 7f665bfff640 1 -- 172.21.15.83:0/3217694852 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660035020 con 0x55e7bf2541b0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.581+0000 7f665bfff640 1 -- 172.21.15.83:0/3217694852 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6660031680 con 0x55e7bf2541b0 2024-09-21T03:17:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.581+0000 7f665bfff640 1 -- 172.21.15.83:0/3217694852 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660004b90 con 0x55e7bf2541b0 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- 172.21.15.83:0/3217694852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 msgr2=0x55e7bf631800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66600044e0 tx=0x7f6660009560 comp rx=0 tx=0).stop 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- 172.21.15.83:0/3217694852 shutdown_connections 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf62cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf25c390 0x55e7bf6257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- 172.21.15.83:0/3217694852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf2541b0 0x55e7bf631800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- 172.21.15.83:0/3217694852 >> 172.21.15.83:0/3217694852 conn(0x55e7bf271460 msgr2=0x55e7bf6250a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- 172.21.15.83:0/3217694852 shutdown_connections 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- 172.21.15.83:0/3217694852 wait complete. 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 -- start start 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 Processor -- start 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf6386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.583+0000 7f667e8fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf6341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f667e8fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66701ae8a0 con 0x55e7bf2541b0 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f667e8fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66701ae1d0 con 0x55e7bf27d070 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f667e8fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66701ab410 con 0x55e7bf25c390 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f667e8fa740 10 client.0 _ll_register_callbacks cb 0x7f6660009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6675921640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf6386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40428/0 (socket says 172.21.15.83:40428) 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 -- 172.21.15.83:0/2370701248 learned_addr learned my addr 172.21.15.83:0/2370701248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6675921640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf6341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 -- 172.21.15.83:0/2370701248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 msgr2=0x55e7bf6341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf6341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 -- 172.21.15.83:0/2370701248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 msgr2=0x55e7bf6386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf6386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 -- 172.21.15.83:0/2370701248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7bf634960 con 0x55e7bf25c390 2024-09-21T03:17:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6675921640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf6341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6676122640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f666c003e50 tx=0x7f666c003f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff6bba21f57ddeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.584+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0052d0 con 0x55e7bf25c390 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f666c005470 con 0x55e7bf25c390 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6660031680 con 0x55e7bf25c390 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f666c033050 con 0x55e7bf25c390 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f667e8fa740 10 client.25123 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e7bf6ac610 con 0x55e7bf25c390 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6660031680 con 0x55e7bf25c390 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f666c024070 con 0x55e7bf25c390 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.585+0000 7f667e8fa740 10 client.25123 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f667e8fa740 10 client.25123 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f667e8fa740 20 client.25123 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf210da0 0x55e7bf6aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f667e8fa740 4 client.25123 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:17:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7bf59cdf0 con 0x55e7bf210da0 2024-09-21T03:17:26.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.586+0000 7f6675921640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf210da0 0x55e7bf6aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:26.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f6675921640 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf210da0 0x55e7bf6aead0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65aebf4ed06a28bc server_cookie=ff337e0d9e6771f5 in_seq=0 out_seq=0 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f6659ffb640 10 client.25123 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f6659ffb640 1 -- 172.21.15.83:0/2370701248 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7bf59cdf0 con 0x55e7bf210da0 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f6659ffb640 10 client.25123 handle_command_reply: tid=0 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f667e8fa740 1 client.25123 shutdown 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 msgr2=0x55e7bf633b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f666c003e50 tx=0x7f666c003f20 comp rx=0 tx=0).stop 2024-09-21T03:17:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 shutdown_connections 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.587+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55e7bf210da0 0x55e7bf6aead0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7bf27d070 0x55e7bf6341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e7bf25c390 0x55e7bf633b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 --2- 172.21.15.83:0/2370701248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7bf2541b0 0x55e7bf6386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 >> 172.21.15.83:0/2370701248 conn(0x55e7bf271460 msgr2=0x55e7bf63b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6659ffb640 0 client.25123 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:17:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 shutdown_connections 2024-09-21T03:17:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 1 -- 172.21.15.83:0/2370701248 wait complete. 2024-09-21T03:17:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f667e8fa740 20 client.25123 trim_cache size 0 max 16384 2024-09-21T03:17:26.599 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:17:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 msgr2=0x7f665c0bd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 0x7f665c0bd020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66680e55d0 tx=0x7f66680a9b60 comp rx=0 tx=0).stop 2024-09-21T03:17:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 msgr2=0x7f667019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667019fcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6668009fd0 tx=0x7f666800ef10 comp rx=0 tx=0).stop 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 shutdown_connections 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f665c0bab10 0x7f665c0bd020 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6670108d20 0x7f667019fcd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66701083d0 0x7f667019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 --2- 172.21.15.83:0/3993269244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6670102ae0 0x7f667019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 >> 172.21.15.83:0/3993269244 conn(0x7f6670079760 msgr2=0x7f667006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 shutdown_connections 2024-09-21T03:17:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:26.588+0000 7f6678386640 1 -- 172.21.15.83:0/3993269244 wait complete. 2024-09-21T03:17:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:26 smithi142 ceph-mon[34512]: pgmap v348: 97 pgs: 97 active+clean; 918 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 78 op/s 2024-09-21T03:17:27.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:17:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:28.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:28 smithi193 ceph-mon[35224]: pgmap v349: 97 pgs: 97 active+clean; 951 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 53 MiB/s wr, 130 op/s 2024-09-21T03:17:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:28 smithi083 ceph-mon[31219]: pgmap v349: 97 pgs: 97 active+clean; 951 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 53 MiB/s wr, 130 op/s 2024-09-21T03:17:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:28 smithi142 ceph-mon[34512]: pgmap v349: 97 pgs: 97 active+clean; 951 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 53 MiB/s wr, 130 op/s 2024-09-21T03:17:29.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 -- 172.21.15.83:0/3822056449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2041083b0 msgr2=0x7fc2041087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 --2- 172.21.15.83:0/3822056449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2041083b0 0x7fc2041087c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009860 tx=0x7fc1f802a340 comp rx=0 tx=0).stop 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 -- 172.21.15.83:0/3822056449 shutdown_connections 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 --2- 172.21.15.83:0/3822056449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc204108d00 0x7fc20410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 --2- 172.21.15.83:0/3822056449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2041083b0 0x7fc2041087c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 --2- 172.21.15.83:0/3822056449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc204102ac0 0x7fc204102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.033+0000 7fc209343640 1 -- 172.21.15.83:0/3822056449 >> 172.21.15.83:0/3822056449 conn(0x7fc204079790 msgr2=0x7fc204079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.034+0000 7fc209343640 1 -- 172.21.15.83:0/3822056449 shutdown_connections 2024-09-21T03:17:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.034+0000 7fc209343640 1 -- 172.21.15.83:0/3822056449 wait complete. 2024-09-21T03:17:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 Processor -- start 2024-09-21T03:17:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 -- start start 2024-09-21T03:17:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2041083b0 0x7fc204198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc204108d00 0x7fc20419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2041129c0 con 0x7fc204108d00 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc204112840 con 0x7fc2041083b0 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.035+0000 7fc209343640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc204112b40 con 0x7fc204102ac0 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40440/0 (socket says 172.21.15.83:40440) 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 -- 172.21.15.83:0/3327326220 learned_addr learned my addr 172.21.15.83:0/3327326220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 -- 172.21.15.83:0/3327326220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2041083b0 msgr2=0x7fc204198930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2041083b0 0x7fc204198930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 -- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc204108d00 msgr2=0x7fc20419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.036+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc204108d00 0x7fc20419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.037+0000 7fc202ffd640 1 -- 172.21.15.83:0/3327326220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20419e200 con 0x7fc204102ac0 2024-09-21T03:17:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.037+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9f0 tx=0x7fc1f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.037+0000 7fc1ebfff640 1 -- 172.21.15.83:0/3327326220 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400cd80 con 0x7fc204102ac0 2024-09-21T03:17:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.038+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20419e4f0 con 0x7fc204102ac0 2024-09-21T03:17:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.038+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2041a2c60 con 0x7fc204102ac0 2024-09-21T03:17:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.037+0000 7fc1ebfff640 1 -- 172.21.15.83:0/3327326220 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40108e0 con 0x7fc204102ac0 2024-09-21T03:17:29.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.038+0000 7fc1ebfff640 1 -- 172.21.15.83:0/3327326220 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4018980 con 0x7fc204102ac0 2024-09-21T03:17:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 Processor -- start 2024-09-21T03:17:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 -- start start 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e488800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e488d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d25e3feea0 con 0x55d25e0b81b0 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.042+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d25e3ee2d0 con 0x55d25e0c0390 2024-09-21T03:17:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d25e3f20e0 con 0x55d25e0e1070 2024-09-21T03:17:29.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40450/0 (socket says 172.21.15.83:40450) 2024-09-21T03:17:29.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 -- 172.21.15.83:0/3107964645 learned_addr learned my addr 172.21.15.83:0/3107964645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:29.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2027fc640 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e488d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 -- 172.21.15.83:0/3107964645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 msgr2=0x55d25e488d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e488d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 -- 172.21.15.83:0/3107964645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 msgr2=0x55d25e488800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e488800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 -- 172.21.15.83:0/3107964645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25e14c650 con 0x55d25e0e1070 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.043+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e470 tx=0x7fc1fc00e870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb213fe61b523945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.044+0000 7fc1e97fa640 1 -- 172.21.15.83:0/3107964645 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004d20 con 0x55d25e0e1070 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.044+0000 7fc1e97fa640 1 -- 172.21.15.83:0/3107964645 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc003e20 con 0x55d25e0e1070 2024-09-21T03:17:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.044+0000 7fc1e97fa640 1 -- 172.21.15.83:0/3107964645 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00c8a0 con 0x55d25e0e1070 2024-09-21T03:17:29.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.044+0000 7fc1ebfff640 1 -- 172.21.15.83:0/3327326220 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc1f4010430 con 0x7fc204102ac0 2024-09-21T03:17:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.045+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3107964645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 msgr2=0x55d25e4882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.045+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e470 tx=0x7fc1fc00e870 comp rx=0 tx=0).stop 2024-09-21T03:17:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.045+0000 7fc1ebfff640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 0x7fc1ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc1ebfff640 1 -- 172.21.15.83:0/3327326220 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc1f40e7910 con 0x7fc204102ac0 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 0x7fc1ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3107964645 shutdown_connections 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e488d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e488800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3107964645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e4882c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.046+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3107964645 >> 172.21.15.83:0/3107964645 conn(0x55d25e0d5460 msgr2=0x55d25e4861b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.047+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3107964645 shutdown_connections 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.047+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 0x7fc1ec0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e470 tx=0x7fc1fc009a00 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.047+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3107964645 wait complete. 2024-09-21T03:17:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.047+0000 7fc20f8b7740 1 -- start start 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.047+0000 7fc20f8b7740 1 Processor -- start 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 0x55d25e493920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e493e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d25e24f700 con 0x55d25e0e1070 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d25e389160 con 0x55d25e0c0390 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d25e496aa0 con 0x55d25e0b81b0 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc20f8b7740 10 client.0 _ll_register_callbacks cb 0x55d25e3a29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55958/0 (socket says 172.21.15.83:55958) 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc2037fe640 1 -- 172.21.15.83:0/2776785529 learned_addr learned my addr 172.21.15.83:0/2776785529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:29.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc202ffd640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 0x55d25e493920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.048+0000 7fc2027fc640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e493e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 -- 172.21.15.83:0/2776785529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 msgr2=0x55d25e493e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e493e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 -- 172.21.15.83:0/2776785529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 msgr2=0x55d25e4943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 -- 172.21.15.83:0/2776785529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25e3b6c90 con 0x55d25e0b81b0 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc2027fc640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e493e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc2037fe640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc1e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.049+0000 7fc202ffd640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 0x55d25e493920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40e50e0 tx=0x7fc1f40b7130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1c4b2d86a66016 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40acf50 con 0x55d25e0b81b0 2024-09-21T03:17:29.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc1f4018d30 con 0x55d25e0b81b0 2024-09-21T03:17:29.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1fc003e20 con 0x55d25e0b81b0 2024-09-21T03:17:29.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1f40acf50 con 0x55d25e0b81b0 2024-09-21T03:17:29.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc20f8b7740 10 client.25135 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:29.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.050+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d25e4a2780 con 0x55d25e0b81b0 2024-09-21T03:17:29.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc003e20 con 0x55d25e0b81b0 2024-09-21T03:17:29.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc1f4010a90 con 0x55d25e0b81b0 2024-09-21T03:17:29.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc20f8b7740 10 client.25135 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:29.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc20f8b7740 10 client.25135 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc20f8b7740 20 client.25135 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e09f9f0 0x55d25e4a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.051+0000 7fc20f8b7740 4 client.25135 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.052+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25e400df0 con 0x55d25e09f9f0 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.052+0000 7fc2027fc640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e09f9f0 0x55d25e4a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.052+0000 7fc1e6ff5640 10 client.25135 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.052+0000 7fc2027fc640 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e09f9f0 0x55d25e4a5c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc844fbf3311e103 server_cookie=b9515a43b2b7aef0 in_seq=0 out_seq=0 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.053+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/2776785529 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d25e400df0 con 0x55d25e09f9f0 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.053+0000 7fc1e6ff5640 10 client.25135 handle_command_reply: tid=0 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.053+0000 7fc20f8b7740 1 client.25135 shutdown 2024-09-21T03:17:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 msgr2=0x55d25e493920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 0x55d25e493920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40e50e0 tx=0x7fc1f40b7130 comp rx=0 tx=0).stop 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 shutdown_connections 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0e1070 0x55d25e4943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e493e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e09f9f0 0x55d25e4a5c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/2776785529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0b81b0 0x55d25e493920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 >> 172.21.15.83:0/2776785529 conn(0x55d25e0d5460 msgr2=0x55d25e48b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc1e6ff5640 0 client.25135 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 shutdown_connections 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 1 -- 172.21.15.83:0/2776785529 wait complete. 2024-09-21T03:17:29.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.054+0000 7fc20f8b7740 20 client.25135 trim_cache size 0 max 16384 2024-09-21T03:17:29.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 Processor -- start 2024-09-21T03:17:29.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 -- start start 2024-09-21T03:17:29.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e485760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e485ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000e8b0 con 0x55d25e0b81b0 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f00104b0 con 0x55d25e0c0390 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc20f8b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000ef80 con 0x55d25e0e1070 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55966/0 (socket says 172.21.15.83:55966) 2024-09-21T03:17:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc2037fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e485ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc202ffd640 1 -- 172.21.15.83:0/1563531887 learned_addr learned my addr 172.21.15.83:0/1563531887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.076+0000 7fc2027fc640 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e485760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 -- 172.21.15.83:0/1563531887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 msgr2=0x55d25e485ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e485ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 -- 172.21.15.83:0/1563531887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 msgr2=0x55d25e495550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 -- 172.21.15.83:0/1563531887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25e491e20 con 0x55d25e0c0390 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc202ffd640 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.077+0000 7fc2027fc640 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e485760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc204079e10 tx=0x7fc1f8030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5348b248467513c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.078+0000 7fc1e8ff9640 1 -- 172.21.15.83:0/1563531887 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f804c070 con 0x55d25e0c0390 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.078+0000 7fc1e8ff9640 1 -- 172.21.15.83:0/1563531887 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8030450 con 0x55d25e0c0390 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.078+0000 7fc1e8ff9640 1 -- 172.21.15.83:0/1563531887 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8037040 con 0x55d25e0c0390 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 -- 172.21.15.83:0/1563531887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 msgr2=0x55d25e485760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e485760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc204079e10 tx=0x7fc1f8030340 comp rx=0 tx=0).stop 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 -- 172.21.15.83:0/1563531887 shutdown_connections 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0e1070 0x55d25e485ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0c0390 0x55d25e485760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/1563531887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e495550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 -- 172.21.15.83:0/1563531887 >> 172.21.15.83:0/1563531887 conn(0x55d25e0d5460 msgr2=0x55d25e494e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 -- 172.21.15.83:0/1563531887 shutdown_connections 2024-09-21T03:17:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.079+0000 7fc20f8b7740 1 -- 172.21.15.83:0/1563531887 wait complete. 2024-09-21T03:17:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.080+0000 7fc20f8b7740 1 -- start start 2024-09-21T03:17:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.080+0000 7fc20f8b7740 1 Processor -- start 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0c0390 0x55d25e4987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 0x55d25e50f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc202ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55980/0 (socket says 172.21.15.83:55980) 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc202ffd640 1 -- 172.21.15.83:0/3053490767 learned_addr learned my addr 172.21.15.83:0/3053490767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc2027fc640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0c0390 0x55d25e4987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000fd20 con 0x55d25e0b81b0 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000f650 con 0x55d25e0e1070 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000c830 con 0x55d25e0c0390 2024-09-21T03:17:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 0x55d25e50f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.081+0000 7fc20f8b7740 10 client.0 _ll_register_callbacks cb 0x7fc1f4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 -- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0c0390 msgr2=0x55d25e4987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0c0390 0x55d25e4987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 -- 172.21.15.83:0/3053490767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 msgr2=0x55d25e4981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25e50fb70 con 0x55d25e0e1070 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc2037fe640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 0x55d25e50f630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009430 tx=0x7fc1fc012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6e3804fe881571 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:29.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.082+0000 7fc1e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.083+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc025070 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.083+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc1fc02e890 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.083+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f8030450 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.084+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1fc01a040 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.084+0000 7fc20f8b7740 10 client.25178 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.084+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d25e510d00 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8030450 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc1fc035020 con 0x55d25e0e1070 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 10 client.25178 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 10 client.25178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:17:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 20 client.25178 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e074da0 0x55d25e513360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 4 client.25178 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d25e3fc8a0 con 0x55d25e074da0 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.085+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e074da0 0x55d25e513360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc1e6ff5640 10 client.25178 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc202ffd640 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e074da0 0x55d25e513360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef8f17d4789268ca server_cookie=cd5c8353b3f513c6 in_seq=0 out_seq=0 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc1e6ff5640 1 -- 172.21.15.83:0/3053490767 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d25e3fc8a0 con 0x55d25e074da0 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc1e6ff5640 10 client.25178 handle_command_reply: tid=0 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 client.25178 shutdown 2024-09-21T03:17:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 msgr2=0x55d25e50f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 0x55d25e50f630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009430 tx=0x7fc1fc012000 comp rx=0 tx=0).stop 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 shutdown_connections 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d25e0e1070 0x55d25e50f630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d25e074da0 0x55d25e513360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25e0c0390 0x55d25e4987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 --2- 172.21.15.83:0/3053490767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25e0b81b0 0x55d25e4981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc1e6ff5640 0 client.25178 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 >> 172.21.15.83:0/3053490767 conn(0x55d25e0d5460 msgr2=0x55d25e494dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 shutdown_connections 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 1 -- 172.21.15.83:0/3053490767 wait complete. 2024-09-21T03:17:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.086+0000 7fc20f8b7740 20 client.25178 trim_cache size 0 max 16384 2024-09-21T03:17:29.093 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 msgr2=0x7fc1ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 0x7fc1ec0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e470 tx=0x7fc1fc009a00 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 msgr2=0x7fc2041983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9f0 tx=0x7fc1f400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 shutdown_connections 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1ec0b6700 0x7fc1ec0b8c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc204108d00 0x7fc20419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2041083b0 0x7fc204198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 --2- 172.21.15.83:0/3327326220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc204102ac0 0x7fc2041983d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 >> 172.21.15.83:0/3327326220 conn(0x7fc204079790 msgr2=0x7fc2041063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 shutdown_connections 2024-09-21T03:17:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:29.087+0000 7fc209343640 1 -- 172.21.15.83:0/3327326220 wait complete. 2024-09-21T03:17:29.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:17:30.293 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:30.561 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:30 smithi083 ceph-mon[31219]: pgmap v350: 97 pgs: 97 active+clean; 954 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 55 MiB/s wr, 310 op/s 2024-09-21T03:17:30.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:30 smithi193 ceph-mon[35224]: pgmap v350: 97 pgs: 97 active+clean; 954 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 55 MiB/s wr, 310 op/s 2024-09-21T03:17:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:30 smithi142 ceph-mon[34512]: pgmap v350: 97 pgs: 97 active+clean; 954 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 55 MiB/s wr, 310 op/s 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.853+0000 7f41534a7640 1 -- 172.21.15.83:0/2106553989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 msgr2=0x7f414c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 --2- 172.21.15.83:0/2106553989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c1087f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f413c009900 tx=0x7f413c02a410 comp rx=0 tx=0).stop 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 -- 172.21.15.83:0/2106553989 shutdown_connections 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 --2- 172.21.15.83:0/2106553989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c108d30 0x7f414c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 --2- 172.21.15.83:0/2106553989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c1087f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 --2- 172.21.15.83:0/2106553989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c102aa0 0x7f414c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 -- 172.21.15.83:0/2106553989 >> 172.21.15.83:0/2106553989 conn(0x7f414c079750 msgr2=0x7f414c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 -- 172.21.15.83:0/2106553989 shutdown_connections 2024-09-21T03:17:30.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.854+0000 7f41534a7640 1 -- 172.21.15.83:0/2106553989 wait complete. 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 Processor -- start 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 -- start start 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c102aa0 0x7f414c118070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c108d30 0x7f414c112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f414c110a70 con 0x7f414c1083e0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1108f0 con 0x7f414c102aa0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f41534a7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f414c110bf0 con 0x7f414c108d30 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35078/0 (socket says 172.21.15.83:35078) 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.855+0000 7f4150a42640 1 -- 172.21.15.83:0/2246150974 learned_addr learned my addr 172.21.15.83:0/2246150974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4151a44640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c108d30 0x7f414c112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4151243640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c102aa0 0x7f414c118070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 -- 172.21.15.83:0/2246150974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c108d30 msgr2=0x7f414c112870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c108d30 0x7f414c112870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 -- 172.21.15.83:0/2246150974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c102aa0 msgr2=0x7f414c118070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c102aa0 0x7f414c118070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 -- 172.21.15.83:0/2246150974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c112f10 con 0x7f414c1083e0 2024-09-21T03:17:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.856+0000 7f4150a42640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f413c02aa00 tx=0x7f413c030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.857+0000 7f41427fc640 1 -- 172.21.15.83:0/2246150974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c03f030 con 0x7f414c1083e0 2024-09-21T03:17:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.857+0000 7f41427fc640 1 -- 172.21.15.83:0/2246150974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f413c03b460 con 0x7f414c1083e0 2024-09-21T03:17:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.857+0000 7f41427fc640 1 -- 172.21.15.83:0/2246150974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c035070 con 0x7f414c1083e0 2024-09-21T03:17:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.857+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c1b55d0 con 0x7f414c1083e0 2024-09-21T03:17:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.857+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1b5970 con 0x7f414c1083e0 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 Processor -- start 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 -- start start 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1bee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b64ea0 con 0x55fad1826390 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b542d0 con 0x55fad181e1b0 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4159a1b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b580e0 con 0x55fad1847070 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f41427fc640 1 -- 172.21.15.83:0/2246150974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f413c042030 con 0x7f414c1083e0 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4151243640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4151243640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58406/0 (socket says 172.21.15.83:58406) 2024-09-21T03:17:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4151243640 1 -- 172.21.15.83:0/1449683955 learned_addr learned my addr 172.21.15.83:0/1449683955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.861+0000 7f4151a44640 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1bee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 -- 172.21.15.83:0/1449683955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 msgr2=0x55fad1bee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1bee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 -- 172.21.15.83:0/1449683955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 msgr2=0x55fad1bef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 -- 172.21.15.83:0/1449683955 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad18b2650 con 0x55fad181e1b0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f4151243640 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f414800bd10 tx=0x7f41480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a27283018fd35d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f41377fe640 1 -- 172.21.15.83:0/1449683955 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148005220 con 0x55fad181e1b0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f41377fe640 1 -- 172.21.15.83:0/1449683955 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148004c50 con 0x55fad181e1b0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.862+0000 7f41377fe640 1 -- 172.21.15.83:0/1449683955 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480117b0 con 0x55fad181e1b0 2024-09-21T03:17:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.863+0000 7f41427fc640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 0x7f41380b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.863+0000 7f41427fc640 1 -- 172.21.15.83:0/2246150974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f413c107cc0 con 0x7f414c1083e0 2024-09-21T03:17:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.864+0000 7f4151a44640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 0x7f41380b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- 172.21.15.83:0/1449683955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 msgr2=0x55fad1beebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4151a44640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 0x7f41380b8d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4144005fd0 tx=0x7f4144009600 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f414800bd10 tx=0x7f41480107a0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- 172.21.15.83:0/1449683955 shutdown_connections 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1beebe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 --2- 172.21.15.83:0/1449683955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1bee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- 172.21.15.83:0/1449683955 >> 172.21.15.83:0/1449683955 conn(0x55fad183b460 msgr2=0x55fad1bec590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- 172.21.15.83:0/1449683955 shutdown_connections 2024-09-21T03:17:30.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- 172.21.15.83:0/1449683955 wait complete. 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.865+0000 7f4159a1b740 1 -- start start 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 Processor -- start 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1bf9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1c06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bfa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35100/0 (socket says 172.21.15.83:35100) 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4150a42640 1 -- 172.21.15.83:0/2310425829 learned_addr learned my addr 172.21.15.83:0/2310425829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fad19b5700 con 0x55fad1826390 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.866+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fad1aef160 con 0x55fad181e1b0 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1bfce80 con 0x55fad1847070 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151243640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1bf9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1c06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4159a1b740 10 client.0 _ll_register_callbacks cb 0x55fad1b089e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 -- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 msgr2=0x55fad1bf9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1bf9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 -- 172.21.15.83:0/2310425829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 msgr2=0x55fad1bfa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bfa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.867+0000 7f4151a44640 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad1b1cc90 con 0x55fad1847070 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4151a44640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1c06170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144011cc0 tx=0x7f4144004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb8eebccfa21586 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4134ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144004d20 con 0x55fad1847070 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4144005020 con 0x55fad1847070 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4148004c50 con 0x55fad1847070 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4144005220 con 0x55fad1847070 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4159a1b740 10 client.25153 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.868+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fad1c08be0 con 0x55fad1847070 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.869+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148004c50 con 0x55fad1847070 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.869+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f414400f040 con 0x55fad1847070 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.869+0000 7f4159a1b740 10 client.25153 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.869+0000 7f4159a1b740 10 client.25153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.869+0000 7f4159a1b740 20 client.25153 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.870+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad18059f0 0x55fad1c0c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.870+0000 7f4159a1b740 4 client.25153 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.870+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad1b66df0 con 0x55fad18059f0 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.870+0000 7f4151243640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad18059f0 0x55fad1c0c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.871+0000 7f4134ff9640 10 client.25153 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.871+0000 7f4151243640 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad18059f0 0x55fad1c0c5e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bd121ab8eafe2da server_cookie=2227578a012b9239 in_seq=0 out_seq=0 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4134ff9640 1 -- 172.21.15.83:0/2310425829 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fad1b66df0 con 0x55fad18059f0 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4134ff9640 10 client.25153 handle_command_reply: tid=0 2024-09-21T03:17:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 client.25153 shutdown 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 msgr2=0x55fad1c06170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1c06170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144011cc0 tx=0x7f4144004a40 comp rx=0 tx=0).stop 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 shutdown_connections 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad18059f0 0x55fad1c0c5e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1847070 0x55fad1c06170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4134ff9640 0 client.25153 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad1826390 0x55fad1bfa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 --2- 172.21.15.83:0/2310425829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad181e1b0 0x55fad1bf9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 >> 172.21.15.83:0/2310425829 conn(0x55fad183b460 msgr2=0x55fad1bf1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 shutdown_connections 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.872+0000 7f4159a1b740 1 -- 172.21.15.83:0/2310425829 wait complete. 2024-09-21T03:17:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.873+0000 7f4159a1b740 20 client.25153 trim_cache size 0 max 16384 2024-09-21T03:17:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.896+0000 7f4159a1b740 1 Processor -- start 2024-09-21T03:17:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.896+0000 7f4159a1b740 1 -- start start 2024-09-21T03:17:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1bebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1bec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4151243640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4151243640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:48690/0 (socket says 172.21.15.83:48690) 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4150a42640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1bebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4151243640 1 -- 172.21.15.83:0/4180924730 learned_addr learned my addr 172.21.15.83:0/4180924730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1be3c0 con 0x55fad181e1b0 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1bffc0 con 0x55fad1826390 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1bea90 con 0x55fad18059f0 2024-09-21T03:17:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.897+0000 7f4151a44640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1bec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 -- 172.21.15.83:0/4180924730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 msgr2=0x55fad1bec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1bec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 -- 172.21.15.83:0/4180924730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 msgr2=0x55fad1bebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1bebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 -- 172.21.15.83:0/4180924730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad1bf1500 con 0x55fad18059f0 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151a44640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1bec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4151243640 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f414800bc30 tx=0x7f4148005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3bb8a444c1009e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.898+0000 7f4136ffd640 1 -- 172.21.15.83:0/4180924730 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148011440 con 0x55fad18059f0 2024-09-21T03:17:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.899+0000 7f4136ffd640 1 -- 172.21.15.83:0/4180924730 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148011a90 con 0x55fad18059f0 2024-09-21T03:17:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.899+0000 7f4136ffd640 1 -- 172.21.15.83:0/4180924730 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414801b850 con 0x55fad18059f0 2024-09-21T03:17:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 msgr2=0x55fad1bfb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f414800bc30 tx=0x7f4148005780 comp rx=0 tx=0).stop 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 shutdown_connections 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1bec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1bebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 --2- 172.21.15.83:0/4180924730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1bfb9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 >> 172.21.15.83:0/4180924730 conn(0x55fad183b460 msgr2=0x55fad1bfb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 shutdown_connections 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- 172.21.15.83:0/4180924730 wait complete. 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 -- start start 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.900+0000 7f4159a1b740 1 Processor -- start 2024-09-21T03:17:30.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1c0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1c0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1bf830 con 0x55fad181e1b0 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1bf160 con 0x55fad1826390 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f414c1bc3a0 con 0x55fad18059f0 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4159a1b740 10 client.0 _ll_register_callbacks cb 0x7f4144011970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4150a42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35122/0 (socket says 172.21.15.83:35122) 2024-09-21T03:17:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1c0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4150a42640 1 -- 172.21.15.83:0/3473701777 learned_addr learned my addr 172.21.15.83:0/3473701777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151a44640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1c0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 -- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 msgr2=0x55fad1c0d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1c0d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 -- 172.21.15.83:0/3473701777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 msgr2=0x55fad1c0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.901+0000 7f4151243640 1 -- 172.21.15.83:0/3473701777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad1c0dbc0 con 0x55fad18059f0 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.902+0000 7f4150a42640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.902+0000 7f4151a44640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1c0d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.902+0000 7f4151243640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1c0fb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4148011010 tx=0x7f414800bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0fcbd127e0f2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.902+0000 7f4134ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.902+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148011e50 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.903+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4148011440 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.903+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4148011a90 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.903+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4148011640 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.903+0000 7f4159a1b740 10 client.25159 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.903+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fad1c76d00 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414803e070 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f414802f420 con 0x55fad18059f0 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 10 client.25159 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 10 client.25159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:17:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 20 client.25159 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad1847070 0x55fad1c79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 4 client.25159 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:17:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4148011a90 con 0x55fad1847070 2024-09-21T03:17:30.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.904+0000 7f4150a42640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad1847070 0x55fad1c79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:30.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4150a42640 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad1847070 0x55fad1c79130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61c9d1b68aa8aece server_cookie=b75bfc19129a737f in_seq=0 out_seq=0 2024-09-21T03:17:30.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4134ff9640 10 client.25159 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4134ff9640 1 -- 172.21.15.83:0/3473701777 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4148011a90 con 0x55fad1847070 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4134ff9640 10 client.25159 handle_command_reply: tid=0 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4159a1b740 1 client.25159 shutdown 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 msgr2=0x55fad1c0fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1c0fb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4148011010 tx=0x7f414800bc60 comp rx=0 tx=0).stop 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.917+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 shutdown_connections 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fad1847070 0x55fad1c79130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fad1826390 0x55fad1c0d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fad181e1b0 0x55fad1c0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4159a1b740 1 --2- 172.21.15.83:0/3473701777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad18059f0 0x55fad1c0fb70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 >> 172.21.15.83:0/3473701777 conn(0x55fad183b460 msgr2=0x55fad1bfb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.918+0000 7f4134ff9640 0 client.25159 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:17:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.919+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 shutdown_connections 2024-09-21T03:17:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.919+0000 7f4159a1b740 1 -- 172.21.15.83:0/3473701777 wait complete. 2024-09-21T03:17:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.919+0000 7f4159a1b740 20 client.25159 trim_cache size 0 max 16384 2024-09-21T03:17:30.922 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.920+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 msgr2=0x7f41380b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.920+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 0x7f41380b8d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4144005fd0 tx=0x7f4144009600 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.920+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 msgr2=0x7f414c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.920+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f413c02aa00 tx=0x7f413c030640 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 shutdown_connections 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f41380b6800 0x7f41380b8d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f414c108d30 0x7f414c112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f414c1083e0 0x7f414c112330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 --2- 172.21.15.83:0/2246150974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f414c102aa0 0x7f414c118070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:30.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 >> 172.21.15.83:0/2246150974 conn(0x7f414c079750 msgr2=0x7f414c106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 shutdown_connections 2024-09-21T03:17:30.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:30.921+0000 7f41534a7640 1 -- 172.21.15.83:0/2246150974 wait complete. 2024-09-21T03:17:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 10.0 s, 3257.4 tps, lat 30.609 ms stddev 30.715 2024-09-21T03:17:31.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:17:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:32.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:32 smithi083 ceph-mon[31219]: pgmap v351: 97 pgs: 97 active+clean; 955 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 51 MiB/s wr, 340 op/s 2024-09-21T03:17:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:32 smithi193 ceph-mon[35224]: pgmap v351: 97 pgs: 97 active+clean; 955 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 51 MiB/s wr, 340 op/s 2024-09-21T03:17:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:32 smithi142 ceph-mon[34512]: pgmap v351: 97 pgs: 97 active+clean; 955 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 51 MiB/s wr, 340 op/s 2024-09-21T03:17:32.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 -- 172.21.15.83:0/467278939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160409a960 msgr2=0x7f160409ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 --2- 172.21.15.83:0/467278939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160409a960 0x7f160409ad70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1600009900 tx=0x7f160002a410 comp rx=0 tx=0).stop 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 -- 172.21.15.83:0/467278939 shutdown_connections 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 --2- 172.21.15.83:0/467278939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 0x7f16040a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 --2- 172.21.15.83:0/467278939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160409a960 0x7f160409ad70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 --2- 172.21.15.83:0/467278939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1604095020 0x7f1604095430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 -- 172.21.15.83:0/467278939 >> 172.21.15.83:0/467278939 conn(0x7f160400cd70 msgr2=0x7f160400d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 -- 172.21.15.83:0/467278939 shutdown_connections 2024-09-21T03:17:32.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.693+0000 7f160b59e640 1 -- 172.21.15.83:0/467278939 wait complete. 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.694+0000 7f160b59e640 1 Processor -- start 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.694+0000 7f160b59e640 1 -- start start 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 0x7f160413ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35146/0 (socket says 172.21.15.83:35146) 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 0x7f1604142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160a59c640 1 -- 172.21.15.83:0/1555672268 learned_addr learned my addr 172.21.15.83:0/1555672268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:32.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16040a4f30 con 0x7f1604095020 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16040a4db0 con 0x7f160409b2b0 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16040a50b0 con 0x7f160409a960 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.695+0000 7f1609d9b640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 0x7f160413ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f160ad9d640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 0x7f1604142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 -- 172.21.15.83:0/1555672268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 msgr2=0x7f1604142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 0x7f1604142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 -- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 msgr2=0x7f160413c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041428a0 con 0x7f160409a960 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f160a59c640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f1609d9b640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 0x7f160413ced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f160002a930 tx=0x7f1600031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.696+0000 7f15fb7fe640 1 -- 172.21.15.83:0/1555672268 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600035070 con 0x7f160409a960 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.697+0000 7f15fb7fe640 1 -- 172.21.15.83:0/1555672268 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1600041e20 con 0x7f160409a960 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.697+0000 7f15fb7fe640 1 -- 172.21.15.83:0/1555672268 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600049630 con 0x7f160409a960 2024-09-21T03:17:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.697+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1604142b30 con 0x7f160409a960 2024-09-21T03:17:32.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.697+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1604004780 con 0x7f160409a960 2024-09-21T03:17:32.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.700+0000 7f16173dc740 1 Processor -- start 2024-09-21T03:17:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 -- start start 2024-09-21T03:17:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fd79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fd7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5f4dea0 con 0x55c6a5c30070 2024-09-21T03:17:32.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5f3d2d0 con 0x55c6a5c0f390 2024-09-21T03:17:32.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f16173dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5f410e0 con 0x55c6a5c071b0 2024-09-21T03:17:32.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35152/0 (socket says 172.21.15.83:35152) 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 -- 172.21.15.83:0/107651814 learned_addr learned my addr 172.21.15.83:0/107651814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160a59c640 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fd79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 -- 172.21.15.83:0/107651814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 msgr2=0x55c6a5fd79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fd79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 -- 172.21.15.83:0/107651814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 msgr2=0x55c6a5fd7ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fd7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.701+0000 7f160ad9d640 1 -- 172.21.15.83:0/107651814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a5c9b650 con 0x55c6a5c30070 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.702+0000 7f160ad9d640 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f160c0524b0 tx=0x7f160c064940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e03c5c73b78ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.702+0000 7f15f8ff9640 1 -- 172.21.15.83:0/107651814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c06ad40 con 0x55c6a5c30070 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.702+0000 7f15f8ff9640 1 -- 172.21.15.83:0/107651814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160c07e830 con 0x55c6a5c30070 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.702+0000 7f15f8ff9640 1 -- 172.21.15.83:0/107651814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c0753e0 con 0x55c6a5c30070 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.703+0000 7f15fb7fe640 1 -- 172.21.15.83:0/1555672268 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f16000038a0 con 0x7f160409a960 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.703+0000 7f16173dc740 1 -- 172.21.15.83:0/107651814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 msgr2=0x55c6a5fd7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.703+0000 7f16173dc740 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f160c0524b0 tx=0x7f160c064940 comp rx=0 tx=0).stop 2024-09-21T03:17:32.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f15fb7fe640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 0x7f15f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f160ad9d640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 0x7f15f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f16173dc740 1 -- 172.21.15.83:0/107651814 shutdown_connections 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f16173dc740 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fd7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f16173dc740 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fd79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f16173dc740 1 --2- 172.21.15.83:0/107651814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fd7460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f16173dc740 1 -- 172.21.15.83:0/107651814 >> 172.21.15.83:0/107651814 conn(0x55c6a5c24460 msgr2=0x55c6a5fd5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.704+0000 7f15fb7fe640 1 -- 172.21.15.83:0/1555672268 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f16001102c0 con 0x7f160409a960 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.705+0000 7f160ad9d640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 0x7f15f40b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f160c0516c0 tx=0x7f160c069e10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.706+0000 7f16173dc740 1 -- 172.21.15.83:0/107651814 shutdown_connections 2024-09-21T03:17:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.707+0000 7f16173dc740 1 -- 172.21.15.83:0/107651814 wait complete. 2024-09-21T03:17:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.707+0000 7f16173dc740 1 -- start start 2024-09-21T03:17:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.707+0000 7f16173dc740 1 Processor -- start 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fe3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5d9e700 con 0x55c6a5c30070 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5ed8160 con 0x55c6a5c071b0 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5fe5c40 con 0x55c6a5c0f390 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f16173dc740 10 client.0 _ll_register_callbacks cb 0x55c6a5ef19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58462/0 (socket says 172.21.15.83:58462) 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 -- 172.21.15.83:0/3869554856 learned_addr learned my addr 172.21.15.83:0/3869554856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160ad9d640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fe3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 -- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 msgr2=0x55c6a5fe2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 -- 172.21.15.83:0/3869554856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 msgr2=0x55c6a5fe3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fe3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 -- 172.21.15.83:0/3869554856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a5f05c90 con 0x55c6a5c071b0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160ad9d640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fe3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f160a59c640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009fd0 tx=0x7f15fc011550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4507d1757472f9df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.708+0000 7f15f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00f460 con 0x55c6a5c071b0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f15fc00f600 con 0x55c6a5c071b0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f160c07e830 con 0x55c6a5c071b0 2024-09-21T03:17:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15fc00f800 con 0x55c6a5c071b0 2024-09-21T03:17:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f16173dc740 10 client.25205 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.709+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6a5ff1900 con 0x55c6a5c071b0 2024-09-21T03:17:32.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.710+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160c07e830 con 0x55c6a5c071b0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f15fc01caa0 con 0x55c6a5c071b0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 10 client.25205 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 10 client.25205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 20 client.25205 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5bee9f0 0x55c6a5ff4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 4 client.25205 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a5f4fdf0 con 0x55c6a5bee9f0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5bee9f0 0x55c6a5ff4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f15f1ffb640 10 client.25205 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.711+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5bee9f0 0x55c6a5ff4ed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40d584a2009885ee server_cookie=f14a8390e0e1048e in_seq=0 out_seq=0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3869554856 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6a5f4fdf0 con 0x55c6a5bee9f0 2024-09-21T03:17:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f15f1ffb640 10 client.25205 handle_command_reply: tid=0 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 client.25205 shutdown 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 msgr2=0x55c6a5fe2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009fd0 tx=0x7f15fc011550 comp rx=0 tx=0).stop 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 shutdown_connections 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5bee9f0 0x55c6a5ff4ed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c30070 0x55c6a5fe3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f15f1ffb640 0 client.25205 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.712+0000 7f16173dc740 1 --2- 172.21.15.83:0/3869554856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe2ab0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.713+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 >> 172.21.15.83:0/3869554856 conn(0x55c6a5c24460 msgr2=0x55c6a5fd8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.713+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 shutdown_connections 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.713+0000 7f16173dc740 1 -- 172.21.15.83:0/3869554856 wait complete. 2024-09-21T03:17:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.713+0000 7f16173dc740 20 client.25205 trim_cache size 0 max 16384 2024-09-21T03:17:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.731+0000 7f16173dc740 1 Processor -- start 2024-09-21T03:17:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 -- start start 2024-09-21T03:17:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1604150880 con 0x55c6a5c0f390 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1604152480 con 0x55c6a5bee9f0 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f16173dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1604150f50 con 0x55c6a5c071b0 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58478/0 (socket says 172.21.15.83:58478) 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 -- 172.21.15.83:0/261324032 learned_addr learned my addr 172.21.15.83:0/261324032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f1609d9b640 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160ad9d640 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 -- 172.21.15.83:0/261324032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 msgr2=0x55c6a5fe4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 -- 172.21.15.83:0/261324032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c0f390 msgr2=0x55c6a5fe4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.732+0000 7f160a59c640 1 -- 172.21.15.83:0/261324032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a5fe0fb0 con 0x55c6a5bee9f0 2024-09-21T03:17:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.733+0000 7f160a59c640 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00f720 tx=0x7f15fc00f750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a065d084772483df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.733+0000 7f15f3fff640 1 -- 172.21.15.83:0/261324032 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc029070 con 0x55c6a5bee9f0 2024-09-21T03:17:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.733+0000 7f15f3fff640 1 -- 172.21.15.83:0/261324032 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc00fe50 con 0x55c6a5bee9f0 2024-09-21T03:17:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.733+0000 7f15f3fff640 1 -- 172.21.15.83:0/261324032 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0048e0 con 0x55c6a5bee9f0 2024-09-21T03:17:32.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.734+0000 7f16173dc740 1 -- 172.21.15.83:0/261324032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 msgr2=0x55c6a5fd8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.734+0000 7f16173dc740 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00f720 tx=0x7f15fc00f750 comp rx=0 tx=0).stop 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 -- 172.21.15.83:0/261324032 shutdown_connections 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c0f390 0x55c6a5fe4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5c071b0 0x55c6a5fe4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 --2- 172.21.15.83:0/261324032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fd8e50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 -- 172.21.15.83:0/261324032 >> 172.21.15.83:0/261324032 conn(0x55c6a5c24460 msgr2=0x55c6a5fe3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 -- 172.21.15.83:0/261324032 shutdown_connections 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.735+0000 7f16173dc740 1 -- 172.21.15.83:0/261324032 wait complete. 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 -- start start 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 Processor -- start 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fde790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1604151cf0 con 0x55c6a5c071b0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1604151620 con 0x55c6a5c0f390 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f160414e860 con 0x55c6a5bee9f0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f16173dc740 10 client.0 _ll_register_callbacks cb 0x7f15fc009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f1609d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f1609d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35188/0 (socket says 172.21.15.83:35188) 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f160ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fde790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.736+0000 7f1609d9b640 1 -- 172.21.15.83:0/3009582121 learned_addr learned my addr 172.21.15.83:0/3009582121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f160a59c640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 -- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5bee9f0 msgr2=0x55c6a5fddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 -- 172.21.15.83:0/3009582121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 msgr2=0x55c6a5fde790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fde790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 -- 172.21.15.83:0/3009582121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a605e8e0 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f160ad9d640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fde790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f1609d9b640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f16000d5e30 tx=0x7f16000d5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f27553b4203f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f15f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16000d6830 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f16000df6c0 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc00fe50 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f16000dac10 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f16173dc740 10 client.15684 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.737+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6a605fb90 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc00fe50 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f16000d5390 con 0x55c6a5c071b0 2024-09-21T03:17:32.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 10 client.15684 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 20 client.15684 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5c30070 0x55c6a6062300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 4 client.15684 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6a5f4b8a0 con 0x55c6a5c30070 2024-09-21T03:17:32.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.738+0000 7f160a59c640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5c30070 0x55c6a6062300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:32.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.739+0000 7f15f1ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.739+0000 7f160a59c640 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5c30070 0x55c6a6062300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fe519b3f462f996 server_cookie=4f7a499f8d073eba in_seq=0 out_seq=0 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.739+0000 7f15f1ffb640 1 -- 172.21.15.83:0/3009582121 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6a5f4b8a0 con 0x55c6a5c30070 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.739+0000 7f15f1ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.739+0000 7f16173dc740 1 client.15684 shutdown 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 msgr2=0x55c6a5fde250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f16000d5e30 tx=0x7f16000d5960 comp rx=0 tx=0).stop 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 shutdown_connections 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c6a5c30070 0x55c6a6062300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f15f1ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6a5c0f390 0x55c6a5fde790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6a5c071b0 0x55c6a5fde250 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 --2- 172.21.15.83:0/3009582121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6a5bee9f0 0x55c6a5fddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.740+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 >> 172.21.15.83:0/3009582121 conn(0x55c6a5c24460 msgr2=0x55c6a5fd8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 shutdown_connections 2024-09-21T03:17:32.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f16173dc740 1 -- 172.21.15.83:0/3009582121 wait complete. 2024-09-21T03:17:32.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f16173dc740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T03:17:32.746 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 msgr2=0x7f15f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 0x7f15f40b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f160c0516c0 tx=0x7f160c069e10 comp rx=0 tx=0).stop 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 msgr2=0x7f160413ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.741+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 0x7f160413ced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f160002a930 tx=0x7f1600031a20 comp rx=0 tx=0).stop 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 shutdown_connections 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f15f40b6680 0x7f15f40b8b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f160409b2b0 0x7f1604142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f160409a960 0x7f160413ced0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 --2- 172.21.15.83:0/1555672268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1604095020 0x7f160413c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:32.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 >> 172.21.15.83:0/1555672268 conn(0x7f160400cd70 msgr2=0x7f16040989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:32.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 shutdown_connections 2024-09-21T03:17:32.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:32.742+0000 7f160b59e640 1 -- 172.21.15.83:0/1555672268 wait complete. 2024-09-21T03:17:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:33 smithi083 ceph-mon[31219]: pgmap v352: 97 pgs: 97 active+clean; 962 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 48 MiB/s wr, 527 op/s 2024-09-21T03:17:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:33 smithi142 ceph-mon[34512]: pgmap v352: 97 pgs: 97 active+clean; 962 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 48 MiB/s wr, 527 op/s 2024-09-21T03:17:33.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:33 smithi193 ceph-mon[35224]: pgmap v352: 97 pgs: 97 active+clean; 962 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 48 MiB/s wr, 527 op/s 2024-09-21T03:17:34.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:17:34.685 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:35.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.308+0000 7fdd484cc640 1 -- 172.21.15.83:0/1673026732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40102a80 msgr2=0x7fdd40102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.308+0000 7fdd484cc640 1 --2- 172.21.15.83:0/1673026732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40102a80 0x7fdd40102e90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009900 tx=0x7fdd3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 -- 172.21.15.83:0/1673026732 shutdown_connections 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 --2- 172.21.15.83:0/1673026732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40108d10 0x7fdd4010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 --2- 172.21.15.83:0/1673026732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd401083c0 0x7fdd401087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 --2- 172.21.15.83:0/1673026732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40102a80 0x7fdd40102e90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 -- 172.21.15.83:0/1673026732 >> 172.21.15.83:0/1673026732 conn(0x7fdd40079740 msgr2=0x7fdd40079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 -- 172.21.15.83:0/1673026732 shutdown_connections 2024-09-21T03:17:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.309+0000 7fdd484cc640 1 -- 172.21.15.83:0/1673026732 wait complete. 2024-09-21T03:17:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.311+0000 7fdd484cc640 1 Processor -- start 2024-09-21T03:17:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.311+0000 7fdd484cc640 1 -- start start 2024-09-21T03:17:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40102a80 0x7fdd4019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd40108d10 0x7fdd401a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40111870 con 0x7fdd40102a80 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401116f0 con 0x7fdd401083c0 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd484cc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401119f0 con 0x7fdd40108d10 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd46268640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40102a80 0x7fdd4019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58496/0 (socket says 172.21.15.83:58496) 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 -- 172.21.15.83:0/2877910653 learned_addr learned my addr 172.21.15.83:0/2877910653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 -- 172.21.15.83:0/2877910653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd40108d10 msgr2=0x7fdd401a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd46a69640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd40108d10 0x7fdd401a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd40108d10 0x7fdd401a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 -- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40102a80 msgr2=0x7fdd4019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40102a80 0x7fdd4019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.312+0000 7fdd45a67640 1 -- 172.21.15.83:0/2877910653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd401a25f0 con 0x7fdd401083c0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000e9f0 tx=0x7fdd3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd377fe640 1 -- 172.21.15.83:0/2877910653 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3000ccc0 con 0x7fdd401083c0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd377fe640 1 -- 172.21.15.83:0/2877910653 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd300054d0 con 0x7fdd401083c0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd401a28e0 con 0x7fdd401083c0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd377fe640 1 -- 172.21.15.83:0/2877910653 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30010430 con 0x7fdd401083c0 2024-09-21T03:17:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.313+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd401a9040 con 0x7fdd401083c0 2024-09-21T03:17:35.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.317+0000 7fdd377fe640 1 -- 172.21.15.83:0/2877910653 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdd300105d0 con 0x7fdd401083c0 2024-09-21T03:17:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.317+0000 7fdd4ea40740 1 Processor -- start 2024-09-21T03:17:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 -- start start 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd377fe640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 0x7fdd2c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 0x563119699800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x563119699d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56311960fea0 con 0x5631192f2070 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5631195ff2d0 con 0x5631192c91b0 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631196030e0 con 0x5631192d1390 2024-09-21T03:17:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd46268640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd46268640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35224/0 (socket says 172.21.15.83:35224) 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd46a69640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 0x7fdd2c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd46268640 1 -- 172.21.15.83:0/2048793302 learned_addr learned my addr 172.21.15.83:0/2048793302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 0x563119699800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd377fe640 1 -- 172.21.15.83:0/2877910653 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdd300e7b00 con 0x7fdd401083c0 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd46a69640 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x563119699d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 -- 172.21.15.83:0/2048793302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 msgr2=0x563119699d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x563119699d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 -- 172.21.15.83:0/2048793302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 msgr2=0x5631196992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.318+0000 7fdd45a67640 1 -- 172.21.15.83:0/2048793302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311935d650 con 0x5631192c91b0 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.319+0000 7fdd46a69640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 0x7fdd2c0b8c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009fd0 tx=0x7fdd38009dc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.319+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 0x563119699800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd300e5270 tx=0x7fdd30007ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef7d0690349c3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.320+0000 7fdd34ff9640 1 -- 172.21.15.83:0/2048793302 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300c96b0 con 0x5631192c91b0 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.320+0000 7fdd34ff9640 1 -- 172.21.15.83:0/2048793302 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd300aa050 con 0x5631192c91b0 2024-09-21T03:17:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.320+0000 7fdd34ff9640 1 -- 172.21.15.83:0/2048793302 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300b33d0 con 0x5631192c91b0 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2048793302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 msgr2=0x563119699800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 0x563119699800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd300e5270 tx=0x7fdd30007ca0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2048793302 shutdown_connections 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x563119699d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192c91b0 0x563119699800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2048793302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2048793302 >> 172.21.15.83:0/2048793302 conn(0x5631192e6460 msgr2=0x5631196971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2048793302 shutdown_connections 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.321+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2048793302 wait complete. 2024-09-21T03:17:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.322+0000 7fdd4ea40740 1 -- start start 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.322+0000 7fdd4ea40740 1 Processor -- start 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.322+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.322+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563119460700 con 0x5631192f2070 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:48766/0 (socket says 172.21.15.83:48766) 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 -- 172.21.15.83:0/1197276627 learned_addr learned my addr 172.21.15.83:0/1197276627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd45a67640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56311959a160 con 0x5631192b09f0 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631196a7aa0 con 0x5631192d1390 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46a69640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd4ea40740 10 client.0 _ll_register_callbacks cb 0x5631195b39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 -- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 msgr2=0x5631196b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 -- 172.21.15.83:0/1197276627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 msgr2=0x5631196a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631195c7c90 con 0x5631192d1390 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd46268640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c032f90 tx=0x7fdd3c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06597a199f8cf06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.323+0000 7fdd29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.324+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c035040 con 0x5631192d1390 2024-09-21T03:17:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.324+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd300aa050 con 0x5631192d1390 2024-09-21T03:17:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.324+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdd3c0317a0 con 0x5631192d1390 2024-09-21T03:17:35.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.333+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c004040 con 0x5631192d1390 2024-09-21T03:17:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.335+0000 7fdd4ea40740 10 client.25186 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:35.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.335+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631196b3750 con 0x5631192d1390 2024-09-21T03:17:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c0054e0 con 0x5631192d1390 2024-09-21T03:17:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fdd3c004a60 con 0x5631192d1390 2024-09-21T03:17:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 10 client.25186 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 20 client.25186 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563119285da0 0x5631196b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 4 client.25186 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd300aa050 con 0x563119285da0 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.339+0000 7fdd45a67640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563119285da0 0x5631196b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.340+0000 7fdd45a67640 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563119285da0 0x5631196b70a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7968613281eab4e0 server_cookie=5e68f63f005ad2a6 in_seq=0 out_seq=0 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.341+0000 7fdd29ffb640 10 client.25186 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.342+0000 7fdd29ffb640 1 -- 172.21.15.83:0/1197276627 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdd300aa050 con 0x563119285da0 2024-09-21T03:17:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.342+0000 7fdd29ffb640 10 client.25186 handle_command_reply: tid=0 2024-09-21T03:17:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.344+0000 7fdd4ea40740 1 client.25186 shutdown 2024-09-21T03:17:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.345+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 msgr2=0x5631196a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.345+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c032f90 tx=0x7fdd3c005f00 comp rx=0 tx=0).stop 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 shutdown_connections 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563119285da0 0x5631196b70a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192f2070 0x5631196a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd29ffb640 0 client.25186 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/1197276627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a4aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.346+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 >> 172.21.15.83:0/1197276627 conn(0x5631192e6460 msgr2=0x56311969a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.347+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 shutdown_connections 2024-09-21T03:17:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.347+0000 7fdd4ea40740 1 -- 172.21.15.83:0/1197276627 wait complete. 2024-09-21T03:17:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.347+0000 7fdd4ea40740 20 client.25186 trim_cache size 0 max 16384 2024-09-21T03:17:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 Processor -- start 2024-09-21T03:17:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 -- start start 2024-09-21T03:17:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 0x56311969aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 0x5631196a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401b17e0 con 0x5631192b09f0 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401b33e0 con 0x563119285da0 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.368+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401b1eb0 con 0x5631192d1390 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd46268640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 0x56311969aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd46268640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 0x56311969aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58542/0 (socket says 172.21.15.83:58542) 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd46268640 1 -- 172.21.15.83:0/4128415147 learned_addr learned my addr 172.21.15.83:0/4128415147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd46a69640 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 0x5631196a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 -- 172.21.15.83:0/4128415147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 msgr2=0x5631196a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 -- 172.21.15.83:0/4128415147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 msgr2=0x56311969aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 0x56311969aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.369+0000 7fdd45a67640 1 -- 172.21.15.83:0/4128415147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563119697aa0 con 0x5631192b09f0 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.370+0000 7fdd45a67640 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 0x5631196a1c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd30095000 tx=0x7fdd30005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4058ad1ffa3a7234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.370+0000 7fdd2bfff640 1 -- 172.21.15.83:0/4128415147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300b8310 con 0x5631192b09f0 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.370+0000 7fdd2bfff640 1 -- 172.21.15.83:0/4128415147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd300b33d0 con 0x5631192b09f0 2024-09-21T03:17:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.370+0000 7fdd2bfff640 1 -- 172.21.15.83:0/4128415147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300b7e80 con 0x5631192b09f0 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 -- 172.21.15.83:0/4128415147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 msgr2=0x5631196a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 0x5631196a1c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd30095000 tx=0x7fdd30005460 comp rx=0 tx=0).stop 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 -- 172.21.15.83:0/4128415147 shutdown_connections 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631192d1390 0x5631196a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192b09f0 0x5631196a1c90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.371+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/4128415147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563119285da0 0x56311969aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.372+0000 7fdd4ea40740 1 -- 172.21.15.83:0/4128415147 >> 172.21.15.83:0/4128415147 conn(0x5631192e6460 msgr2=0x56311969a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.372+0000 7fdd4ea40740 1 -- 172.21.15.83:0/4128415147 shutdown_connections 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.372+0000 7fdd4ea40740 1 -- 172.21.15.83:0/4128415147 wait complete. 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.373+0000 7fdd4ea40740 1 -- start start 2024-09-21T03:17:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.373+0000 7fdd4ea40740 1 Processor -- start 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563119285da0 0x5631196a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 0x5631197292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401b2c50 con 0x5631192d1390 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401b2580 con 0x5631192b09f0 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401af7c0 con 0x563119285da0 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd4ea40740 10 client.0 _ll_register_callbacks cb 0x7fdd3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd46a69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 0x5631197292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd46a69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 0x5631197292b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35248/0 (socket says 172.21.15.83:35248) 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd46a69640 1 -- 172.21.15.83:0/2130587075 learned_addr learned my addr 172.21.15.83:0/2130587075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.374+0000 7fdd46268640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563119285da0 0x5631196a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 -- 172.21.15.83:0/2130587075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563119285da0 msgr2=0x5631196a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563119285da0 0x5631196a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 -- 172.21.15.83:0/2130587075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 msgr2=0x5631197292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 0x5631197292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 -- 172.21.15.83:0/2130587075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631197297f0 con 0x5631192b09f0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd45a67640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196a95e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd30095000 tx=0x7fdd300aa050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b82cb53e20e284 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300b8310 con 0x5631192b09f0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdd300aab60 con 0x5631192b09f0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.375+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd300b33d0 con 0x5631192b09f0 2024-09-21T03:17:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd300046c0 con 0x5631192b09f0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 10 client.25229 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56311972a8c0 con 0x5631192b09f0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd300b33d0 con 0x5631192b09f0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fdd300d2020 con 0x5631192b09f0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 10 client.25229 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 10 client.25229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 20 client.25229 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631192f2070 0x56311972caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 4 client.25229 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.376+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563119611df0 con 0x5631192f2070 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.377+0000 7fdd46268640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631192f2070 0x56311972caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.377+0000 7fdd46268640 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631192f2070 0x56311972caf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bd98dbd8098c3b server_cookie=bc399b51e324a9f7 in_seq=0 out_seq=0 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.377+0000 7fdd29ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd29ffb640 1 -- 172.21.15.83:0/2130587075 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563119611df0 con 0x5631192f2070 2024-09-21T03:17:35.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd29ffb640 10 client.25229 handle_command_reply: tid=0 2024-09-21T03:17:35.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 client.25229 shutdown 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 msgr2=0x5631196a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196a95e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd30095000 tx=0x7fdd300aa050 comp rx=0 tx=0).stop 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 shutdown_connections 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631192f2070 0x56311972caf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631192d1390 0x5631197292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd29ffb640 0 client.25229 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631192b09f0 0x5631196a95e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 --2- 172.21.15.83:0/2130587075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563119285da0 0x5631196a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 >> 172.21.15.83:0/2130587075 conn(0x5631192e6460 msgr2=0x5631196a5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 shutdown_connections 2024-09-21T03:17:35.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 1 -- 172.21.15.83:0/2130587075 wait complete. 2024-09-21T03:17:35.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.378+0000 7fdd4ea40740 20 client.25229 trim_cache size 0 max 16384 2024-09-21T03:17:35.384 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 msgr2=0x7fdd2c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 0x7fdd2c0b8c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009fd0 tx=0x7fdd38009dc0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 msgr2=0x7fdd4019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000e9f0 tx=0x7fdd3000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 shutdown_connections 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdd2c0b6780 0x7fdd2c0b8c90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd40108d10 0x7fdd401a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd401083c0 0x7fdd4019cc20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 --2- 172.21.15.83:0/2877910653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40102a80 0x7fdd4019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 >> 172.21.15.83:0/2877910653 conn(0x7fdd40079740 msgr2=0x7fdd40100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 shutdown_connections 2024-09-21T03:17:35.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:35.379+0000 7fdd484cc640 1 -- 172.21.15.83:0/2877910653 wait complete. 2024-09-21T03:17:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 15.0 s, 3341.6 tps, lat 29.998 ms stddev 31.265 2024-09-21T03:17:36.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:36 smithi193 ceph-mon[35224]: pgmap v353: 97 pgs: 97 active+clean; 976 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 598 op/s 2024-09-21T03:17:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:36 smithi083 ceph-mon[31219]: pgmap v353: 97 pgs: 97 active+clean; 976 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 598 op/s 2024-09-21T03:17:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:36 smithi142 ceph-mon[34512]: pgmap v353: 97 pgs: 97 active+clean; 976 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 20 MiB/s wr, 598 op/s 2024-09-21T03:17:37.090 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:17:37.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:17:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 -- 172.21.15.83:0/842184024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e01021f0 msgr2=0x7f04e0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 --2- 172.21.15.83:0/842184024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e01021f0 0x7f04e0102600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009900 tx=0x7f04dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 -- 172.21.15.83:0/842184024 shutdown_connections 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 --2- 172.21.15.83:0/842184024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0108480 0x7f04e010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 --2- 172.21.15.83:0/842184024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e0107b30 0x7f04e0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 --2- 172.21.15.83:0/842184024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e01021f0 0x7f04e0102600 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.046+0000 7f04e7732640 1 -- 172.21.15.83:0/842184024 >> 172.21.15.83:0/842184024 conn(0x7f04e0077c60 msgr2=0x7f04e00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.047+0000 7f04e7732640 1 -- 172.21.15.83:0/842184024 shutdown_connections 2024-09-21T03:17:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.047+0000 7f04e7732640 1 -- 172.21.15.83:0/842184024 wait complete. 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.047+0000 7f04e7732640 1 Processor -- start 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 -- start start 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 0x7f04e01a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 0x7f04e01a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e4ccd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e4ccd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:48798/0 (socket says 172.21.15.83:48798) 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 0x7f04e01a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35268/0 (socket says 172.21.15.83:35268) 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e54ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e54ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58558/0 (socket says 172.21.15.83:58558) 2024-09-21T03:17:38.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01118c0 con 0x7f04e0108480 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0111740 con 0x7f04e01021f0 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e4ccd640 1 -- 172.21.15.83:0/1422795110 learned_addr learned my addr 172.21.15.83:0/1422795110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.048+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0111a40 con 0x7f04e0107b30 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 -- 172.21.15.83:0/1422795110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 msgr2=0x7f04e01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 -- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 msgr2=0x7f04e01a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 0x7f04e01a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 -- 172.21.15.83:0/1422795110 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e01a6a30 con 0x7f04e0107b30 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e54ce640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04d000ea80 tx=0x7f04d000ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04d67fc640 1 -- 172.21.15.83:0/1422795110 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cda0 con 0x7f04e0107b30 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04d67fc640 1 -- 172.21.15.83:0/1422795110 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00035f0 con 0x7f04e0107b30 2024-09-21T03:17:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e01a6d20 con 0x7f04e0107b30 2024-09-21T03:17:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04d67fc640 1 -- 172.21.15.83:0/1422795110 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0016670 con 0x7f04e0107b30 2024-09-21T03:17:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.049+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01ad480 con 0x7f04e0107b30 2024-09-21T03:17:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.053+0000 7f04edca6740 1 Processor -- start 2024-09-21T03:17:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.055+0000 7f04edca6740 1 -- start start 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b00e490 0x55606b3de970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55606b354e60 con 0x55606b037350 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55606b344290 con 0x55606b016670 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04edca6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55606b3480a0 con 0x55606b00e490 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04d67fc640 1 -- 172.21.15.83:0/1422795110 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f04d001d060 con 0x7f04e0107b30 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35274/0 (socket says 172.21.15.83:35274) 2024-09-21T03:17:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.056+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2133904176 learned_addr learned my addr 172.21.15.83:0/2133904176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2133904176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b00e490 msgr2=0x55606b3de970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b00e490 0x55606b3de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2133904176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 msgr2=0x55606b3deeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2133904176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606b0a2930 con 0x55606b037350 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.057+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009fd0 tx=0x7f04d80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a5b11d9f89b21f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.058+0000 7f04d67fc640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 0x7f04cc0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.058+0000 7f04d67fc640 1 -- 172.21.15.83:0/1422795110 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04d001b070 con 0x7f04e0107b30 2024-09-21T03:17:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.062+0000 7f04cb7fe640 1 -- 172.21.15.83:0/2133904176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8004d20 con 0x55606b037350 2024-09-21T03:17:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.062+0000 7f04cb7fe640 1 -- 172.21.15.83:0/2133904176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d8004050 con 0x55606b037350 2024-09-21T03:17:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.062+0000 7f04e54ce640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 0x7f04cc0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 -- 172.21.15.83:0/2133904176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 msgr2=0x55606b3de430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009fd0 tx=0x7f04d80118d0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 -- 172.21.15.83:0/2133904176 shutdown_connections 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04e54ce640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 0x7f04cc0b8c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04dc031440 tx=0x7f04dc032f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3deeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b00e490 0x55606b3de970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 --2- 172.21.15.83:0/2133904176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b037350 0x55606b3de430 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.063+0000 7f04edca6740 1 -- 172.21.15.83:0/2133904176 >> 172.21.15.83:0/2133904176 conn(0x55606b02b740 msgr2=0x55606b3dc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.064+0000 7f04edca6740 1 -- 172.21.15.83:0/2133904176 shutdown_connections 2024-09-21T03:17:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.064+0000 7f04edca6740 1 -- 172.21.15.83:0/2133904176 wait complete. 2024-09-21T03:17:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.064+0000 7f04edca6740 1 -- start start 2024-09-21T03:17:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.064+0000 7f04edca6740 1 Processor -- start 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 0x55606b3e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04e4ccd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55606b1a5890 con 0x55606b016670 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04e4ccd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35286/0 (socket says 172.21.15.83:35286) 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55606b2ded40 con 0x55606b00e490 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04e4ccd640 1 -- 172.21.15.83:0/3335488594 learned_addr learned my addr 172.21.15.83:0/3335488594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55606b3ecc60 con 0x55606b037350 2024-09-21T03:17:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.067+0000 7f04edca6740 10 client.0 _ll_register_callbacks cb 0x55606b342c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.070+0000 7f04e54ce640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 0x55606b3e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.070+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 -- 172.21.15.83:0/3335488594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 msgr2=0x55606b3f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 -- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 msgr2=0x55606b3e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 0x55606b3e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 -- 172.21.15.83:0/3335488594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606b3f6560 con 0x55606b016670 2024-09-21T03:17:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e54ce640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 0x55606b3e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:17:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aa190 tx=0x7f04d00acdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da7013867e8a0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.071+0000 7f04c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:17:38.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.072+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d001a030 con 0x55606b016670 2024-09-21T03:17:38.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.072+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04d0098190 con 0x55606b016670 2024-09-21T03:17:38.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.072+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d8004050 con 0x55606b016670 2024-09-21T03:17:38.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.073+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f04d00e4ca0 con 0x55606b016670 2024-09-21T03:17:38.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.073+0000 7f04edca6740 10 client.15714 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:38.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.073+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55606b3f8ee0 con 0x55606b016670 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00aca30 con 0x55606b016670 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f04d000b040 con 0x55606b016670 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 10 client.15714 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 20 client.15714 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606aff5cd0 0x55606b3fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 4 client.15714 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04d8004050 con 0x55606aff5cd0 2024-09-21T03:17:38.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.074+0000 7f04e54ce640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606aff5cd0 0x55606b3fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.075+0000 7f04c8ff9640 10 client.15714 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:38.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.075+0000 7f04e54ce640 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606aff5cd0 0x55606b3fc400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e810ff1e4779d8d9 server_cookie=59e5c0ed8e24aff6 in_seq=0 out_seq=0 2024-09-21T03:17:38.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.076+0000 7f04c8ff9640 1 -- 172.21.15.83:0/3335488594 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f04d8004050 con 0x55606aff5cd0 2024-09-21T03:17:38.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.076+0000 7f04c8ff9640 10 client.15714 handle_command_reply: tid=0 2024-09-21T03:17:38.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.076+0000 7f04edca6740 1 client.15714 shutdown 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 msgr2=0x55606b3ea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aa190 tx=0x7f04d00acdb0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 shutdown_connections 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606aff5cd0 0x55606b3fc400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b016670 0x55606b3ea510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 --2- 172.21.15.83:0/3335488594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b00e490 0x55606b3e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 >> 172.21.15.83:0/3335488594 conn(0x55606b02b740 msgr2=0x55606b3e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.077+0000 7f04c8ff9640 0 client.15714 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:38.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.079+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 shutdown_connections 2024-09-21T03:17:38.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.079+0000 7f04edca6740 1 -- 172.21.15.83:0/3335488594 wait complete. 2024-09-21T03:17:38.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.079+0000 7f04edca6740 20 client.15714 trim_cache size 0 max 16384 2024-09-21T03:17:38.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.102+0000 7f04edca6740 1 Processor -- start 2024-09-21T03:17:38.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.102+0000 7f04edca6740 1 -- start start 2024-09-21T03:17:38.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3e5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b5bb0 con 0x55606b00e490 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b77b0 con 0x55606b016670 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04edca6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b6280 con 0x55606b037350 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:48820/0 (socket says 172.21.15.83:48820) 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2558029161 learned_addr learned my addr 172.21.15.83:0/2558029161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.103+0000 7f04e54ce640 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2558029161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 msgr2=0x55606b3f5ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e4ccd640 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2558029161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 msgr2=0x55606b3e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 -- 172.21.15.83:0/2558029161 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606b3eb090 con 0x55606b037350 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.104+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009f70 tx=0x7f04d8003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8af6a6da647bef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.105+0000 7f04caffd640 1 -- 172.21.15.83:0/2558029161 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d80049a0 con 0x55606b037350 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.105+0000 7f04caffd640 1 -- 172.21.15.83:0/2558029161 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d8027070 con 0x55606b037350 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.105+0000 7f04caffd640 1 -- 172.21.15.83:0/2558029161 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8005640 con 0x55606b037350 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- 172.21.15.83:0/2558029161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 msgr2=0x55606b3eba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009f70 tx=0x7f04d8003c70 comp rx=0 tx=0).stop 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- 172.21.15.83:0/2558029161 shutdown_connections 2024-09-21T03:17:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b037350 0x55606b3eba00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b016670 0x55606b3f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 --2- 172.21.15.83:0/2558029161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- 172.21.15.83:0/2558029161 >> 172.21.15.83:0/2558029161 conn(0x55606b02b740 msgr2=0x55606b3eb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- 172.21.15.83:0/2558029161 shutdown_connections 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- 172.21.15.83:0/2558029161 wait complete. 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.106+0000 7f04edca6740 1 -- start start 2024-09-21T03:17:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.107+0000 7f04edca6740 1 Processor -- start 2024-09-21T03:17:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b016670 0x55606b3eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b7020 con 0x55606b00e490 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b6950 con 0x55606b037350 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.108+0000 7f04edca6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b3b90 con 0x55606b016670 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58580/0 (socket says 172.21.15.83:58580) 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 -- 172.21.15.83:0/771535551 learned_addr learned my addr 172.21.15.83:0/771535551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04edca6740 10 client.0 _ll_register_callbacks cb 0x7f04d00a8dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e54ce640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 -- 172.21.15.83:0/771535551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b016670 msgr2=0x55606b3eeb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b016670 0x55606b3eeb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 -- 172.21.15.83:0/771535551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 msgr2=0x55606b3f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:38.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e5ccf640 1 -- 172.21.15.83:0/771535551 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606b3ef830 con 0x55606b037350 2024-09-21T03:17:38.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.109+0000 7f04e54ce640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04e5ccf640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d8009fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5026a846f3854d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8033070 con 0x55606b037350 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04d8004db0 con 0x55606b037350 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55606b3e4a20 con 0x55606b037350 2024-09-21T03:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f04d803d050 con 0x55606b037350 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04edca6740 10 client.25238 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.110+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55606b467d30 con 0x55606b037350 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55606b3e4a20 con 0x55606b037350 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f04d800f750 con 0x55606b037350 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 10 client.25238 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 20 client.25238 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606afcb080 0x55606b46a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 4 client.25238 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.111+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55606b356db0 con 0x55606afcb080 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.112+0000 7f04e54ce640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606afcb080 0x55606b46a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.112+0000 7f04c8ff9640 10 client.25238 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.112+0000 7f04e54ce640 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606afcb080 0x55606b46a330 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2190a92cbbf5ad server_cookie=9f53a76f5cdee3b4 in_seq=0 out_seq=0 2024-09-21T03:17:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04c8ff9640 1 -- 172.21.15.83:0/771535551 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55606b356db0 con 0x55606afcb080 2024-09-21T03:17:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04c8ff9640 10 client.25238 handle_command_reply: tid=0 2024-09-21T03:17:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 client.25238 shutdown 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 msgr2=0x55606b3ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d8009fc0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 shutdown_connections 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55606afcb080 0x55606b46a330 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04c8ff9640 0 client.25238 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55606b037350 0x55606b3ef0c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55606b016670 0x55606b3eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 --2- 172.21.15.83:0/771535551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606b00e490 0x55606b3f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 >> 172.21.15.83:0/771535551 conn(0x55606b02b740 msgr2=0x55606b3e1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 shutdown_connections 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 1 -- 172.21.15.83:0/771535551 wait complete. 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.114+0000 7f04edca6740 20 client.25238 trim_cache size 0 max 16384 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d0cbca43-4c1e-4314-b9b4-5725bc86d92f", 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:17:38.117 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 msgr2=0x7f04cc0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 0x7f04cc0b8c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04dc031440 tx=0x7f04dc032f40 comp rx=0 tx=0).stop 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 msgr2=0x7f04e01a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04d000ea80 tx=0x7f04d000ef40 comp rx=0 tx=0).stop 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 shutdown_connections 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04cc0b6780 0x7f04cc0b8c90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04e0108480 0x7f04e01a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0107b30 0x7f04e01a1060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 --2- 172.21.15.83:0/1422795110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04e01021f0 0x7f04e01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 >> 172.21.15.83:0/1422795110 conn(0x7f04e0077c60 msgr2=0x7f04e0105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 shutdown_connections 2024-09-21T03:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:38.115+0000 7f04e7732640 1 -- 172.21.15.83:0/1422795110 wait complete. 2024-09-21T03:17:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:38 smithi193 ceph-mon[35224]: pgmap v354: 97 pgs: 97 active+clean; 979 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-21T03:17:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:38 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:17:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:17:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:38.621 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:38 smithi083 ceph-mon[31219]: pgmap v354: 97 pgs: 97 active+clean; 979 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-21T03:17:38.621 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:38 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:17:38.621 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:38.621 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:17:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:38 smithi142 ceph-mon[34512]: pgmap v354: 97 pgs: 97 active+clean; 979 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-21T03:17:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:38 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:17:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:17:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:39 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:39 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:39 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:17:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 -- 172.21.15.83:0/1489761477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c4102a80 msgr2=0x7f45c4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 --2- 172.21.15.83:0/1489761477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c4102a80 0x7f45c4102e90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009900 tx=0x7f45b402a410 comp rx=0 tx=0).stop 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 -- 172.21.15.83:0/1489761477 shutdown_connections 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 --2- 172.21.15.83:0/1489761477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 --2- 172.21.15.83:0/1489761477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c41083c0 0x7f45c41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 --2- 172.21.15.83:0/1489761477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c4102a80 0x7f45c4102e90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.390+0000 7f45c948a640 1 -- 172.21.15.83:0/1489761477 >> 172.21.15.83:0/1489761477 conn(0x7f45c4079740 msgr2=0x7f45c4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.391+0000 7f45c948a640 1 -- 172.21.15.83:0/1489761477 shutdown_connections 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.391+0000 7f45c948a640 1 -- 172.21.15.83:0/1489761477 wait complete. 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.392+0000 7f45c948a640 1 Processor -- start 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.392+0000 7f45c948a640 1 -- start start 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c41083c0 0x7f45c4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c419db80 0x7f45c419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4112980 con 0x7f45c41083c0 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4112800 con 0x7f45c4108d10 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c948a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4112b00 con 0x7f45c419db80 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44336/0 (socket says 172.21.15.83:44336) 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 -- 172.21.15.83:0/942376829 learned_addr learned my addr 172.21.15.83:0/942376829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c41083c0 0x7f45c4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45c37fe640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c419db80 0x7f45c419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 -- 172.21.15.83:0/942376829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c419db80 msgr2=0x7f45c419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c419db80 0x7f45c419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 -- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c41083c0 msgr2=0x7f45c4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c41083c0 0x7f45c4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.393+0000 7f45bbfff640 1 -- 172.21.15.83:0/942376829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c419e760 con 0x7f45c4108d10 2024-09-21T03:17:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45bbfff640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9f0 tx=0x7f45ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45c0ff9640 1 -- 172.21.15.83:0/942376829 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ccc0 con 0x7f45c4108d10 2024-09-21T03:17:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45c0ff9640 1 -- 172.21.15.83:0/942376829 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0054d0 con 0x7f45c4108d10 2024-09-21T03:17:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c419ecf0 con 0x7f45c4108d10 2024-09-21T03:17:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c419f200 con 0x7f45c4108d10 2024-09-21T03:17:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.394+0000 7f45c0ff9640 1 -- 172.21.15.83:0/942376829 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac010430 con 0x7f45c4108d10 2024-09-21T03:17:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45c0ff9640 1 -- 172.21.15.83:0/942376829 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f45ac0105d0 con 0x7f45c4108d10 2024-09-21T03:17:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 Processor -- start 2024-09-21T03:17:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 -- start start 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d29b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d2a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a82c9fe50 con 0x556a82982070 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556a82c8f280 con 0x556a829591b0 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.398+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a82c93090 con 0x556a82961390 2024-09-21T03:17:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42424/0 (socket says 172.21.15.83:42424) 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 -- 172.21.15.83:0/2480564544 learned_addr learned my addr 172.21.15.83:0/2480564544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45bbfff640 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d2a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d29b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c0ff9640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 0x7f45b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 -- 172.21.15.83:0/2480564544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 msgr2=0x556a82d2a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d2a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 -- 172.21.15.83:0/2480564544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 msgr2=0x556a82d29b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d29b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 -- 172.21.15.83:0/2480564544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a829ed650 con 0x556a82982070 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 0x7f45b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c0ff9640 1 -- 172.21.15.83:0/942376829 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f45ac0e7780 con 0x7f45c4108d10 2024-09-21T03:17:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.399+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00afd0 tx=0x7f45bc00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3923c5b5d28489e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.400+0000 7f45b97fa640 1 -- 172.21.15.83:0/2480564544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc024430 con 0x556a82982070 2024-09-21T03:17:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.400+0000 7f45b97fa640 1 -- 172.21.15.83:0/2480564544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc012110 con 0x556a82982070 2024-09-21T03:17:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.400+0000 7f45b97fa640 1 -- 172.21.15.83:0/2480564544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc005720 con 0x556a82982070 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.401+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2480564544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 msgr2=0x556a82d29650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.401+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00afd0 tx=0x7f45bc00f850 comp rx=0 tx=0).stop 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45c37fe640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 0x7f45b00b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45bc020000 tx=0x7f45bc01c030 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2480564544 shutdown_connections 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d2a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d29b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2480564544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d29650 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2480564544 >> 172.21.15.83:0/2480564544 conn(0x556a82976460 msgr2=0x556a82d27540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2480564544 shutdown_connections 2024-09-21T03:17:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.402+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2480564544 wait complete. 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 -- start start 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 Processor -- start 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.403+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a82af0700 con 0x556a829591b0 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556a82c2a110 con 0x556a82961390 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a82d37e30 con 0x556a82982070 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42440/0 (socket says 172.21.15.83:42440) 2024-09-21T03:17:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45c2ffd640 1 -- 172.21.15.83:0/3876597728 learned_addr learned my addr 172.21.15.83:0/3876597728 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45cf9fe740 10 client.0 _ll_register_callbacks cb 0x556a82c43990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45c37fe640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 -- 172.21.15.83:0/3876597728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 msgr2=0x556a82d41180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d41180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 -- 172.21.15.83:0/3876597728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 msgr2=0x556a82d34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 -- 172.21.15.83:0/3876597728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a82c57c40 con 0x556a82961390 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45bbfff640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d35260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0e4f50 tx=0x7f45ac0b6f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b421c55159d035d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.404+0000 7f45aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:17:40.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.405+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0a9ca0 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.405+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f45ac0b6040 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.405+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45bc012110 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.405+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45ac0ad160 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.406+0000 7f45cf9fe740 10 client.25247 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.406+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a82d43c10 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.406+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc012110 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.406+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f45ac0ac320 con 0x556a82961390 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 10 client.25247 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 10 client.25247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 20 client.25247 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a829409f0 0x556a82d47140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 4 client.25247 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:17:40.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a82ca1da0 con 0x556a829409f0 2024-09-21T03:17:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.407+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a829409f0 0x556a82d47140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.408+0000 7f45aa7fc640 10 client.25247 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.410+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a829409f0 0x556a82d47140 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489fb6bbb8ae15ca server_cookie=ce0292cb5ddff521 in_seq=0 out_seq=0 2024-09-21T03:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.410+0000 7f45aa7fc640 1 -- 172.21.15.83:0/3876597728 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a82ca1da0 con 0x556a829409f0 2024-09-21T03:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.410+0000 7f45aa7fc640 10 client.25247 handle_command_reply: tid=0 2024-09-21T03:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.410+0000 7f45cf9fe740 1 client.25247 shutdown 2024-09-21T03:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 msgr2=0x556a82d35260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d35260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0e4f50 tx=0x7f45ac0b6f20 comp rx=0 tx=0).stop 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 shutdown_connections 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a829409f0 0x556a82d47140 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d41180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d35260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/3876597728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45aa7fc640 0 client.25247 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 >> 172.21.15.83:0/3876597728 conn(0x556a82976460 msgr2=0x556a82d2c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 shutdown_connections 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 1 -- 172.21.15.83:0/3876597728 wait complete. 2024-09-21T03:17:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.411+0000 7f45cf9fe740 20 client.25247 trim_cache size 0 max 16384 2024-09-21T03:17:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.442+0000 7f45cf9fe740 1 Processor -- start 2024-09-21T03:17:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.442+0000 7f45cf9fe740 1 -- start start 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d36950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45a400e550 con 0x556a829591b0 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45a4010150 con 0x556a82961390 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41616/0 (socket says 172.21.15.83:41616) 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45c37fe640 1 -- 172.21.15.83:0/2092326054 learned_addr learned my addr 172.21.15.83:0/2092326054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45bbfff640 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d36950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.443+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45a400ec20 con 0x556a82982070 2024-09-21T03:17:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 -- 172.21.15.83:0/2092326054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 msgr2=0x556a82d36950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d36950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 -- 172.21.15.83:0/2092326054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 msgr2=0x556a82d2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 -- 172.21.15.83:0/2092326054 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a82d35fe0 con 0x556a82982070 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.444+0000 7f45c37fe640 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004d50 tx=0x7f45bc004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ca3c98d37c638 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.445+0000 7f45b8ff9640 1 -- 172.21.15.83:0/2092326054 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc012ca0 con 0x556a82982070 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.445+0000 7f45b8ff9640 1 -- 172.21.15.83:0/2092326054 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc003e20 con 0x556a82982070 2024-09-21T03:17:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.445+0000 7f45b8ff9640 1 -- 172.21.15.83:0/2092326054 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc005030 con 0x556a82982070 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.445+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 msgr2=0x556a82d26b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.445+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004d50 tx=0x7f45bc004ce0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 shutdown_connections 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82982070 0x556a82d26b70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a82961390 0x556a82d36950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/2092326054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a829591b0 0x556a82d2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 >> 172.21.15.83:0/2092326054 conn(0x556a82976460 msgr2=0x556a82d36240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 shutdown_connections 2024-09-21T03:17:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 -- 172.21.15.83:0/2092326054 wait complete. 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 -- start start 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.446+0000 7f45cf9fe740 1 Processor -- start 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d3d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45a400f9c0 con 0x556a82982070 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45a400f2f0 con 0x556a829591b0 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45a400c530 con 0x556a82961390 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45cf9fe740 10 client.0 _ll_register_callbacks cb 0x7f45ac0a94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41628/0 (socket says 172.21.15.83:41628) 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d39070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42458/0 (socket says 172.21.15.83:42458) 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45bbfff640 1 -- 172.21.15.83:0/1074452502 learned_addr learned my addr 172.21.15.83:0/1074452502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.447+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d3d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 -- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 msgr2=0x556a82d3d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d3d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 -- 172.21.15.83:0/1074452502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 msgr2=0x556a82d39070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:17:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d39070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 -- 172.21.15.83:0/1074452502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a82d395b0 con 0x556a82961390 2024-09-21T03:17:40.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.448+0000 7f45bbfff640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9c0 tx=0x7f45ac0acef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236b139507f98fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:17:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.449+0000 7f45aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:17:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.449+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0a9ca0 con 0x556a82961390 2024-09-21T03:17:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f45ac0cf240 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45bc003e20 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45ac0cf440 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 10 client.25222 fetch_fsmap learned FSMap version 31 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a82db1800 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc003e20 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f45ac0b32c0 con 0x556a82961390 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 10 client.25222 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 20 client.25222 populate_metadata read hostname 'smithi083' 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.450+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a82915da0 0x556a82db3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.451+0000 7f45cf9fe740 4 client.25222 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.451+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a82c9d850 con 0x556a82915da0 2024-09-21T03:17:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.451+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a82915da0 0x556a82db3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.451+0000 7f45aa7fc640 10 client.25222 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.451+0000 7f45c2ffd640 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a82915da0 0x556a82db3e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10bbf6a3058b06ef server_cookie=998794b3c24b810f in_seq=0 out_seq=0 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45aa7fc640 1 -- 172.21.15.83:0/1074452502 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x556a82c9d850 con 0x556a82915da0 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45aa7fc640 10 client.25222 handle_command_reply: tid=0 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 client.25222 shutdown 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 msgr2=0x556a82d38b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9c0 tx=0x7f45ac0acef0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 shutdown_connections 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x556a82915da0 0x556a82db3e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a82982070 0x556a82d39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a82961390 0x556a82d38b30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45aa7fc640 0 client.25222 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 --2- 172.21.15.83:0/1074452502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556a829591b0 0x556a82d3d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 >> 172.21.15.83:0/1074452502 conn(0x556a82976460 msgr2=0x556a82d36b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 shutdown_connections 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 1 -- 172.21.15.83:0/1074452502 wait complete. 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:17:40.456 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d0cbca43-4c1e-4314-b9b4-5725bc86d92f", 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:17:40.457 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:17:40.458 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.452+0000 7f45cf9fe740 20 client.25222 trim_cache size 0 max 16384 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 msgr2=0x7f45b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 0x7f45b00b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45bc020000 tx=0x7f45bc01c030 comp rx=0 tx=0).stop 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 msgr2=0x7f45c41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9f0 tx=0x7f45ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 shutdown_connections 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45b00b6700 0x7f45b00b8c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45c419db80 0x7f45c419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45c4108d10 0x7f45c41989e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 --2- 172.21.15.83:0/942376829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c41083c0 0x7f45c4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:17:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 >> 172.21.15.83:0/942376829 conn(0x7f45c4079740 msgr2=0x7f45c406bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:17:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 shutdown_connections 2024-09-21T03:17:40.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:17:40.454+0000 7f45c948a640 1 -- 172.21.15.83:0/942376829 wait complete. 2024-09-21T03:17:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:40 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:40 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:40 smithi193 ceph-mon[35224]: pgmap v355: 97 pgs: 97 active+clean; 985 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 878 op/s 2024-09-21T03:17:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:40 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:40 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:40 smithi083 ceph-mon[31219]: pgmap v355: 97 pgs: 97 active+clean; 985 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 878 op/s 2024-09-21T03:17:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:40 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:40 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:17:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:40 smithi142 ceph-mon[34512]: pgmap v355: 97 pgs: 97 active+clean; 985 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 878 op/s 2024-09-21T03:17:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 20.0 s, 3236.2 tps, lat 30.882 ms stddev 44.395 2024-09-21T03:17:41.315 INFO:tasks.cephfs.filesystem:scrub status for tag:d0cbca43-4c1e-4314-b9b4-5725bc86d92f - {'path': '/', 'tag': 'd0cbca43-4c1e-4314-b9b4-5725bc86d92f', 'options': 'recursive,force'} 2024-09-21T03:17:42.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:42 smithi193 ceph-mon[35224]: pgmap v356: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 757 op/s 2024-09-21T03:17:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:42 smithi083 ceph-mon[31219]: pgmap v356: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 757 op/s 2024-09-21T03:17:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:42 smithi142 ceph-mon[34512]: pgmap v356: 97 pgs: 97 active+clean; 986 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 757 op/s 2024-09-21T03:17:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:44 smithi193 ceph-mon[35224]: pgmap v357: 97 pgs: 97 active+clean; 1002 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-21T03:17:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:44 smithi083 ceph-mon[31219]: pgmap v357: 97 pgs: 97 active+clean; 1002 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-21T03:17:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:44 smithi142 ceph-mon[34512]: pgmap v357: 97 pgs: 97 active+clean; 1002 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-21T03:17:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 25.0 s, 3471.0 tps, lat 28.669 ms stddev 28.728 2024-09-21T03:17:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:46 smithi193 ceph-mon[35224]: pgmap v358: 97 pgs: 97 active+clean; 1008 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-09-21T03:17:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:46 smithi083 ceph-mon[31219]: pgmap v358: 97 pgs: 97 active+clean; 1008 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-09-21T03:17:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:46 smithi142 ceph-mon[34512]: pgmap v358: 97 pgs: 97 active+clean; 1008 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.8 MiB/s wr, 807 op/s 2024-09-21T03:17:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:48 smithi193 ceph-mon[35224]: pgmap v359: 97 pgs: 97 active+clean; 1009 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 764 op/s 2024-09-21T03:17:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:48 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:48 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:48 smithi083 ceph-mon[31219]: pgmap v359: 97 pgs: 97 active+clean; 1009 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 764 op/s 2024-09-21T03:17:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:48 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:48 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:48 smithi142 ceph-mon[34512]: pgmap v359: 97 pgs: 97 active+clean; 1009 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 764 op/s 2024-09-21T03:17:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:48 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:48 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:17:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:49.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:49.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:17:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:17:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:50 smithi083 ceph-mon[31219]: pgmap v360: 97 pgs: 97 active+clean; 1013 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-09-21T03:17:50.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:50 smithi193 ceph-mon[35224]: pgmap v360: 97 pgs: 97 active+clean; 1013 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-09-21T03:17:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:50 smithi142 ceph-mon[34512]: pgmap v360: 97 pgs: 97 active+clean; 1013 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.9 MiB/s wr, 922 op/s 2024-09-21T03:17:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 30.0 s, 3684.2 tps, lat 27.277 ms stddev 27.413 2024-09-21T03:17:51.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:51 smithi193 ceph-mon[35224]: pgmap v361: 97 pgs: 97 active+clean; 1018 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-09-21T03:17:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:51 smithi083 ceph-mon[31219]: pgmap v361: 97 pgs: 97 active+clean; 1018 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-09-21T03:17:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:51 smithi142 ceph-mon[34512]: pgmap v361: 97 pgs: 97 active+clean; 1018 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-09-21T03:17:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:54 smithi193 ceph-mon[35224]: pgmap v362: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 852 op/s 2024-09-21T03:17:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:54 smithi083 ceph-mon[31219]: pgmap v362: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 852 op/s 2024-09-21T03:17:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:54 smithi142 ceph-mon[34512]: pgmap v362: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 14 MiB/s wr, 852 op/s 2024-09-21T03:17:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 35.0 s, 3487.4 tps, lat 28.667 ms stddev 29.659 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: pgmap v363: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:17:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:56 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: pgmap v363: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-09-21T03:17:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:17:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:17:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:17:56.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:56 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: pgmap v363: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:17:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:17:56.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:56 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:17:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:57 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:58.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:17:57 smithi142 ceph-mon[34512]: pgmap v364: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 792 op/s 2024-09-21T03:17:58.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:57 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:58.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:17:57 smithi193 ceph-mon[35224]: pgmap v364: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 792 op/s 2024-09-21T03:17:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:57 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:17:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:17:57 smithi083 ceph-mon[31219]: pgmap v364: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 792 op/s 2024-09-21T03:18:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:00 smithi193 ceph-mon[35224]: pgmap v365: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 908 op/s 2024-09-21T03:18:00.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:00 smithi083 ceph-mon[31219]: pgmap v365: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 908 op/s 2024-09-21T03:18:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:00 smithi142 ceph-mon[34512]: pgmap v365: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 908 op/s 2024-09-21T03:18:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 40.0 s, 3500.2 tps, lat 28.029 ms stddev 29.575 2024-09-21T03:18:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:02 smithi193 ceph-mon[35224]: pgmap v366: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-09-21T03:18:02.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:02 smithi083 ceph-mon[31219]: pgmap v366: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-09-21T03:18:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:02 smithi142 ceph-mon[34512]: pgmap v366: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-09-21T03:18:04.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:04 smithi193 ceph-mon[35224]: pgmap v367: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 1.16k op/s 2024-09-21T03:18:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:04 smithi083 ceph-mon[31219]: pgmap v367: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 1.16k op/s 2024-09-21T03:18:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:04 smithi142 ceph-mon[34512]: pgmap v367: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 33 MiB/s wr, 1.16k op/s 2024-09-21T03:18:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 45.0 s, 2919.6 tps, lat 34.884 ms stddev 89.366 2024-09-21T03:18:06.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:06 smithi193 ceph-mon[35224]: pgmap v368: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 37 MiB/s wr, 1.23k op/s 2024-09-21T03:18:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:06 smithi083 ceph-mon[31219]: pgmap v368: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 37 MiB/s wr, 1.23k op/s 2024-09-21T03:18:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:06 smithi142 ceph-mon[34512]: pgmap v368: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 37 MiB/s wr, 1.23k op/s 2024-09-21T03:18:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:08 smithi193 ceph-mon[35224]: pgmap v369: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 45 MiB/s wr, 1.41k op/s 2024-09-21T03:18:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:08 smithi083 ceph-mon[31219]: pgmap v369: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 45 MiB/s wr, 1.41k op/s 2024-09-21T03:18:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:08 smithi142 ceph-mon[34512]: pgmap v369: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 45 MiB/s wr, 1.41k op/s 2024-09-21T03:18:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:10 smithi193 ceph-mon[35224]: pgmap v370: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 44 MiB/s wr, 1.49k op/s 2024-09-21T03:18:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:10 smithi083 ceph-mon[31219]: pgmap v370: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 44 MiB/s wr, 1.49k op/s 2024-09-21T03:18:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:10 smithi142 ceph-mon[34512]: pgmap v370: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 44 MiB/s wr, 1.49k op/s 2024-09-21T03:18:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 50.0 s, 3566.8 tps, lat 28.030 ms stddev 29.469 2024-09-21T03:18:11.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:18:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:12.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:12 smithi193 ceph-mon[35224]: pgmap v371: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 43 MiB/s wr, 1.34k op/s 2024-09-21T03:18:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:12 smithi083 ceph-mon[31219]: pgmap v371: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 43 MiB/s wr, 1.34k op/s 2024-09-21T03:18:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:12 smithi142 ceph-mon[34512]: pgmap v371: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 43 MiB/s wr, 1.34k op/s 2024-09-21T03:18:13.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.412+0000 7fa2fe7fc640 1 -- 172.21.15.83:0/249120960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00318a0 con 0x7fa300102a90 2024-09-21T03:18:13.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 -- 172.21.15.83:0/249120960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa300102a90 msgr2=0x7fa300102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 --2- 172.21.15.83:0/249120960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa300102a90 0x7fa300102ea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009900 tx=0x7fa2f002a410 comp rx=0 tx=0).stop 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 -- 172.21.15.83:0/249120960 shutdown_connections 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 --2- 172.21.15.83:0/249120960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa30010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 --2- 172.21.15.83:0/249120960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3001083d0 0x7fa3001087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 --2- 172.21.15.83:0/249120960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa300102a90 0x7fa300102ea0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.414+0000 7fa30618a640 1 -- 172.21.15.83:0/249120960 >> 172.21.15.83:0/249120960 conn(0x7fa300079730 msgr2=0x7fa300079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.415+0000 7fa30618a640 1 -- 172.21.15.83:0/249120960 shutdown_connections 2024-09-21T03:18:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.415+0000 7fa30618a640 1 -- 172.21.15.83:0/249120960 wait complete. 2024-09-21T03:18:13.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 Processor -- start 2024-09-21T03:18:13.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 -- start start 2024-09-21T03:18:13.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 0x7fa30019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa3001a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa2feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa2feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37054/0 (socket says 172.21.15.83:37054) 2024-09-21T03:18:13.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001118d0 con 0x7fa3001083d0 2024-09-21T03:18:13.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa300111750 con 0x7fa300108d20 2024-09-21T03:18:13.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa30618a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa300111a50 con 0x7fa300102a90 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa2feffd640 1 -- 172.21.15.83:0/1578669650 learned_addr learned my addr 172.21.15.83:0/1578669650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.416+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 0x7fa30019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2fffff640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa3001a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 -- 172.21.15.83:0/1578669650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 msgr2=0x7fa30019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 0x7fa30019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 -- 172.21.15.83:0/1578669650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 msgr2=0x7fa3001a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa3001a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 -- 172.21.15.83:0/1578669650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3001a2740 con 0x7fa3001083d0 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 0x7fa30019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2fffff640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa3001a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.417+0000 7fa2feffd640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400e9c0 tx=0x7fa2f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.418+0000 7fa2fcff9640 1 -- 172.21.15.83:0/1578669650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400ccc0 con 0x7fa3001083d0 2024-09-21T03:18:13.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.418+0000 7fa2fcff9640 1 -- 172.21.15.83:0/1578669650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f40054d0 con 0x7fa3001083d0 2024-09-21T03:18:13.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.418+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3001a2a30 con 0x7fa3001083d0 2024-09-21T03:18:13.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.418+0000 7fa2fcff9640 1 -- 172.21.15.83:0/1578669650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4010430 con 0x7fa3001083d0 2024-09-21T03:18:13.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.418+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3001a9190 con 0x7fa3001083d0 2024-09-21T03:18:13.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.422+0000 7fa30c6fe740 1 Processor -- start 2024-09-21T03:18:13.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 -- start start 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fc2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fc2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb15f38e50 con 0x55cb15c1b070 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cb15f28280 con 0x55cb15bf21b0 2024-09-21T03:18:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cb15f2c090 con 0x55cb15bfa390 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2fcff9640 1 -- 172.21.15.83:0/1578669650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa2f4005020 con 0x7fa3001083d0 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fc2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33846/0 (socket says 172.21.15.83:33846) 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fc2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37064/0 (socket says 172.21.15.83:37064) 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.423+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2799148678 learned_addr learned my addr 172.21.15.83:0/2799148678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2799148678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 msgr2=0x55cb15fc2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fc2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2799148678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 msgr2=0x55cb15fc2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fc2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2799148678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb15c86650 con 0x55cb15bf21b0 2024-09-21T03:18:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009900 tx=0x7fa2f0005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5209648e852d5770 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2799148678 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0040070 con 0x55cb15bf21b0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2799148678 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0031720 con 0x55cb15bf21b0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2799148678 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0035040 con 0x55cb15bf21b0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2fcff9640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 0x7fa2e80b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2fcff9640 1 -- 172.21.15.83:0/1578669650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f40e79b0 con 0x7fa3001083d0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.424+0000 7fa2fffff640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 0x7fa2e80b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2799148678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 msgr2=0x55cb15fc27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa2fffff640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 0x7fa2e80b8cd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005fd0 tx=0x7fa2f8009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009900 tx=0x7fa2f0005e20 comp rx=0 tx=0).stop 2024-09-21T03:18:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2799148678 shutdown_connections 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fc2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fc27b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2799148678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fc2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2799148678 >> 172.21.15.83:0/2799148678 conn(0x55cb15c0f460 msgr2=0x55cb15fc0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2799148678 shutdown_connections 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.425+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2799148678 wait complete. 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.426+0000 7fa30c6fe740 1 -- start start 2024-09-21T03:18:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.426+0000 7fa30c6fe740 1 Processor -- start 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fcdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fe24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33858/0 (socket says 172.21.15.83:33858) 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/3532025313 learned_addr learned my addr 172.21.15.83:0/3532025313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb15d89700 con 0x55cb15c1b070 2024-09-21T03:18:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa2feffd640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fcdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa2fffff640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fe24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cb15ec3110 con 0x55cb15bf21b0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cb15fd09f0 con 0x55cb15bfa390 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.427+0000 7fa30c6fe740 10 client.0 _ll_register_callbacks cb 0x55cb15edc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/3532025313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 msgr2=0x55cb15fcdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fcdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/3532025313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 msgr2=0x55cb15fe24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fe24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb15ef0c40 con 0x55cb15bf21b0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2feffd640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fcdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00042f0 tx=0x7fa2f00053f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ce76a48c652f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.428+0000 7fa2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.429+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f004b070 con 0x55cb15bf21b0 2024-09-21T03:18:13.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.429+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f0003800 con 0x55cb15bf21b0 2024-09-21T03:18:13.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.429+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f0031720 con 0x55cb15bf21b0 2024-09-21T03:18:13.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.429+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2f0057050 con 0x55cb15bf21b0 2024-09-21T03:18:13.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 10 client.25265 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:13.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb15fe4f00 con 0x55cb15bf21b0 2024-09-21T03:18:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0031720 con 0x55cb15bf21b0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa2f003c5b0 con 0x55cb15bf21b0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 10 client.25265 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 10 client.25265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 20 client.25265 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15bd99f0 0x55cb15fe8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 4 client.25265 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.430+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb15f3ada0 con 0x55cb15bd99f0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.431+0000 7fa2feffd640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15bd99f0 0x55cb15fe8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.431+0000 7fa2deffd640 10 client.25265 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.431+0000 7fa2feffd640 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15bd99f0 0x55cb15fe8260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b409c6048b2a50aa server_cookie=8f06d83a58a2ceaa in_seq=0 out_seq=0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa2deffd640 1 -- 172.21.15.83:0/3532025313 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb15f3ada0 con 0x55cb15bd99f0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa2deffd640 10 client.25265 handle_command_reply: tid=0 2024-09-21T03:18:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 client.25265 shutdown 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 msgr2=0x55cb15fcd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00042f0 tx=0x7fa2f00053f0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 shutdown_connections 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15bd99f0 0x55cb15fe8260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15c1b070 0x55cb15fe24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa2deffd640 0 client.25265 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fcdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/3532025313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fcd9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 >> 172.21.15.83:0/3532025313 conn(0x55cb15c0f460 msgr2=0x55cb15fc5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 shutdown_connections 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 1 -- 172.21.15.83:0/3532025313 wait complete. 2024-09-21T03:18:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.432+0000 7fa30c6fe740 20 client.25265 trim_cache size 0 max 16384 2024-09-21T03:18:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.456+0000 7fa30c6fe740 1 Processor -- start 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 -- start start 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fbf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fbfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ad3e0 con 0x55cb15bd99f0 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001af040 con 0x55cb15bf21b0 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001adae0 con 0x55cb15bfa390 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa2feffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fbf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fcf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37080/0 (socket says 172.21.15.83:37080) 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/684445004 learned_addr learned my addr 172.21.15.83:0/684445004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.457+0000 7fa2fffff640 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fbfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 -- 172.21.15.83:0/684445004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 msgr2=0x55cb15fbfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fbfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 -- 172.21.15.83:0/684445004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 msgr2=0x55cb15fcf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fcf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 -- 172.21.15.83:0/684445004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb15fcec60 con 0x55cb15bf21b0 2024-09-21T03:18:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.458+0000 7fa2feffd640 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fbf710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40e5180 tx=0x7fa2f40e9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e4bb1e0be23cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.459+0000 7fa2e57fa640 1 -- 172.21.15.83:0/684445004 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40e7e60 con 0x55cb15bf21b0 2024-09-21T03:18:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.459+0000 7fa2e57fa640 1 -- 172.21.15.83:0/684445004 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f40b2030 con 0x55cb15bf21b0 2024-09-21T03:18:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.459+0000 7fa2e57fa640 1 -- 172.21.15.83:0/684445004 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40adf80 con 0x55cb15bf21b0 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- 172.21.15.83:0/684445004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 msgr2=0x55cb15fbf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fbf710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40e5180 tx=0x7fa2f40e9040 comp rx=0 tx=0).stop 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- 172.21.15.83:0/684445004 shutdown_connections 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bfa390 0x55cb15fbfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bf21b0 0x55cb15fbf710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/684445004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fcf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- 172.21.15.83:0/684445004 >> 172.21.15.83:0/684445004 conn(0x55cb15c0f460 msgr2=0x55cb15fceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- 172.21.15.83:0/684445004 shutdown_connections 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- 172.21.15.83:0/684445004 wait complete. 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.460+0000 7fa30c6fe740 1 -- start start 2024-09-21T03:18:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 Processor -- start 2024-09-21T03:18:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 0x55cb15fe9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 0x55cb15fe9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ae8b0 con 0x55cb15bd99f0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ae1e0 con 0x55cb15bfa390 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001ab3c0 con 0x55cb15bf21b0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa30c6fe740 10 client.0 _ll_register_callbacks cb 0x7fa2f80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2feffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 0x55cb15fe9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37086/0 (socket says 172.21.15.83:37086) 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2611657060 learned_addr learned my addr 172.21.15.83:0/2611657060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2fffff640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 0x55cb15fe9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2611657060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 msgr2=0x55cb15fe9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 0x55cb15fe9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 msgr2=0x55cb15fe9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 0x55cb15fe9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2ff7fe640 1 -- 172.21.15.83:0/2611657060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb160519e0 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2fffff640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 0x55cb15fe9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.461+0000 7fa2feffd640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 0x55cb15fe9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2ff7fe640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009900 tx=0x7fa2f0032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdb8a89655bcd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f004b040 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f40b2030 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f00314a0 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2f0035030 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 10 client.15753 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb16052be0 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f40b2030 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa2f0003800 con 0x55cb15bd99f0 2024-09-21T03:18:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 10 client.15753 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 20 client.15753 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15c1b070 0x55cb16055490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 4 client.15753 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.462+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb15f36850 con 0x55cb15c1b070 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.463+0000 7fa2feffd640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15c1b070 0x55cb16055490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.463+0000 7fa2deffd640 10 client.15753 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.464+0000 7fa2feffd640 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15c1b070 0x55cb16055490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d984b08dca02d3 server_cookie=d67b53a39b764441 in_seq=0 out_seq=0 2024-09-21T03:18:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa2deffd640 1 -- 172.21.15.83:0/2611657060 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55cb15f36850 con 0x55cb15c1b070 2024-09-21T03:18:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa2deffd640 10 client.15753 handle_command_reply: tid=0 2024-09-21T03:18:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 client.15753 shutdown 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 msgr2=0x55cb15fe91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009900 tx=0x7fa2f0032ed0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 shutdown_connections 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cb15c1b070 0x55cb16055490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa2deffd640 0 client.15753 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb15bfa390 0x55cb15fe9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb15bf21b0 0x55cb15fe9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 --2- 172.21.15.83:0/2611657060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb15bd99f0 0x55cb15fe91d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.465+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 >> 172.21.15.83:0/2611657060 conn(0x55cb15c0f460 msgr2=0x55cb15fcee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.466+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 shutdown_connections 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.466+0000 7fa30c6fe740 1 -- 172.21.15.83:0/2611657060 wait complete. 2024-09-21T03:18:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.466+0000 7fa30c6fe740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:18:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:18:13.472 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 msgr2=0x7fa2e80b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 0x7fa2e80b8cd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005fd0 tx=0x7fa2f8009500 comp rx=0 tx=0).stop 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 msgr2=0x7fa30019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400e9c0 tx=0x7fa2f400ee80 comp rx=0 tx=0).stop 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 shutdown_connections 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b67c0 0x7fa2e80b8cd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa300108d20 0x7fa3001a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3001083d0 0x7fa30019cd70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 --2- 172.21.15.83:0/1578669650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa300102a90 0x7fa30019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 >> 172.21.15.83:0/1578669650 conn(0x7fa300079730 msgr2=0x7fa300106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 shutdown_connections 2024-09-21T03:18:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:13.467+0000 7fa30618a640 1 -- 172.21.15.83:0/1578669650 wait complete. 2024-09-21T03:18:14.280 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:18:14.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:18:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:14 smithi193 ceph-mon[35224]: pgmap v372: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 44 MiB/s wr, 1.42k op/s 2024-09-21T03:18:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:14 smithi083 ceph-mon[31219]: pgmap v372: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 44 MiB/s wr, 1.42k op/s 2024-09-21T03:18:14.652 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:14 smithi142 ceph-mon[34512]: pgmap v372: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 44 MiB/s wr, 1.42k op/s 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 -- 172.21.15.83:0/3311153068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4103140 msgr2=0x7f22f4103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/3311153068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4103140 0x7f22f4103550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009900 tx=0x7f22e402a440 comp rx=0 tx=0).stop 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 -- 172.21.15.83:0/3311153068 shutdown_connections 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/3311153068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/3311153068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f41083b0 0x7f22f41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/3311153068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f4103140 0x7f22f4103550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 -- 172.21.15.83:0/3311153068 >> 172.21.15.83:0/3311153068 conn(0x7f22f4077c60 msgr2=0x7f22f40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 -- 172.21.15.83:0/3311153068 shutdown_connections 2024-09-21T03:18:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.189+0000 7f22f9a9e640 1 -- 172.21.15.83:0/3311153068 wait complete. 2024-09-21T03:18:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.190+0000 7f22f9a9e640 1 Processor -- start 2024-09-21T03:18:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.190+0000 7f22f9a9e640 1 -- start start 2024-09-21T03:18:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.190+0000 7f22f9a9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 0x7f22f419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.190+0000 7f22f9a9e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 0x7f22f419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.190+0000 7f22f9a9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f9a9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112aa0 con 0x7f22f4108d00 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f9a9e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112920 con 0x7f22f41083b0 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f9a9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4112c20 con 0x7f22f4103140 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 0x7f22f419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 0x7f22f419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33892/0 (socket says 172.21.15.83:33892) 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f2ffd640 1 -- 172.21.15.83:0/2635305066 learned_addr learned my addr 172.21.15.83:0/2635305066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 0x7f22f419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.191+0000 7f22f3fff640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 -- 172.21.15.83:0/2635305066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 msgr2=0x7f22f419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 0x7f22f419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 -- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 msgr2=0x7f22f419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 -- 172.21.15.83:0/2635305066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f41a0610 con 0x7f22f4103140 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f3fff640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.192+0000 7f22f37fe640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 0x7f22f419a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22e402a960 tx=0x7f22e40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.193+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e403f030 con 0x7f22f4103140 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.193+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e4003cb0 con 0x7f22f4103140 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.193+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4035070 con 0x7f22f4103140 2024-09-21T03:18:15.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.193+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a08a0 con 0x7f22f4103140 2024-09-21T03:18:15.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.194+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a7060 con 0x7f22f4103140 2024-09-21T03:18:15.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.195+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f4103ec0 con 0x7f22f4103140 2024-09-21T03:18:15.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.198+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f22e4042030 con 0x7f22f4103140 2024-09-21T03:18:15.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.199+0000 7f22f0ff9640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 0x7f22dc0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:15.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.200+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f22e4107a30 con 0x7f22f4103140 2024-09-21T03:18:15.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.200+0000 7f22f2ffd640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 0x7f22dc0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:15.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.201+0000 7f22f2ffd640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 0x7f22dc0b8d60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22f41055b0 tx=0x7f22e8009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:15.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.202+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f22e40d2380 con 0x7f22f4103140 2024-09-21T03:18:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:15 smithi083 ceph-mon[31219]: pgmap v373: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-21T03:18:15.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.547+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f41a7230 con 0x7f22f4103140 2024-09-21T03:18:15.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.547+0000 7f22f0ff9640 1 -- 172.21.15.83:0/2635305066 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11991 (secure 0 0 0) 0x7f22e40d72c0 con 0x7f22f4103140 2024-09-21T03:18:15.549 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:18:15.550 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"btime":"2024-09-21T03:17:56:891811+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:17:56.429534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:18:15.550 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-09-21T03:18:15.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 msgr2=0x7f22dc0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 0x7f22dc0b8d60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22f41055b0 tx=0x7f22e8009190 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 msgr2=0x7f22f419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 0x7f22f419a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22e402a960 tx=0x7f22e40042f0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 shutdown_connections 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f22dc0b6850 0x7f22dc0b8d60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22f4108d00 0x7f22f419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22f41083b0 0x7f22f419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 --2- 172.21.15.83:0/2635305066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22f4103140 0x7f22f419a6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 >> 172.21.15.83:0/2635305066 conn(0x7f22f4077c60 msgr2=0x7f22f4106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 shutdown_connections 2024-09-21T03:18:15.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:15.551+0000 7f22f9a9e640 1 -- 172.21.15.83:0/2635305066 wait complete. 2024-09-21T03:18:15.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:15 smithi193 ceph-mon[35224]: pgmap v373: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-21T03:18:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:15 smithi142 ceph-mon[34512]: pgmap v373: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-21T03:18:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 55.0 s, 3450.6 tps, lat 28.977 ms stddev 29.885 2024-09-21T03:18:16.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:18:16.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:16 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2635305066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:18:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:16 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2635305066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:18:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:16 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2635305066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:18:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:17 smithi083 ceph-mon[31219]: pgmap v374: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-09-21T03:18:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:17 smithi142 ceph-mon[34512]: pgmap v374: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 -- 172.21.15.83:0/1534180442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410a3c0 msgr2=0x7f998410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 --2- 172.21.15.83:0/1534180442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410a3c0 0x7f998410a7d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402a410 comp rx=0 tx=0).stop 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 -- 172.21.15.83:0/1534180442 shutdown_connections 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 --2- 172.21.15.83:0/1534180442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9984118900 0x7f998411ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 --2- 172.21.15.83:0/1534180442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998410ad10 0x7f99841183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 --2- 172.21.15.83:0/1534180442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410a3c0 0x7f998410a7d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 -- 172.21.15.83:0/1534180442 >> 172.21.15.83:0/1534180442 conn(0x7f99840712f0 msgr2=0x7f9984073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 -- 172.21.15.83:0/1534180442 shutdown_connections 2024-09-21T03:18:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.741+0000 7f99888b1640 1 -- 172.21.15.83:0/1534180442 wait complete. 2024-09-21T03:18:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 Processor -- start 2024-09-21T03:18:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 -- start start 2024-09-21T03:18:17.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 0x7f99841a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 0x7f99841a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9984111040 con 0x7f998410ad10 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9984110ec0 con 0x7f9984118900 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f99888b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99841111c0 con 0x7f998410a3c0 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f998259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 0x7f99841a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 0x7f99841a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f9982d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f9982d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33908/0 (socket says 172.21.15.83:33908) 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f9982d9d640 1 -- 172.21.15.83:0/1393924244 learned_addr learned my addr 172.21.15.83:0/1393924244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.743+0000 7f998259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 0x7f99841a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:42846/0 (socket says 172.21.15.83:42846) 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 -- 172.21.15.83:0/1393924244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 msgr2=0x7f99841a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 0x7f99841a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 -- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 msgr2=0x7f99841a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 0x7f99841a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 -- 172.21.15.83:0/1393924244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99841a6a60 con 0x7f9984118900 2024-09-21T03:18:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 0x7f99841a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f9982d9d640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f997c01fd10 tx=0x7f997c01d650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f99737fe640 1 -- 172.21.15.83:0/1393924244 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c004b20 con 0x7f9984118900 2024-09-21T03:18:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99841a6d50 con 0x7f9984118900 2024-09-21T03:18:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f99737fe640 1 -- 172.21.15.83:0/1393924244 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f997c004cc0 con 0x7f9984118900 2024-09-21T03:18:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f99737fe640 1 -- 172.21.15.83:0/1393924244 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0226e0 con 0x7f9984118900 2024-09-21T03:18:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.744+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841ad4b0 con 0x7f9984118900 2024-09-21T03:18:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.747+0000 7f998ee25740 1 Processor -- start 2024-09-21T03:18:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f99737fe640 1 -- 172.21.15.83:0/1393924244 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f997c022880 con 0x7f9984118900 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 -- start start 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958ba16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d9390 0x556958ba2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556958b17ea0 con 0x5569587d9390 2024-09-21T03:18:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556958b072d0 con 0x5569587d11b0 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.748+0000 7f998ee25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556958b0b0e0 con 0x5569587fa070 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.749+0000 7f998259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.749+0000 7f998259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33916/0 (socket says 172.21.15.83:33916) 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.749+0000 7f998259c640 1 -- 172.21.15.83:0/1357656929 learned_addr learned my addr 172.21.15.83:0/1357656929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.749+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958ba16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.749+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d9390 0x556958ba2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.750+0000 7f9981d9b640 1 -- 172.21.15.83:0/1357656929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 msgr2=0x556958ba1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.750+0000 7f99737fe640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 0x7f996c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f9981d9b640 1 -- 172.21.15.83:0/1357656929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d9390 msgr2=0x556958ba2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d9390 0x556958ba2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f9981d9b640 1 -- 172.21.15.83:0/1357656929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556958865650 con 0x5569587fa070 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f998259c640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f9982d9d640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 0x7f996c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.751+0000 7f99737fe640 1 -- 172.21.15.83:0/1393924244 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f997c0f88c0 con 0x7f9984118900 2024-09-21T03:18:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.752+0000 7f9982d9d640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 0x7f996c0b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f997c0be7d0 tx=0x7f997c0ef230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.757+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958ba16a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9978005d00 tx=0x7f9978005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e0312919f99798 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.758+0000 7f9970ff9640 1 -- 172.21.15.83:0/1357656929 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978004020 con 0x5569587fa070 2024-09-21T03:18:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.758+0000 7f9970ff9640 1 -- 172.21.15.83:0/1357656929 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99780054d0 con 0x5569587fa070 2024-09-21T03:18:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.758+0000 7f9970ff9640 1 -- 172.21.15.83:0/1357656929 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978010b60 con 0x5569587fa070 2024-09-21T03:18:17.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.760+0000 7f998ee25740 1 -- 172.21.15.83:0/1357656929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 msgr2=0x556958ba16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.760+0000 7f998ee25740 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958ba16a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9978005d00 tx=0x7f9978005c90 comp rx=0 tx=0).stop 2024-09-21T03:18:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 -- 172.21.15.83:0/1357656929 shutdown_connections 2024-09-21T03:18:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d9390 0x556958ba2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d11b0 0x556958ba1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 --2- 172.21.15.83:0/1357656929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958ba16a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 -- 172.21.15.83:0/1357656929 >> 172.21.15.83:0/1357656929 conn(0x5569587ee460 msgr2=0x556958b9f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.761+0000 7f998ee25740 1 -- 172.21.15.83:0/1357656929 shutdown_connections 2024-09-21T03:18:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.762+0000 7f998ee25740 1 -- 172.21.15.83:0/1357656929 wait complete. 2024-09-21T03:18:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.762+0000 7f998ee25740 1 -- start start 2024-09-21T03:18:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.762+0000 7f998ee25740 1 Processor -- start 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958bad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958bb91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556958968700 con 0x5569587d11b0 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556958aa2160 con 0x5569587d9390 2024-09-21T03:18:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556958bafe80 con 0x5569587fa070 2024-09-21T03:18:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998ee25740 10 client.0 _ll_register_callbacks cb 0x556958abb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37124/0 (socket says 172.21.15.83:37124) 2024-09-21T03:18:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f9981d9b640 1 -- 172.21.15.83:0/1918723266 learned_addr learned my addr 172.21.15.83:0/1918723266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.763+0000 7f998259c640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958bad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958bb91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f998259c640 1 -- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 msgr2=0x556958bb91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f998259c640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958bb91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f998259c640 1 -- 172.21.15.83:0/1918723266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 msgr2=0x556958bace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f998259c640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.764+0000 7f998259c640 1 -- 172.21.15.83:0/1918723266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556958acfc90 con 0x5569587d9390 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.765+0000 7f998259c640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958bad360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x556958ba0320 tx=0x7f99740314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293e2f2dbe1e9941 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.765+0000 7f9969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.766+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974035040 con 0x5569587d9390 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.766+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99780054d0 con 0x5569587d9390 2024-09-21T03:18:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.766+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9974031810 con 0x5569587d9390 2024-09-21T03:18:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.766+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9974031b10 con 0x5569587d9390 2024-09-21T03:18:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.767+0000 7f998ee25740 10 client.25289 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.767+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556958bbbc90 con 0x5569587d9390 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99780054d0 con 0x5569587d9390 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f99740047b0 con 0x5569587d9390 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 10 client.25289 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 10 client.25289 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 20 client.25289 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5569587b89f0 0x556958bbf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 4 client.25289 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:17.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.769+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556958b19df0 con 0x5569587b89f0 2024-09-21T03:18:17.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.770+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5569587b89f0 0x556958bbf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.770+0000 7f9969ffb640 10 client.25289 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:18:17.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.770+0000 7f9981d9b640 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5569587b89f0 0x556958bbf590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5d7d23380f5e51a server_cookie=dc204b4ca7e19d8b in_seq=0 out_seq=0 2024-09-21T03:18:17.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.771+0000 7f9969ffb640 1 -- 172.21.15.83:0/1918723266 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556958b19df0 con 0x5569587b89f0 2024-09-21T03:18:17.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.771+0000 7f9969ffb640 10 client.25289 handle_command_reply: tid=0 2024-09-21T03:18:17.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.771+0000 7f998ee25740 1 client.25289 shutdown 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.772+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 msgr2=0x556958bad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.772+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958bad360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x556958ba0320 tx=0x7f99740314a0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 shutdown_connections 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5569587b89f0 0x556958bbf590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958bb91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958bad360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 --2- 172.21.15.83:0/1918723266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f9969ffb640 0 client.25289 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 >> 172.21.15.83:0/1918723266 conn(0x5569587ee460 msgr2=0x556958ba4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 shutdown_connections 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 1 -- 172.21.15.83:0/1918723266 wait complete. 2024-09-21T03:18:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.773+0000 7f998ee25740 20 client.25289 trim_cache size 0 max 16384 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 Processor -- start 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 -- start start 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958ba2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958b9ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958b9f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958ba2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958ba2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37126/0 (socket says 172.21.15.83:37126) 2024-09-21T03:18:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958b9ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f9981d9b640 1 -- 172.21.15.83:0/2261264261 learned_addr learned my addr 172.21.15.83:0/2261264261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b5cc0 con 0x5569587d11b0 2024-09-21T03:18:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b7920 con 0x5569587d9390 2024-09-21T03:18:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b63c0 con 0x5569587fa070 2024-09-21T03:18:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.791+0000 7f9981d9b640 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958b9f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 -- 172.21.15.83:0/2261264261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 msgr2=0x556958b9f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958b9f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 -- 172.21.15.83:0/2261264261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 msgr2=0x556958ba2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958ba2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 -- 172.21.15.83:0/2261264261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556958ba3f30 con 0x5569587d9390 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.792+0000 7f998259c640 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958b9ebc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f997402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e145098374899d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.793+0000 7f996bfff640 1 -- 172.21.15.83:0/2261264261 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974035030 con 0x5569587d9390 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.793+0000 7f996bfff640 1 -- 172.21.15.83:0/2261264261 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99740048c0 con 0x5569587d9390 2024-09-21T03:18:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.793+0000 7f996bfff640 1 -- 172.21.15.83:0/2261264261 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997404f450 con 0x5569587d9390 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 msgr2=0x556958b9ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958b9ebc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f997402a410 comp rx=0 tx=0).stop 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 shutdown_connections 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587fa070 0x556958b9f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587d9390 0x556958b9ebc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 --2- 172.21.15.83:0/2261264261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958ba2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 >> 172.21.15.83:0/2261264261 conn(0x5569587ee460 msgr2=0x556958ba26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 shutdown_connections 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.794+0000 7f998ee25740 1 -- 172.21.15.83:0/2261264261 wait complete. 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 -- start start 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 Processor -- start 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bb5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 0x556958bb0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587fa070 0x556958bb13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b7190 con 0x5569587d11b0 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b6ac0 con 0x5569587fa070 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b3ca0 con 0x5569587d9390 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998ee25740 10 client.0 _ll_register_callbacks cb 0x7f9974005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bb5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f998259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 0x556958bb0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f9981d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bb5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37142/0 (socket says 172.21.15.83:37142) 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f9981d9b640 1 -- 172.21.15.83:0/4128632229 learned_addr learned my addr 172.21.15.83:0/4128632229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.795+0000 7f9981d9b640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587fa070 0x556958bb13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.796+0000 7f998259c640 1 -- 172.21.15.83:0/4128632229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587fa070 msgr2=0x556958bb13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.796+0000 7f998259c640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587fa070 0x556958bb13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.796+0000 7f998259c640 1 -- 172.21.15.83:0/4128632229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 msgr2=0x556958bb5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.796+0000 7f998259c640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bb5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.796+0000 7f998259c640 1 -- 172.21.15.83:0/4128632229 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556958bb1b30 con 0x5569587d9390 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.797+0000 7f998259c640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 0x556958bb0e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f9974032d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a6a86ba25f7b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.797+0000 7f9969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974035070 con 0x5569587d9390 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9974004fa0 con 0x5569587d9390 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99740048c0 con 0x5569587d9390 2024-09-21T03:18:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99740055e0 con 0x5569587d9390 2024-09-21T03:18:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f998ee25740 10 client.25258 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556958c29f00 con 0x5569587d9390 2024-09-21T03:18:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.798+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99740048c0 con 0x5569587d9390 2024-09-21T03:18:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9974003710 con 0x5569587d9390 2024-09-21T03:18:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 10 client.25258 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 10 client.25258 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:18:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 20 client.25258 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55695878dda0 0x556958c2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 4 client.25258 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:18:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556958b158a0 con 0x55695878dda0 2024-09-21T03:18:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.800+0000 7f9981d9b640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55695878dda0 0x556958c2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f9969ffb640 10 client.25258 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:18:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f9981d9b640 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55695878dda0 0x556958c2c4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e52ce82d39cde138 server_cookie=86368596b220f637 in_seq=0 out_seq=0 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f9969ffb640 1 -- 172.21.15.83:0/4128632229 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556958b158a0 con 0x55695878dda0 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f9969ffb640 10 client.25258 handle_command_reply: tid=0 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 client.25258 shutdown 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 msgr2=0x556958bb0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 0x556958bb0e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f9974032d40 comp rx=0 tx=0).stop 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 shutdown_connections 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55695878dda0 0x556958c2c4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569587fa070 0x556958bb13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569587d9390 0x556958bb0e00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 --2- 172.21.15.83:0/4128632229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569587d11b0 0x556958bb5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f9969ffb640 0 client.25258 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 >> 172.21.15.83:0/4128632229 conn(0x5569587ee460 msgr2=0x556958ba26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 shutdown_connections 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 1 -- 172.21.15.83:0/4128632229 wait complete. 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.801+0000 7f998ee25740 20 client.25258 trim_cache size 0 max 16384 2024-09-21T03:18:17.805 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:18:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 msgr2=0x7f996c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 0x7f996c0b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f997c0be7d0 tx=0x7f997c0ef230 comp rx=0 tx=0).stop 2024-09-21T03:18:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 msgr2=0x7f99841a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f997c01fd10 tx=0x7f997c01d650 comp rx=0 tx=0).stop 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 shutdown_connections 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f996c0b6680 0x7f996c0b8b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984118900 0x7f99841a6330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f998410ad10 0x7f99841a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 --2- 172.21.15.83:0/1393924244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f998410a3c0 0x7f99841a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 >> 172.21.15.83:0/1393924244 conn(0x7f99840712f0 msgr2=0x7f9984118d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 shutdown_connections 2024-09-21T03:18:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:17.802+0000 7f99888b1640 1 -- 172.21.15.83:0/1393924244 wait complete. 2024-09-21T03:18:17.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:17 smithi193 ceph-mon[35224]: pgmap v374: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 874 op/s 2024-09-21T03:18:18.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:18:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.523+0000 7fc0e610d640 1 -- 172.21.15.83:0/3521560509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 msgr2=0x7fc0e01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.523+0000 7fc0e610d640 1 --2- 172.21.15.83:0/3521560509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01087d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009900 tx=0x7fc0d402a410 comp rx=0 tx=0).stop 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 -- 172.21.15.83:0/3521560509 shutdown_connections 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 --2- 172.21.15.83:0/3521560509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 0x7fc0e010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 --2- 172.21.15.83:0/3521560509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01087d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 --2- 172.21.15.83:0/3521560509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 -- 172.21.15.83:0/3521560509 >> 172.21.15.83:0/3521560509 conn(0x7fc0e0079740 msgr2=0x7fc0e0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 -- 172.21.15.83:0/3521560509 shutdown_connections 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.524+0000 7fc0e610d640 1 -- 172.21.15.83:0/3521560509 wait complete. 2024-09-21T03:18:19.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.525+0000 7fc0e610d640 1 Processor -- start 2024-09-21T03:18:19.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.525+0000 7fc0e610d640 1 -- start start 2024-09-21T03:18:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 0x7fc0e019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0112980 con 0x7fc0e01083c0 2024-09-21T03:18:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0112800 con 0x7fc0e0102a80 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0e610d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0112b00 con 0x7fc0e0108d10 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60040/0 (socket says 172.21.15.83:60040) 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2120441743 learned_addr learned my addr 172.21.15.83:0/2120441743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0dffff640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 0x7fc0e019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2120441743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 msgr2=0x7fc0e019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 0x7fc0e019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 msgr2=0x7fc0e01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.526+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2120441743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e019f2c0 con 0x7fc0e0102a80 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e9f0 tx=0x7fc0d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0dcff9640 1 -- 172.21.15.83:0/2120441743 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000cd80 con 0x7fc0e0102a80 2024-09-21T03:18:19.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0dcff9640 1 -- 172.21.15.83:0/2120441743 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d00108e0 con 0x7fc0e0102a80 2024-09-21T03:18:19.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019f5b0 con 0x7fc0e0102a80 2024-09-21T03:18:19.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0dcff9640 1 -- 172.21.15.83:0/2120441743 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0018980 con 0x7fc0e0102a80 2024-09-21T03:18:19.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.527+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e006b820 con 0x7fc0e0102a80 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 Processor -- start 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 -- start start 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0dcff9640 1 -- 172.21.15.83:0/2120441743 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc0d0010430 con 0x7fc0e0102a80 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 0x55d9c4886800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 0x55d9c4886d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9c47fcea0 con 0x55d9c44b61b0 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d9c47ec2d0 con 0x55d9c44df070 2024-09-21T03:18:19.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0ec681740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9c47f00e0 con 0x55d9c44be390 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.532+0000 7fc0dffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 0x55d9c4886800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60042/0 (socket says 172.21.15.83:60042) 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dffff640 1 -- 172.21.15.83:0/2480441441 learned_addr learned my addr 172.21.15.83:0/2480441441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 0x55d9c4886d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2480441441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 msgr2=0x55d9c4886d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 0x55d9c4886d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2480441441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 msgr2=0x55d9c48862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2480441441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c454a650 con 0x55d9c44b61b0 2024-09-21T03:18:19.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 0x55d9c4886d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dffff640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dcff9640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 0x7fc0c80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 0x55d9c4886800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00e51d0 tx=0x7fc0d00eb610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5251129847f4c49e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dffff640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 0x7fc0c80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0dcff9640 1 -- 172.21.15.83:0/2120441743 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0d00e7a00 con 0x7fc0e0102a80 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.533+0000 7fc0c5ffb640 1 -- 172.21.15.83:0/2480441441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00a9f20 con 0x55d9c44b61b0 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.534+0000 7fc0dffff640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 0x7fc0c80b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8011830 tx=0x7fc0d8009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.534+0000 7fc0c5ffb640 1 -- 172.21.15.83:0/2480441441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d0010720 con 0x55d9c44b61b0 2024-09-21T03:18:19.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.534+0000 7fc0c5ffb640 1 -- 172.21.15.83:0/2480441441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00ad040 con 0x55d9c44b61b0 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 -- 172.21.15.83:0/2480441441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 msgr2=0x55d9c4886800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 0x55d9c4886800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00e51d0 tx=0x7fc0d00eb610 comp rx=0 tx=0).stop 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 -- 172.21.15.83:0/2480441441 shutdown_connections 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44be390 0x55d9c4886d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44b61b0 0x55d9c4886800 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2480441441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c48862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 -- 172.21.15.83:0/2480441441 >> 172.21.15.83:0/2480441441 conn(0x55d9c44d3460 msgr2=0x55d9c48841b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.535+0000 7fc0ec681740 1 -- 172.21.15.83:0/2480441441 shutdown_connections 2024-09-21T03:18:19.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.536+0000 7fc0ec681740 1 -- 172.21.15.83:0/2480441441 wait complete. 2024-09-21T03:18:19.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.536+0000 7fc0ec681740 1 -- start start 2024-09-21T03:18:19.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 Processor -- start 2024-09-21T03:18:19.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4891f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c489dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9c464d700 con 0x55d9c44be390 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4787160 con 0x55d9c44df070 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4894a40 con 0x55d9c44b61b0 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55456/0 (socket says 172.21.15.83:55456) 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2031308092 learned_addr learned my addr 172.21.15.83:0/2031308092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4891f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0ec681740 10 client.0 _ll_register_callbacks cb 0x55d9c47a09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.537+0000 7fc0dffff640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c489dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2031308092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 msgr2=0x55d9c489dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c489dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2031308092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 msgr2=0x55d9c4891f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4891f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 -- 172.21.15.83:0/2031308092 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c47b4c90 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4891f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00ad5d0 tx=0x7fc0d0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33e4555ed9171 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00ad220 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0d00b7060 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9c47fedf0 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0d00ad6a0 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0ec681740 10 client.25270 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9c48a0760 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.538+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d9c47fedf0 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc0d00f20c0 con 0x55d9c44b61b0 2024-09-21T03:18:19.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 10 client.25270 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 10 client.25270 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 20 client.25270 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c449d9f0 0x55d9c48a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 4 client.25270 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c47fa8a0 con 0x55d9c449d9f0 2024-09-21T03:18:19.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.539+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c449d9f0 0x55d9c48a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.540+0000 7fc0deffd640 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c449d9f0 0x55d9c48a3ac0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb95745333bd283e server_cookie=ed09411fc103b6e1 in_seq=0 out_seq=0 2024-09-21T03:18:19.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.540+0000 7fc0beffd640 10 client.25270 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:18:19.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.541+0000 7fc0beffd640 1 -- 172.21.15.83:0/2031308092 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9c47fa8a0 con 0x55d9c449d9f0 2024-09-21T03:18:19.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.541+0000 7fc0beffd640 10 client.25270 handle_command_reply: tid=0 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.541+0000 7fc0ec681740 1 client.25270 shutdown 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 msgr2=0x55d9c48919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00ad5d0 tx=0x7fc0d0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 shutdown_connections 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c489dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4891f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c449d9f0 0x55d9c48a3ac0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 --2- 172.21.15.83:0/2031308092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c48919c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0beffd640 0 client.25270 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 >> 172.21.15.83:0/2031308092 conn(0x55d9c44d3460 msgr2=0x55d9c48894a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 shutdown_connections 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 1 -- 172.21.15.83:0/2031308092 wait complete. 2024-09-21T03:18:19.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.542+0000 7fc0ec681740 20 client.25270 trim_cache size 0 max 16384 2024-09-21T03:18:19.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 Processor -- start 2024-09-21T03:18:19.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 -- start start 2024-09-21T03:18:19.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c4887a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c4883ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ad400 con 0x55d9c44be390 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01af060 con 0x55d9c44df070 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.559+0000 7fc0ec681740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01adb00 con 0x55d9c44b61b0 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46004/0 (socket says 172.21.15.83:46004) 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 -- 172.21.15.83:0/4292730408 learned_addr learned my addr 172.21.15.83:0/4292730408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0dffff640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c4883ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c4887a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 -- 172.21.15.83:0/4292730408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 msgr2=0x55d9c4887a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c4887a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 -- 172.21.15.83:0/4292730408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 msgr2=0x55d9c4883ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c4883ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 -- 172.21.15.83:0/4292730408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c488c5f0 con 0x55d9c44be390 2024-09-21T03:18:19.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0dffff640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c4883ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0deffd640 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4031440 tx=0x7fc0d402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fe1482dff14f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.560+0000 7fc0c57fa640 1 -- 172.21.15.83:0/4292730408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4035040 con 0x55d9c44be390 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.561+0000 7fc0c57fa640 1 -- 172.21.15.83:0/4292730408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40054f0 con 0x55d9c44be390 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.561+0000 7fc0c57fa640 1 -- 172.21.15.83:0/4292730408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004c50 con 0x55d9c44be390 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- 172.21.15.83:0/4292730408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 msgr2=0x55d9c4883760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4031440 tx=0x7fc0d402ab10 comp rx=0 tx=0).stop 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- 172.21.15.83:0/4292730408 shutdown_connections 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44df070 0x55d9c4883ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c4883760 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 --2- 172.21.15.83:0/4292730408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44b61b0 0x55d9c4887a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- 172.21.15.83:0/4292730408 >> 172.21.15.83:0/4292730408 conn(0x55d9c44d3460 msgr2=0x55d9c48872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- 172.21.15.83:0/4292730408 shutdown_connections 2024-09-21T03:18:19.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- 172.21.15.83:0/4292730408 wait complete. 2024-09-21T03:18:19.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.562+0000 7fc0ec681740 1 -- start start 2024-09-21T03:18:19.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.563+0000 7fc0ec681740 1 Processor -- start 2024-09-21T03:18:19.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44b61b0 0x55d9c48a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c48a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ae8d0 con 0x55d9c44be390 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ae200 con 0x55d9c44b61b0 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ab3e0 con 0x55d9c44df070 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.564+0000 7fc0ec681740 10 client.0 _ll_register_callbacks cb 0x7fc0d8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55470/0 (socket says 172.21.15.83:55470) 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 -- 172.21.15.83:0/3697103344 learned_addr learned my addr 172.21.15.83:0/3697103344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0deffd640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c48a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44b61b0 0x55d9c48a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 -- 172.21.15.83:0/3697103344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44b61b0 msgr2=0x55d9c48a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44b61b0 0x55d9c48a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 -- 172.21.15.83:0/3697103344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 msgr2=0x55d9c48a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c48a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0dffff640 1 -- 172.21.15.83:0/3697103344 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c48a5110 con 0x55d9c44df070 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.565+0000 7fc0deffd640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c48a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0dffff640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8010f30 tx=0x7fc0d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663f948c17664598 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d802e060 con 0x55d9c44df070 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0d801ebb0 con 0x55d9c44df070 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d40054f0 con 0x55d9c44df070 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0d801edf0 con 0x55d9c44df070 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0ec681740 10 client.25276 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:19.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.566+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9c4916800 con 0x55d9c44df070 2024-09-21T03:18:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.567+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d80338e0 con 0x55d9c44df070 2024-09-21T03:18:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc0d801d630 con 0x55d9c44df070 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 10 client.25276 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 10 client.25276 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 20 client.25276 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c4472da0 0x55d9c4918ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 4 client.25276 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc0d40054f0 con 0x55d9c4472da0 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c4472da0 0x55d9c4918ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0df7fe640 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c4472da0 0x55d9c4918ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35400ddbab9c4f84 server_cookie=892b28873c392444 in_seq=0 out_seq=0 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0beffd640 10 client.25276 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0beffd640 1 -- 172.21.15.83:0/3697103344 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc0d40054f0 con 0x55d9c4472da0 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0beffd640 10 client.25276 handle_command_reply: tid=0 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.568+0000 7fc0ec681740 1 client.25276 shutdown 2024-09-21T03:18:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 msgr2=0x55d9c48a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8010f30 tx=0x7fc0d8004a40 comp rx=0 tx=0).stop 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 shutdown_connections 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9c4472da0 0x55d9c4918ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0beffd640 0 client.25276 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9c44df070 0x55d9c48a4b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9c44be390 0x55d9c48a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 --2- 172.21.15.83:0/3697103344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9c44b61b0 0x55d9c48a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 >> 172.21.15.83:0/3697103344 conn(0x55d9c44d3460 msgr2=0x55d9c4887bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 shutdown_connections 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.569+0000 7fc0ec681740 1 -- 172.21.15.83:0/3697103344 wait complete. 2024-09-21T03:18:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.570+0000 7fc0ec681740 20 client.25276 trim_cache size 0 max 16384 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 msgr2=0x7fc0c80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 0x7fc0c80b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8011830 tx=0x7fc0d8009500 comp rx=0 tx=0).stop 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 msgr2=0x7fc0e0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e9f0 tx=0x7fc0d000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 shutdown_connections 2024-09-21T03:18:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0c80b6700 0x7fc0c80b8c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0108d10 0x7fc0e019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e01083c0 0x7fc0e01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 --2- 172.21.15.83:0/2120441743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0e0102a80 0x7fc0e0199350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 >> 172.21.15.83:0/2120441743 conn(0x7fc0e0079740 msgr2=0x7fc0e0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 shutdown_connections 2024-09-21T03:18:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:19.571+0000 7fc0e610d640 1 -- 172.21.15.83:0/2120441743 wait complete. 2024-09-21T03:18:20.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:18:20.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:20 smithi193 ceph-mon[35224]: pgmap v375: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-09-21T03:18:20.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:20 smithi083 ceph-mon[31219]: pgmap v375: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-09-21T03:18:20.670 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:20 smithi142 ceph-mon[34512]: pgmap v375: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-09-21T03:18:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 60.0 s, 3583.8 tps, lat 27.837 ms stddev 28.388 2024-09-21T03:18:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 -- 172.21.15.83:0/764720019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c109e00 msgr2=0x7f500c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 --2- 172.21.15.83:0/764720019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c109e00 0x7f500c110630 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5004009ca0 tx=0x7f500402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 -- 172.21.15.83:0/764720019 shutdown_connections 2024-09-21T03:18:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 --2- 172.21.15.83:0/764720019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c109e00 0x7f500c110630 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 --2- 172.21.15.83:0/764720019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 --2- 172.21.15.83:0/764720019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c101f70 0x7f500c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 -- 172.21.15.83:0/764720019 >> 172.21.15.83:0/764720019 conn(0x7f500c077c60 msgr2=0x7f500c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 -- 172.21.15.83:0/764720019 shutdown_connections 2024-09-21T03:18:21.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.221+0000 7f5010a72640 1 -- 172.21.15.83:0/764720019 wait complete. 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.222+0000 7f5010a72640 1 Processor -- start 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.223+0000 7f5010a72640 1 -- start start 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.223+0000 7f5010a72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 0x7f500c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.223+0000 7f5010a72640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.223+0000 7f500a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 0x7f500c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.223+0000 7f5010a72640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c109e00 0x7f500c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f500a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 0x7f500c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46038/0 (socket says 172.21.15.83:46038) 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5010a72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f500c113ac0 con 0x7f500c101f70 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5010a72640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f500c113940 con 0x7f500c1094b0 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5010a72640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f500c113c40 con 0x7f500c109e00 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60090/0 (socket says 172.21.15.83:60090) 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 -- 172.21.15.83:0/3414244771 learned_addr learned my addr 172.21.15.83:0/3414244771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:21.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f500ad9d640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c109e00 0x7f500c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 -- 172.21.15.83:0/3414244771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c109e00 msgr2=0x7f500c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c109e00 0x7f500c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 -- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 msgr2=0x7f500c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 0x7f500c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 -- 172.21.15.83:0/3414244771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c1a0460 con 0x7f500c1094b0 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5009d9b640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f500000e9f0 tx=0x7f500000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f4ffb7fe640 1 -- 172.21.15.83:0/3414244771 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500000ccc0 con 0x7f500c1094b0 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f4ffb7fe640 1 -- 172.21.15.83:0/3414244771 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50000108e0 con 0x7f500c1094b0 2024-09-21T03:18:21.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.224+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c1a0750 con 0x7f500c1094b0 2024-09-21T03:18:21.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.225+0000 7f4ffb7fe640 1 -- 172.21.15.83:0/3414244771 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000018860 con 0x7f500c1094b0 2024-09-21T03:18:21.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.225+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c06cde0 con 0x7f500c1094b0 2024-09-21T03:18:21.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.228+0000 7f4ffb7fe640 1 -- 172.21.15.83:0/3414244771 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5000010430 con 0x7f500c1094b0 2024-09-21T03:18:21.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 Processor -- start 2024-09-21T03:18:21.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 -- start start 2024-09-21T03:18:21.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567c670dea0 con 0x5567c63cf390 2024-09-21T03:18:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567c66fd2d0 con 0x5567c63c71b0 2024-09-21T03:18:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.229+0000 7f5016fe6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567c67010e0 con 0x5567c63f0070 2024-09-21T03:18:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55504/0 (socket says 172.21.15.83:55504) 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 -- 172.21.15.83:0/1344041409 learned_addr learned my addr 172.21.15.83:0/1344041409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500a59c640 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f5009d9b640 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f4ffb7fe640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 0x7f4ff40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 -- 172.21.15.83:0/1344041409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 msgr2=0x5567c67952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 -- 172.21.15.83:0/1344041409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 msgr2=0x5567c67a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 -- 172.21.15.83:0/1344041409 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c645b650 con 0x5567c63f0070 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.230+0000 7f500ad9d640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 0x7f4ff40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.231+0000 7f500ad9d640 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5004009480 tx=0x7f5004009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251df5c40f6d0007 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.231+0000 7f4ff8ff9640 1 -- 172.21.15.83:0/1344041409 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004042040 con 0x5567c63f0070 2024-09-21T03:18:21.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.231+0000 7f4ff8ff9640 1 -- 172.21.15.83:0/1344041409 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5004036070 con 0x5567c63f0070 2024-09-21T03:18:21.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.231+0000 7f4ff8ff9640 1 -- 172.21.15.83:0/1344041409 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50040042a0 con 0x5567c63f0070 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.231+0000 7f4ffb7fe640 1 -- 172.21.15.83:0/3414244771 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f50000e7930 con 0x7f500c1094b0 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.232+0000 7f5016fe6740 1 -- 172.21.15.83:0/1344041409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 msgr2=0x5567c6794d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.232+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5004009480 tx=0x7f5004009800 comp rx=0 tx=0).stop 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.232+0000 7f500ad9d640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 0x7f4ff40b8c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5004009a60 tx=0x7f5004003f30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.233+0000 7f5016fe6740 1 -- 172.21.15.83:0/1344041409 shutdown_connections 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.233+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.233+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.233+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1344041409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c6794d20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.233+0000 7f5016fe6740 1 -- 172.21.15.83:0/1344041409 >> 172.21.15.83:0/1344041409 conn(0x5567c63e4460 msgr2=0x5567c6797240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.234+0000 7f5016fe6740 1 -- 172.21.15.83:0/1344041409 shutdown_connections 2024-09-21T03:18:21.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.234+0000 7f5016fe6740 1 -- 172.21.15.83:0/1344041409 wait complete. 2024-09-21T03:18:21.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 -- start start 2024-09-21T03:18:21.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 Processor -- start 2024-09-21T03:18:21.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c67b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567c655e700 con 0x5567c63cf390 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567c6698160 con 0x5567c63c71b0 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567c67a8170 con 0x5567c63f0070 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5016fe6740 10 client.0 _ll_register_callbacks cb 0x5567c66b19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f500a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f500a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60118/0 (socket says 172.21.15.83:60118) 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f500a59c640 1 -- 172.21.15.83:0/815401232 learned_addr learned my addr 172.21.15.83:0/815401232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.235+0000 7f500ad9d640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c67b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 -- 172.21.15.83:0/815401232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 msgr2=0x5567c67b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c67b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 -- 172.21.15.83:0/815401232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 msgr2=0x5567c67abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 -- 172.21.15.83:0/815401232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c66c5c90 con 0x5567c63c71b0 2024-09-21T03:18:21.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500ad9d640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c67b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:21.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f5009d9b640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:21.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f500a59c640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc013710 tx=0x7f4ffc013b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798eb5a72a36fdda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.236+0000 7f4ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:21.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.237+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc020060 con 0x5567c63c71b0 2024-09-21T03:18:21.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.237+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4ffc011dd0 con 0x5567c63c71b0 2024-09-21T03:18:21.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.237+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5004036070 con 0x5567c63c71b0 2024-09-21T03:18:21.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.238+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ffc00f450 con 0x5567c63c71b0 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.238+0000 7f5016fe6740 10 client.25325 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5567c67b9f00 con 0x5567c63c71b0 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ffc00faa0 con 0x5567c63c71b0 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4ffc01a030 con 0x5567c63c71b0 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 10 client.25325 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 10 client.25325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 20 client.25325 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c63ae9f0 0x5567c67bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 4 client.25325 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5004036070 con 0x5567c63ae9f0 2024-09-21T03:18:21.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.239+0000 7f5009d9b640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c63ae9f0 0x5567c67bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.240+0000 7f4ff1ffb640 10 client.25325 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:18:21.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.240+0000 7f5009d9b640 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c63ae9f0 0x5567c67bd480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdd3c1745b37eccb server_cookie=444c333cbc4011a4 in_seq=0 out_seq=0 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.241+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/815401232 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5004036070 con 0x5567c63ae9f0 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.241+0000 7f4ff1ffb640 10 client.25325 handle_command_reply: tid=0 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.241+0000 7f5016fe6740 1 client.25325 shutdown 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.241+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 msgr2=0x5567c67ab900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.241+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc013710 tx=0x7f4ffc013b10 comp rx=0 tx=0).stop 2024-09-21T03:18:21.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 shutdown_connections 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c63ae9f0 0x5567c67bd480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63f0070 0x5567c67b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63cf390 0x5567c67abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 --2- 172.21.15.83:0/815401232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c67ab900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 >> 172.21.15.83:0/815401232 conn(0x5567c63e4460 msgr2=0x5567c67961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f4ff1ffb640 0 client.25325 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:18:21.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 shutdown_connections 2024-09-21T03:18:21.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 1 -- 172.21.15.83:0/815401232 wait complete. 2024-09-21T03:18:21.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.242+0000 7f5016fe6740 20 client.25325 trim_cache size 0 max 16384 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.259+0000 7f5016fe6740 1 Processor -- start 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.259+0000 7f5016fe6740 1 -- start start 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c6796d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 0x5567c67a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f500c073440 con 0x5567c63f0070 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f500c0750a0 con 0x5567c63c71b0 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5016fe6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f500c073b40 con 0x5567c63cf390 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f500ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 0x5567c67a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f500a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c6796d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55520/0 (socket says 172.21.15.83:55520) 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f500ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 0x5567c67a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46072/0 (socket says 172.21.15.83:46072) 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f500a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c6796d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60124/0 (socket says 172.21.15.83:60124) 2024-09-21T03:18:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 -- 172.21.15.83:0/1863976543 learned_addr learned my addr 172.21.15.83:0/1863976543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 -- 172.21.15.83:0/1863976543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 msgr2=0x5567c6796d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c6796d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 -- 172.21.15.83:0/1863976543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 msgr2=0x5567c67a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 0x5567c67a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 -- 172.21.15.83:0/1863976543 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c6797b30 con 0x5567c63cf390 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.260+0000 7f5009d9b640 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50000aaa10 tx=0x7f50000096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01af5f513aa293d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.261+0000 7f4ff3fff640 1 -- 172.21.15.83:0/1863976543 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000adea0 con 0x5567c63cf390 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.261+0000 7f4ff3fff640 1 -- 172.21.15.83:0/1863976543 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50000b2280 con 0x5567c63cf390 2024-09-21T03:18:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.261+0000 7f4ff3fff640 1 -- 172.21.15.83:0/1863976543 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000b3410 con 0x5567c63cf390 2024-09-21T03:18:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- 172.21.15.83:0/1863976543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 msgr2=0x5567c67972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50000aaa10 tx=0x7f50000096a0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- 172.21.15.83:0/1863976543 shutdown_connections 2024-09-21T03:18:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63f0070 0x5567c67a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67972d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 --2- 172.21.15.83:0/1863976543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63c71b0 0x5567c6796d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- 172.21.15.83:0/1863976543 >> 172.21.15.83:0/1863976543 conn(0x5567c63e4460 msgr2=0x5567c67a6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- 172.21.15.83:0/1863976543 shutdown_connections 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- 172.21.15.83:0/1863976543 wait complete. 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.262+0000 7f5016fe6740 1 -- start start 2024-09-21T03:18:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.263+0000 7f5016fe6740 1 Processor -- start 2024-09-21T03:18:21.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.263+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63c71b0 0x5567c67b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.263+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.263+0000 7f5016fe6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63f0070 0x5567c67b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.263+0000 7f5016fe6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f500c074910 con 0x5567c63c71b0 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5016fe6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f500c074240 con 0x5567c63f0070 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f500ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63f0070 0x5567c67b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55530/0 (socket says 172.21.15.83:55530) 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 -- 172.21.15.83:0/2299274752 learned_addr learned my addr 172.21.15.83:0/2299274752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f500c071440 con 0x5567c63cf390 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5016fe6740 10 client.0 _ll_register_callbacks cb 0x7f4ffc0096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 -- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63f0070 msgr2=0x5567c67b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:21.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63f0070 0x5567c67b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 -- 172.21.15.83:0/2299274752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63c71b0 msgr2=0x5567c67b7ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63c71b0 0x5567c67b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.264+0000 7f5009d9b640 1 -- 172.21.15.83:0/2299274752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c67b3c70 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f5009d9b640 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f500000cf30 tx=0x7f500000c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6aad3d377f09b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f4ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000ac7e0 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50000b2280 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f50000a90f0 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5000010720 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f5016fe6740 10 client.25294 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5567c6827d30 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.265+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50000b2280 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f50000e7ce0 con 0x5567c63cf390 2024-09-21T03:18:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 10 client.25294 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 10 client.25294 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 20 client.25294 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c6383da0 0x5567c682a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 4 client.25294 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567c670fdf0 con 0x5567c6383da0 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f500a59c640 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c6383da0 0x5567c682a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f4ff1ffb640 10 client.25294 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.266+0000 7f500a59c640 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c6383da0 0x5567c682a500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2ca6afb43456a61 server_cookie=5c4e1592239d9921 in_seq=0 out_seq=0 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f4ff1ffb640 1 -- 172.21.15.83:0/2299274752 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567c670fdf0 con 0x5567c6383da0 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f4ff1ffb640 10 client.25294 handle_command_reply: tid=0 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 client.25294 shutdown 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 msgr2=0x5567c67b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f500000cf30 tx=0x7f500000c7d0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 shutdown_connections 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5567c6383da0 0x5567c682a500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c63f0070 0x5567c67b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c63cf390 0x5567c67b30c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f4ff1ffb640 0 client.25294 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 --2- 172.21.15.83:0/2299274752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567c63c71b0 0x5567c67b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.267+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 >> 172.21.15.83:0/2299274752 conn(0x5567c63e4460 msgr2=0x5567c6793750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.268+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 shutdown_connections 2024-09-21T03:18:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.268+0000 7f5016fe6740 1 -- 172.21.15.83:0/2299274752 wait complete. 2024-09-21T03:18:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.268+0000 7f5016fe6740 20 client.25294 trim_cache size 0 max 16384 2024-09-21T03:18:21.273 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 msgr2=0x7f4ff40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 0x7f4ff40b8c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5004009a60 tx=0x7f5004003f30 comp rx=0 tx=0).stop 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 msgr2=0x7f500c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f500000e9f0 tx=0x7f500000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 shutdown_connections 2024-09-21T03:18:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4ff40b6780 0x7f4ff40b8c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f500c109e00 0x7f500c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f500c1094b0 0x7f500c19aa90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 --2- 172.21.15.83:0/3414244771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f500c101f70 0x7f500c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 >> 172.21.15.83:0/3414244771 conn(0x7f500c077c60 msgr2=0x7f500c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 shutdown_connections 2024-09-21T03:18:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:21.269+0000 7f5010a72640 1 -- 172.21.15.83:0/3414244771 wait complete. 2024-09-21T03:18:22.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:18:22.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:22 smithi083 ceph-mon[31219]: pgmap v376: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T03:18:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:22 smithi193 ceph-mon[35224]: pgmap v376: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T03:18:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:22 smithi142 ceph-mon[34512]: pgmap v376: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 705 op/s 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 -- 172.21.15.83:0/3048408823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c109e10 msgr2=0x7f8d6c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 --2- 172.21.15.83:0/3048408823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c109e10 0x7f8d6c1106d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d6402a8d0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 -- 172.21.15.83:0/3048408823 shutdown_connections 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 --2- 172.21.15.83:0/3048408823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c109e10 0x7f8d6c1106d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 --2- 172.21.15.83:0/3048408823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 --2- 172.21.15.83:0/3048408823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c102010 0x7f8d6c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 -- 172.21.15.83:0/3048408823 >> 172.21.15.83:0/3048408823 conn(0x7f8d6c077c60 msgr2=0x7f8d6c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 -- 172.21.15.83:0/3048408823 shutdown_connections 2024-09-21T03:18:22.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.967+0000 7f8d70e77640 1 -- 172.21.15.83:0/3048408823 wait complete. 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 Processor -- start 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 -- start start 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 0x7f8d6c102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c109e10 0x7f8d6c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c113b00 con 0x7f8d6c102010 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c113980 con 0x7f8d6c109e10 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d70e77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c113c80 con 0x7f8d6c1094c0 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.968+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 0x7f8d6c102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 0x7f8d6c102fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46088/0 (socket says 172.21.15.83:46088) 2024-09-21T03:18:22.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d6ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c109e10 0x7f8d6c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d6a59c640 1 -- 172.21.15.83:0/1327748454 learned_addr learned my addr 172.21.15.83:0/1327748454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 -- 172.21.15.83:0/1327748454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c109e10 msgr2=0x7f8d6c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c109e10 0x7f8d6c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 -- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 msgr2=0x7f8d6c102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 0x7f8d6c102fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 -- 172.21.15.83:0/1327748454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c1a8730 con 0x7f8d6c1094c0 2024-09-21T03:18:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.969+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1034f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c075ab0 tx=0x7f8d6000b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.970+0000 7f8d5b7fe640 1 -- 172.21.15.83:0/1327748454 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60015020 con 0x7f8d6c1094c0 2024-09-21T03:18:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.970+0000 7f8d5b7fe640 1 -- 172.21.15.83:0/1327748454 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d60011ce0 con 0x7f8d6c1094c0 2024-09-21T03:18:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.970+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c1a8a20 con 0x7f8d6c1094c0 2024-09-21T03:18:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.970+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c1a8f80 con 0x7f8d6c1094c0 2024-09-21T03:18:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.971+0000 7f8d5b7fe640 1 -- 172.21.15.83:0/1327748454 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60010dd0 con 0x7f8d6c1094c0 2024-09-21T03:18:22.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.974+0000 7f8d773eb740 1 Processor -- start 2024-09-21T03:18:22.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.974+0000 7f8d5b7fe640 1 -- 172.21.15.83:0/1327748454 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8d60018020 con 0x7f8d6c1094c0 2024-09-21T03:18:22.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 -- start start 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 0x5652027906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c01b0 0x565202790be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x565202791120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565202706ea0 con 0x5652023c8390 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5652026f62d0 con 0x5652023e9070 2024-09-21T03:18:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d773eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652026fa0e0 con 0x5652023c01b0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x565202791120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d6ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 0x5652027906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d6ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 0x5652027906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60168/0 (socket says 172.21.15.83:60168) 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d6ad9d640 1 -- 172.21.15.83:0/745834443 learned_addr learned my addr 172.21.15.83:0/745834443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c01b0 0x565202790be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.975+0000 7f8d5b7fe640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 0x7f8d540b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 0x7f8d540b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 -- 172.21.15.83:0/745834443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c01b0 msgr2=0x565202790be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c01b0 0x565202790be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 -- 172.21.15.83:0/745834443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 msgr2=0x5652027906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 0x5652027906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d5b7fe640 1 -- 172.21.15.83:0/1327748454 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f8d600e6970 con 0x7f8d6c1094c0 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 -- 172.21.15.83:0/745834443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565202454650 con 0x5652023c8390 2024-09-21T03:18:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x565202791120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d600e4140 tx=0x7f8d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0bb0b7cad30380 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 0x7f8d540b8ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d6402a8d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.976+0000 7f8d58ff9640 1 -- 172.21.15.83:0/745834443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600abfb0 con 0x5652023c8390 2024-09-21T03:18:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.977+0000 7f8d58ff9640 1 -- 172.21.15.83:0/745834443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600ac350 con 0x5652023c8390 2024-09-21T03:18:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.977+0000 7f8d58ff9640 1 -- 172.21.15.83:0/745834443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600b6260 con 0x5652023c8390 2024-09-21T03:18:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 -- 172.21.15.83:0/745834443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 msgr2=0x565202791120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x565202791120 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d600e4140 tx=0x7f8d600046c0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 -- 172.21.15.83:0/745834443 shutdown_connections 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x565202791120 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c01b0 0x565202790be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 --2- 172.21.15.83:0/745834443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023e9070 0x5652027906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 -- 172.21.15.83:0/745834443 >> 172.21.15.83:0/745834443 conn(0x5652023dd460 msgr2=0x56520278e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 -- 172.21.15.83:0/745834443 shutdown_connections 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.978+0000 7f8d773eb740 1 -- 172.21.15.83:0/745834443 wait complete. 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.979+0000 7f8d773eb740 1 -- start start 2024-09-21T03:18:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.979+0000 7f8d773eb740 1 Processor -- start 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 0x56520279bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x56520279c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 0x56520279bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x56520279c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 0x5652027a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x56520279c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46104/0 (socket says 172.21.15.83:46104) 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 0x56520279bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60172/0 (socket says 172.21.15.83:60172) 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d69d9b640 1 -- 172.21.15.83:0/2878384394 learned_addr learned my addr 172.21.15.83:0/2878384394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565202557700 con 0x5652023c8390 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565202691160 con 0x5652023c01b0 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.980+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56520279ee20 con 0x5652023e9070 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 0x5652027a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d773eb740 10 client.0 _ll_register_callbacks cb 0x5652026aa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 -- 172.21.15.83:0/2878384394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 msgr2=0x56520279bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 0x56520279bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 -- 172.21.15.83:0/2878384394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 msgr2=0x56520279c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x56520279c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652026bec90 con 0x5652023e9070 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 0x5652027a8170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d6402adf0 tx=0x7f8d640095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea46256a1d6fd115 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.981+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004450 con 0x5652023e9070 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.982+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f8d640045f0 con 0x5652023e9070 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.982+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d600ac350 con 0x5652023e9070 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.982+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d640047f0 con 0x5652023e9070 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.982+0000 7f8d773eb740 10 client.25306 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:22.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.982+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5652027aab50 con 0x5652023e9070 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600ac350 con 0x5652023e9070 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8d6404a040 con 0x5652023e9070 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 10 client.25306 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 10 client.25306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 20 client.25306 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023a79f0 0x5652027ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 4 client.25306 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.983+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565202708df0 con 0x5652023a79f0 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.984+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023a79f0 0x5652027ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:22.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.984+0000 7f8d51ffb640 10 client.25306 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.984+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023a79f0 0x5652027ae1d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b82f4c3771b28e2 server_cookie=6efd2c5fa2e56c32 in_seq=0 out_seq=0 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d51ffb640 1 -- 172.21.15.83:0/2878384394 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565202708df0 con 0x5652023a79f0 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d51ffb640 10 client.25306 handle_command_reply: tid=0 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d773eb740 1 client.25306 shutdown 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 msgr2=0x5652027a8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 0x5652027a8170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d6402adf0 tx=0x7f8d640095c0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.985+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 shutdown_connections 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023a79f0 0x5652027ae1d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d51ffb640 0 client.25306 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:18:22.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023e9070 0x5652027a8170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c8390 0x56520279c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 --2- 172.21.15.83:0/2878384394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023c01b0 0x56520279bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 >> 172.21.15.83:0/2878384394 conn(0x5652023dd460 msgr2=0x5652027916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 shutdown_connections 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 1 -- 172.21.15.83:0/2878384394 wait complete. 2024-09-21T03:18:22.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:22.986+0000 7f8d773eb740 20 client.25306 trim_cache size 0 max 16384 2024-09-21T03:18:23.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.004+0000 7f8d773eb740 1 Processor -- start 2024-09-21T03:18:23.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.004+0000 7f8d773eb740 1 -- start start 2024-09-21T03:18:23.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c00e620 con 0x5652023c01b0 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60174/0 (socket says 172.21.15.83:60174) 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d6a59c640 1 -- 172.21.15.83:0/253117292 learned_addr learned my addr 172.21.15.83:0/253117292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c010280 con 0x5652023a79f0 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.005+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c00ed20 con 0x5652023c8390 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 -- 172.21.15.83:0/253117292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 msgr2=0x5652027a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 -- 172.21.15.83:0/253117292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 msgr2=0x5652027a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 -- 172.21.15.83:0/253117292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56520278ef00 con 0x5652023a79f0 2024-09-21T03:18:23.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:23.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.006+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c104a00 tx=0x7f8d5c00a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755c3fd12154260a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:23.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.007+0000 7f8d53fff640 1 -- 172.21.15.83:0/253117292 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c01e6c0 con 0x5652023a79f0 2024-09-21T03:18:23.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.007+0000 7f8d53fff640 1 -- 172.21.15.83:0/253117292 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c01ed10 con 0x5652023a79f0 2024-09-21T03:18:23.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.007+0000 7f8d53fff640 1 -- 172.21.15.83:0/253117292 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c024e20 con 0x5652023a79f0 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 msgr2=0x56520279c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c104a00 tx=0x7f8d5c00a380 comp rx=0 tx=0).stop 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 shutdown_connections 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 --2- 172.21.15.83:0/253117292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x56520279c220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.008+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 >> 172.21.15.83:0/253117292 conn(0x5652023dd460 msgr2=0x56520279bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.009+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 shutdown_connections 2024-09-21T03:18:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.009+0000 7f8d773eb740 1 -- 172.21.15.83:0/253117292 wait complete. 2024-09-21T03:18:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 -- start start 2024-09-21T03:18:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 Processor -- start 2024-09-21T03:18:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x5652027a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c00faf0 con 0x5652023c01b0 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c00f420 con 0x5652023a79f0 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c00c600 con 0x5652023c8390 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.010+0000 7f8d773eb740 10 client.0 _ll_register_callbacks cb 0x7f8d6402f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x5652027a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46118/0 (socket says 172.21.15.83:46118) 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d69d9b640 1 -- 172.21.15.83:0/1239360756 learned_addr learned my addr 172.21.15.83:0/1239360756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:23.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6ad9d640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 -- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 msgr2=0x5652027a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 -- 172.21.15.83:0/1239360756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 msgr2=0x5652027a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.011+0000 7f8d6a59c640 1 -- 172.21.15.83:0/1239360756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565202817c00 con 0x5652023a79f0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.012+0000 7f8d6a59c640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x5652027a00e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c104a00 tx=0x7f8d5c024620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9136f16717a71a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.012+0000 7f8d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.012+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c032030 con 0x5652023a79f0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.012+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f8d5c024cf0 con 0x5652023a79f0 2024-09-21T03:18:23.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.012+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5c01ed10 con 0x5652023a79f0 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d5c005050 con 0x5652023a79f0 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 10 client.25346 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565202818eb0 con 0x5652023a79f0 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c0056a0 con 0x5652023a79f0 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8d5c0046c0 con 0x5652023a79f0 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 10 client.25346 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 10 client.25346 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 20 client.25346 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023e9070 0x56520281b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 4 client.25346 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8d5c01ed10 con 0x5652023e9070 2024-09-21T03:18:23.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.013+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023e9070 0x56520281b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.014+0000 7f8d51ffb640 10 client.25346 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.014+0000 7f8d69d9b640 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023e9070 0x56520281b620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84f1db3c311b6741 server_cookie=bc579c2c4640b8bf in_seq=0 out_seq=0 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.014+0000 7f8d51ffb640 1 -- 172.21.15.83:0/1239360756 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8d5c01ed10 con 0x5652023e9070 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.014+0000 7f8d51ffb640 10 client.25346 handle_command_reply: tid=0 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.014+0000 7f8d773eb740 1 client.25346 shutdown 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.015+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 msgr2=0x5652027a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.015+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x5652027a00e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c104a00 tx=0x7f8d5c024620 comp rx=0 tx=0).stop 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 shutdown_connections 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5652023e9070 0x56520281b620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652023c8390 0x5652027a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652023c01b0 0x5652027a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 --2- 172.21.15.83:0/1239360756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652023a79f0 0x5652027a00e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 >> 172.21.15.83:0/1239360756 conn(0x5652023dd460 msgr2=0x5652027916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d51ffb640 0 client.25346 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 shutdown_connections 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 1 -- 172.21.15.83:0/1239360756 wait complete. 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.016+0000 7f8d773eb740 20 client.25346 trim_cache size 0 max 16384 2024-09-21T03:18:23.019 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:18:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 msgr2=0x7f8d540b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 0x7f8d540b8ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d6402a8d0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 msgr2=0x7f8d6c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1034f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c075ab0 tx=0x7f8d6000b850 comp rx=0 tx=0).stop 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 shutdown_connections 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d540b67d0 0x7f8d540b8ce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d6c109e10 0x7f8d6c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d6c1094c0 0x7f8d6c1034f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 --2- 172.21.15.83:0/1327748454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6c102010 0x7f8d6c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.017+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 >> 172.21.15.83:0/1327748454 conn(0x7f8d6c077c60 msgr2=0x7f8d6c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.018+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 shutdown_connections 2024-09-21T03:18:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:23.018+0000 7f8d70e77640 1 -- 172.21.15.83:0/1327748454 wait complete. 2024-09-21T03:18:23.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:18:24.092 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:24.516 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:24 smithi083 ceph-mon[31219]: pgmap v377: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 835 op/s 2024-09-21T03:18:24.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:24 smithi193 ceph-mon[35224]: pgmap v377: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 835 op/s 2024-09-21T03:18:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 -- 172.21.15.83:0/1259463872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac108cf0 msgr2=0x7fb3ac10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/1259463872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac10f520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ca0 tx=0x7fb3a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 -- 172.21.15.83:0/1259463872 shutdown_connections 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/1259463872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac10f520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/1259463872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac1083a0 0x7fb3ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/1259463872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3ac102a60 0x7fb3ac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.640+0000 7fb3b47ac640 1 -- 172.21.15.83:0/1259463872 >> 172.21.15.83:0/1259463872 conn(0x7fb3ac079730 msgr2=0x7fb3ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.641+0000 7fb3b47ac640 1 -- 172.21.15.83:0/1259463872 shutdown_connections 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.641+0000 7fb3b47ac640 1 -- 172.21.15.83:0/1259463872 wait complete. 2024-09-21T03:18:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.642+0000 7fb3b47ac640 1 Processor -- start 2024-09-21T03:18:24.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.642+0000 7fb3b47ac640 1 -- start start 2024-09-21T03:18:24.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.642+0000 7fb3b47ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac102a60 0x7fb3ac19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.642+0000 7fb3b47ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3ac1083a0 0x7fb3ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b47ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b47ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac111860 con 0x7fb3ac1083a0 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b47ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1116e0 con 0x7fb3ac102a60 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b47ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1119e0 con 0x7fb3ac108cf0 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55578/0 (socket says 172.21.15.83:55578) 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 -- 172.21.15.83:0/2661829328 learned_addr learned my addr 172.21.15.83:0/2661829328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2548640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac102a60 0x7fb3ac19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 -- 172.21.15.83:0/2661829328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac102a60 msgr2=0x7fb3ac19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac102a60 0x7fb3ac19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 -- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3ac1083a0 msgr2=0x7fb3ac19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3ac1083a0 0x7fb3ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.643+0000 7fb3b2d49640 1 -- 172.21.15.83:0/2661829328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac1a25f0 con 0x7fb3ac108cf0 2024-09-21T03:18:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3b2d49640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009430 tx=0x7fb3a40316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3a37fe640 1 -- 172.21.15.83:0/2661829328 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4003b70 con 0x7fb3ac108cf0 2024-09-21T03:18:24.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac1a2880 con 0x7fb3ac108cf0 2024-09-21T03:18:24.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3a37fe640 1 -- 172.21.15.83:0/2661829328 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a4031910 con 0x7fb3ac108cf0 2024-09-21T03:18:24.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3a37fe640 1 -- 172.21.15.83:0/2661829328 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a402f9e0 con 0x7fb3ac108cf0 2024-09-21T03:18:24.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.644+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac1a9040 con 0x7fb3ac108cf0 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.648+0000 7fb3bad20740 1 Processor -- start 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.648+0000 7fb3a37fe640 1 -- 172.21.15.83:0/2661829328 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb3a4004040 con 0x7fb3ac108cf0 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.648+0000 7fb3bad20740 1 -- start start 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f949070 0x56186fcf8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56186fc66ea0 con 0x56186f949070 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56186fc562d0 con 0x56186f9201b0 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3bad20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56186fc5a0e0 con 0x56186f928390 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60210/0 (socket says 172.21.15.83:60210) 2024-09-21T03:18:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 -- 172.21.15.83:0/4016231121 learned_addr learned my addr 172.21.15.83:0/4016231121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f949070 0x56186fcf8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 -- 172.21.15.83:0/4016231121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 msgr2=0x56186fcf9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 -- 172.21.15.83:0/4016231121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f949070 msgr2=0x56186fcf8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f949070 0x56186fcf8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.649+0000 7fb3b2548640 1 -- 172.21.15.83:0/4016231121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186f9b4650 con 0x56186f9201b0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.650+0000 7fb3a37fe640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 0x7fb3980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.650+0000 7fb3a37fe640 1 -- 172.21.15.83:0/2661829328 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb3a4108b00 con 0x7fb3ac108cf0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.651+0000 7fb3b2548640 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bce0 tx=0x7fb3a8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9b60b0b168cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.651+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 0x7fb3980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.651+0000 7fb3a0ff9640 1 -- 172.21.15.83:0/4016231121 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8015400 con 0x56186f9201b0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.651+0000 7fb3a0ff9640 1 -- 172.21.15.83:0/4016231121 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a801a070 con 0x56186f9201b0 2024-09-21T03:18:24.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.651+0000 7fb3a0ff9640 1 -- 172.21.15.83:0/4016231121 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8015400 con 0x56186f9201b0 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 -- 172.21.15.83:0/4016231121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 msgr2=0x56186fcf93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bce0 tx=0x7fb3a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 0x7fb3980b8c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0058c0 tx=0x7fb39c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 -- 172.21.15.83:0/4016231121 shutdown_connections 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fcf93f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 --2- 172.21.15.83:0/4016231121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f949070 0x56186fcf8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 -- 172.21.15.83:0/4016231121 >> 172.21.15.83:0/4016231121 conn(0x56186f93d460 msgr2=0x56186fcf6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.653+0000 7fb3bad20740 1 -- 172.21.15.83:0/4016231121 shutdown_connections 2024-09-21T03:18:24.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.654+0000 7fb3bad20740 1 -- 172.21.15.83:0/4016231121 wait complete. 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.654+0000 7fb3bad20740 1 -- start start 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.654+0000 7fb3bad20740 1 Processor -- start 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 0x56186fcf3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f949070 0x56186fcf39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56186fab7700 con 0x56186f9201b0 2024-09-21T03:18:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56186fbf1160 con 0x56186f928390 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56186fcf61e0 con 0x56186f949070 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3bad20740 10 client.0 _ll_register_callbacks cb 0x56186fc0a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46144/0 (socket says 172.21.15.83:46144) 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3b2548640 1 -- 172.21.15.83:0/2369188416 learned_addr learned my addr 172.21.15.83:0/2369188416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3b2d49640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f949070 0x56186fcf39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.655+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 0x56186fcf3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 -- 172.21.15.83:0/2369188416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f949070 msgr2=0x56186fcf39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f949070 0x56186fcf39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 -- 172.21.15.83:0/2369188416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 msgr2=0x56186fcf2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 -- 172.21.15.83:0/2369188416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186fc1ec90 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 0x56186fcf3490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac105f80 tx=0x7fb39c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2103237027caf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.656+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c010840 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a801a070 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb39c0109e0 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb39c010be0 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 10 client.25358 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56186fd1ba90 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a801a070 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb39c01e3f0 con 0x56186f928390 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 10 client.25358 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 20 client.25358 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f9079f0 0x56186fd1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 4 client.25358 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.657+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186fc68df0 con 0x56186f9079f0 2024-09-21T03:18:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.658+0000 7fb3b2548640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f9079f0 0x56186fd1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.658+0000 7fb3b2548640 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f9079f0 0x56186fd1f0a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c1388bddcb6d7e server_cookie=ca42e480b2472e9a in_seq=0 out_seq=0 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.658+0000 7fb395ffb640 10 client.25358 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb395ffb640 1 -- 172.21.15.83:0/2369188416 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56186fc68df0 con 0x56186f9079f0 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb395ffb640 10 client.25358 handle_command_reply: tid=0 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 client.25358 shutdown 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 msgr2=0x56186fcf3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 0x56186fcf3490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac105f80 tx=0x7fb39c005850 comp rx=0 tx=0).stop 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 shutdown_connections 2024-09-21T03:18:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f9079f0 0x56186fd1f0a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f949070 0x56186fcf39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f928390 0x56186fcf3490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 --2- 172.21.15.83:0/2369188416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb395ffb640 0 client.25358 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 >> 172.21.15.83:0/2369188416 conn(0x56186f93d460 msgr2=0x56186fcfbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 shutdown_connections 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.659+0000 7fb3bad20740 1 -- 172.21.15.83:0/2369188416 wait complete. 2024-09-21T03:18:24.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.660+0000 7fb3bad20740 20 client.25358 trim_cache size 0 max 16384 2024-09-21T03:18:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:24 smithi142 ceph-mon[34512]: pgmap v377: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.9 MiB/s wr, 835 op/s 2024-09-21T03:18:24.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.678+0000 7fb3bad20740 1 Processor -- start 2024-09-21T03:18:24.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.678+0000 7fb3bad20740 1 -- start start 2024-09-21T03:18:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1ad410 con 0x56186f9201b0 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1af070 con 0x56186f93d460 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3bad20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1adb10 con 0x56186f928390 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b1d47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b1d47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46154/0 (socket says 172.21.15.83:46154) 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60222/0 (socket says 172.21.15.83:60222) 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b1d47640 1 -- 172.21.15.83:0/3986518193 learned_addr learned my addr 172.21.15.83:0/3986518193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2d49640 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 -- 172.21.15.83:0/3986518193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 msgr2=0x56186fcf49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 -- 172.21.15.83:0/3986518193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 msgr2=0x56186fcf4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.679+0000 7fb3b2548640 1 -- 172.21.15.83:0/3986518193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186fcfbcd0 con 0x56186f93d460 2024-09-21T03:18:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.680+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.680+0000 7fb3b2548640 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80143c0 tx=0x7fb3a8015f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b28edb2ab7860ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.680+0000 7fb397fff640 1 -- 172.21.15.83:0/3986518193 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8015400 con 0x56186f93d460 2024-09-21T03:18:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.680+0000 7fb397fff640 1 -- 172.21.15.83:0/3986518193 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a801a070 con 0x56186f93d460 2024-09-21T03:18:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.680+0000 7fb397fff640 1 -- 172.21.15.83:0/3986518193 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003850 con 0x56186f93d460 2024-09-21T03:18:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 -- 172.21.15.83:0/3986518193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 msgr2=0x56186fcf0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80143c0 tx=0x7fb3a8015f60 comp rx=0 tx=0).stop 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 -- 172.21.15.83:0/3986518193 shutdown_connections 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f928390 0x56186fcf49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f9201b0 0x56186fcf4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 --2- 172.21.15.83:0/3986518193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f93d460 0x56186fcf0a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 -- 172.21.15.83:0/3986518193 >> 172.21.15.83:0/3986518193 conn(0x56186fd1f3c0 msgr2=0x56186fcf0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 -- 172.21.15.83:0/3986518193 shutdown_connections 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.681+0000 7fb3bad20740 1 -- 172.21.15.83:0/3986518193 wait complete. 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 -- start start 2024-09-21T03:18:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 Processor -- start 2024-09-21T03:18:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fd15440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 0x56186fd15a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f93d460 0x56186fd101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1ae8e0 con 0x56186f928390 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1ae210 con 0x56186f9201b0 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.682+0000 7fb3bad20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1ab430 con 0x56186f93d460 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b1d47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 0x56186fd15a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b1d47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 0x56186fd15a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46156/0 (socket says 172.21.15.83:46156) 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b1d47640 1 -- 172.21.15.83:0/1365166879 learned_addr learned my addr 172.21.15.83:0/1365166879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3bad20740 10 client.0 _ll_register_callbacks cb 0x7fb39c005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fd15440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 -- 172.21.15.83:0/1365166879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f93d460 msgr2=0x56186fd101f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f93d460 0x56186fd101f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 -- 172.21.15.83:0/1365166879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 msgr2=0x56186fd15a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 0x56186fd15a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 -- 172.21.15.83:0/1365166879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186fd10900 con 0x56186f9201b0 2024-09-21T03:18:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.683+0000 7fb3b2548640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fd15440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8017ff0 tx=0x7fb3a8005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33156464368f2a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.684+0000 7fb395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.684+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8036060 con 0x56186f9201b0 2024-09-21T03:18:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.684+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb3a8015400 con 0x56186f9201b0 2024-09-21T03:18:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.684+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a801a070 con 0x56186f9201b0 2024-09-21T03:18:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3a8015640 con 0x56186f9201b0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 10 client.25364 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56186fd8a3c0 con 0x56186f9201b0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a801a070 con 0x56186f9201b0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb3a802a3d0 con 0x56186f9201b0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 10 client.25364 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 20 client.25364 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f949070 0x56186fd8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 4 client.25364 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56186fc648a0 con 0x56186f949070 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.685+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f949070 0x56186fd8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb395ffb640 10 client.25364 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3b1d47640 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f949070 0x56186fd8c8e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6b50042cf8e28f server_cookie=28804f515dae0d41 in_seq=0 out_seq=0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb395ffb640 1 -- 172.21.15.83:0/1365166879 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56186fc648a0 con 0x56186f949070 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb395ffb640 10 client.25364 handle_command_reply: tid=0 2024-09-21T03:18:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3bad20740 1 client.25364 shutdown 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 msgr2=0x56186fd15440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fd15440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8017ff0 tx=0x7fb3a8005280 comp rx=0 tx=0).stop 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 shutdown_connections 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.686+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56186f949070 0x56186fd8c8e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56186f93d460 0x56186fd101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56186f928390 0x56186fd15a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 --2- 172.21.15.83:0/1365166879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56186f9201b0 0x56186fd15440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb395ffb640 0 client.25364 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 >> 172.21.15.83:0/1365166879 conn(0x56186fd1f3c0 msgr2=0x56186fcecbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 shutdown_connections 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 1 -- 172.21.15.83:0/1365166879 wait complete. 2024-09-21T03:18:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.687+0000 7fb3bad20740 20 client.25364 trim_cache size 0 max 16384 2024-09-21T03:18:24.693 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:18:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 msgr2=0x7fb3980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 0x7fb3980b8c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0058c0 tx=0x7fb39c005f90 comp rx=0 tx=0).stop 2024-09-21T03:18:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 msgr2=0x7fb3ac1a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009430 tx=0x7fb3a40316f0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 shutdown_connections 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3980b6700 0x7fb3980b8c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3ac108cf0 0x7fb3ac1a1ec0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3ac1083a0 0x7fb3ac19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 --2- 172.21.15.83:0/2661829328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3ac102a60 0x7fb3ac19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 >> 172.21.15.83:0/2661829328 conn(0x7fb3ac079730 msgr2=0x7fb3ac104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 shutdown_connections 2024-09-21T03:18:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:24.689+0000 7fb3b47ac640 1 -- 172.21.15.83:0/2661829328 wait complete. 2024-09-21T03:18:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 65.0 s, 3452.0 tps, lat 29.007 ms stddev 29.459 2024-09-21T03:18:26.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:26 smithi193 ceph-mon[35224]: pgmap v378: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-21T03:18:26.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:18:26.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:18:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:26 smithi083 ceph-mon[31219]: pgmap v378: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-21T03:18:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:26 smithi142 ceph-mon[34512]: pgmap v378: 97 pgs: 97 active+clean; 1.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-21T03:18:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.596+0000 7f3a49828640 1 -- 172.21.15.83:0/2949668319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a441094c0 msgr2=0x7f3a441098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.596+0000 7f3a49828640 1 --2- 172.21.15.83:0/2949668319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a441094c0 0x7f3a441098d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c01acb0 tx=0x7f3a3c03b900 comp rx=0 tx=0).stop 2024-09-21T03:18:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 -- 172.21.15.83:0/2949668319 shutdown_connections 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 --2- 172.21.15.83:0/2949668319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a44110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 --2- 172.21.15.83:0/2949668319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a441094c0 0x7f3a441098d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 --2- 172.21.15.83:0/2949668319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a44103b80 0x7f3a44103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 -- 172.21.15.83:0/2949668319 >> 172.21.15.83:0/2949668319 conn(0x7f3a4407b8d0 msgr2=0x7f3a4407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 -- 172.21.15.83:0/2949668319 shutdown_connections 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.597+0000 7f3a49828640 1 -- 172.21.15.83:0/2949668319 wait complete. 2024-09-21T03:18:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.598+0000 7f3a49828640 1 Processor -- start 2024-09-21T03:18:27.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 -- start start 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a441094c0 0x7f3a4410c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a4410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406f6b0 con 0x7f3a44103b80 2024-09-21T03:18:27.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406f530 con 0x7f3a44109e10 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.599+0000 7f3a49828640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406f830 con 0x7f3a441094c0 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46186/0 (socket says 172.21.15.83:46186) 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a42ffd640 1 -- 172.21.15.83:0/552418264 learned_addr learned my addr 172.21.15.83:0/552418264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a4410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a427fc640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a441094c0 0x7f3a4410c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 -- 172.21.15.83:0/552418264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a441094c0 msgr2=0x7f3a4410c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a441094c0 0x7f3a4410c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 -- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 msgr2=0x7f3a44110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 -- 172.21.15.83:0/552418264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4419dff0 con 0x7f3a44109e10 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.600+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a4410ce70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c710 tx=0x7f3a3800cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.601+0000 7f3a48826640 1 -- 172.21.15.83:0/552418264 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38004220 con 0x7f3a44109e10 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.601+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4419e1c0 con 0x7f3a44109e10 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.601+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4419e680 con 0x7f3a44109e10 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.601+0000 7f3a48826640 1 -- 172.21.15.83:0/552418264 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3800ce20 con 0x7f3a44109e10 2024-09-21T03:18:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.601+0000 7f3a48826640 1 -- 172.21.15.83:0/552418264 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38003d50 con 0x7f3a44109e10 2024-09-21T03:18:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.604+0000 7f3a4fd9c740 1 Processor -- start 2024-09-21T03:18:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.605+0000 7f3a48826640 1 -- 172.21.15.83:0/552418264 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3a3800f470 con 0x7f3a44109e10 2024-09-21T03:18:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.605+0000 7f3a4fd9c740 1 -- start start 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba7a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581cb9f0e60 con 0x5581cb6d3350 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581cb9e0290 con 0x5581cb6b2670 2024-09-21T03:18:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.606+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581cb9e40a0 con 0x5581cb6aa490 2024-09-21T03:18:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a48826640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 0x7f3a2c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba7a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba7a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55634/0 (socket says 172.21.15.83:55634) 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a42ffd640 1 -- 172.21.15.83:0/527581075 learned_addr learned my addr 172.21.15.83:0/527581075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a48826640 1 -- 172.21.15.83:0/552418264 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3a380e6fa0 con 0x7f3a44109e10 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 0x7f3a2c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 -- 172.21.15.83:0/527581075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 msgr2=0x5581cba7a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba7a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 -- 172.21.15.83:0/527581075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 msgr2=0x5581cba7a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba7a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.607+0000 7f3a427fc640 1 -- 172.21.15.83:0/527581075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581cb73e930 con 0x5581cb6b2670 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.608+0000 7f3a427fc640 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba7aeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c01ac00 tx=0x7f3a3c01a470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298c9375f218b007 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.608+0000 7f3a437fe640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 0x7f3a2c0b8c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005fd0 tx=0x7f3a38079bd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.608+0000 7f3a29ffb640 1 -- 172.21.15.83:0/527581075 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c052070 con 0x5581cb6b2670 2024-09-21T03:18:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.608+0000 7f3a29ffb640 1 -- 172.21.15.83:0/527581075 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c041ab0 con 0x5581cb6b2670 2024-09-21T03:18:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.608+0000 7f3a29ffb640 1 -- 172.21.15.83:0/527581075 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c055040 con 0x5581cb6b2670 2024-09-21T03:18:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.609+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/527581075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 msgr2=0x5581cba7aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.609+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba7aeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c01ac00 tx=0x7f3a3c01a470 comp rx=0 tx=0).stop 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/527581075 shutdown_connections 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba7aeb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba7a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/527581075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba7a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/527581075 >> 172.21.15.83:0/527581075 conn(0x5581cb6c7740 msgr2=0x5581cba78320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/527581075 shutdown_connections 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.610+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/527581075 wait complete. 2024-09-21T03:18:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 -- start start 2024-09-21T03:18:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 Processor -- start 2024-09-21T03:18:27.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba86040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.611+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581cb841890 con 0x5581cb6d3350 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581cb97ad40 con 0x5581cb6b2670 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581cba8cfc0 con 0x5581cb6aa490 2024-09-21T03:18:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a4fd9c740 10 client.0 _ll_register_callbacks cb 0x5581cb9dec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba86040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba86040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55642/0 (socket says 172.21.15.83:55642) 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a427fc640 1 -- 172.21.15.83:0/3539410278 learned_addr learned my addr 172.21.15.83:0/3539410278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a427fc640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 -- 172.21.15.83:0/3539410278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 msgr2=0x5581cba86040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba86040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 -- 172.21.15.83:0/3539410278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 msgr2=0x5581cba96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba96310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 -- 172.21.15.83:0/3539410278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cba968d0 con 0x5581cb6b2670 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a427fc640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba86580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5581cba790b0 tx=0x7f3a34011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970aee4b411beacd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34015870 con 0x5581cb6b2670 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.612+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3a34015a10 con 0x5581cb6b2670 2024-09-21T03:18:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3c041ab0 con 0x5581cb6b2670 2024-09-21T03:18:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a34015c10 con 0x5581cb6b2670 2024-09-21T03:18:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 10 client.25373 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5581cba889a0 con 0x5581cb6b2670 2024-09-21T03:18:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c041ab0 con 0x5581cb6b2670 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3a3401d400 con 0x5581cb6b2670 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 10 client.25373 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 20 client.25373 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb691cd0 0x5581cba8bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 4 client.25373 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.613+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581cb9f2db0 con 0x5581cb691cd0 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.614+0000 7f3a427fc640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb691cd0 0x5581cba8bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.614+0000 7f3a22ffd640 10 client.25373 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.614+0000 7f3a427fc640 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb691cd0 0x5581cba8bf90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4984d436399308 server_cookie=dc46ae1b5789d8e3 in_seq=0 out_seq=0 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.615+0000 7f3a22ffd640 1 -- 172.21.15.83:0/3539410278 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581cb9f2db0 con 0x5581cb691cd0 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.615+0000 7f3a22ffd640 10 client.25373 handle_command_reply: tid=0 2024-09-21T03:18:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.615+0000 7f3a4fd9c740 1 client.25373 shutdown 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 msgr2=0x5581cba86580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba86580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5581cba790b0 tx=0x7f3a34011a00 comp rx=0 tx=0).stop 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 shutdown_connections 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb691cd0 0x5581cba8bf90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6d3350 0x5581cba96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cba86580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a22ffd640 0 client.25373 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/3539410278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6aa490 0x5581cba86040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.616+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 >> 172.21.15.83:0/3539410278 conn(0x5581cb6c7740 msgr2=0x5581cba7d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.617+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 shutdown_connections 2024-09-21T03:18:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.617+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/3539410278 wait complete. 2024-09-21T03:18:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.617+0000 7f3a4fd9c740 20 client.25373 trim_cache size 0 max 16384 2024-09-21T03:18:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 Processor -- start 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 -- start start 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 0x5581cba7d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6b2670 0x5581cba7da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3000e5e0 con 0x5581cb691cd0 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a30010240 con 0x5581cb6aa490 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3000ece0 con 0x5581cb6b2670 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46222/0 (socket says 172.21.15.83:46222) 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.643+0000 7f3a427fc640 1 -- 172.21.15.83:0/2074338603 learned_addr learned my addr 172.21.15.83:0/2074338603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 0x5581cba7d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a427fc640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6b2670 0x5581cba7da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 -- 172.21.15.83:0/2074338603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6b2670 msgr2=0x5581cba7da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6b2670 0x5581cba7da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 -- 172.21.15.83:0/2074338603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 msgr2=0x5581cba96280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 -- 172.21.15.83:0/2074338603 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581cba816d0 con 0x5581cb6aa490 2024-09-21T03:18:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a427fc640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.644+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 0x5581cba7d530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a34008010 tx=0x7f3a34009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea65ee7fea863392 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.645+0000 7f3a297fa640 1 -- 172.21.15.83:0/2074338603 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3401d400 con 0x5581cb6aa490 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.645+0000 7f3a297fa640 1 -- 172.21.15.83:0/2074338603 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3401d5a0 con 0x5581cb6aa490 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.645+0000 7f3a297fa640 1 -- 172.21.15.83:0/2074338603 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34027a70 con 0x5581cb6aa490 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.646+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/2074338603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 msgr2=0x5581cba7d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.646+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 0x5581cba7d530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a34008010 tx=0x7f3a34009e80 comp rx=0 tx=0).stop 2024-09-21T03:18:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.646+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/2074338603 shutdown_connections 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.646+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb6b2670 0x5581cba7da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.647+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6aa490 0x5581cba7d530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.647+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/2074338603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb691cd0 0x5581cba96280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.647+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/2074338603 >> 172.21.15.83:0/2074338603 conn(0x5581cb6c7740 msgr2=0x5581cba846c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.647+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/2074338603 shutdown_connections 2024-09-21T03:18:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.647+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/2074338603 wait complete. 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.648+0000 7f3a4fd9c740 1 -- start start 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.648+0000 7f3a4fd9c740 1 Processor -- start 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb691cd0 0x5581cba9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cbb02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3000fab0 con 0x5581cb6aa490 2024-09-21T03:18:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3000f3e0 con 0x5581cb6b2670 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3000c5c0 con 0x5581cb691cd0 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a4fd9c740 10 client.0 _ll_register_callbacks cb 0x7f3a34009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46224/0 (socket says 172.21.15.83:46224) 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 -- 172.21.15.83:0/4139872139 learned_addr learned my addr 172.21.15.83:0/4139872139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a427fc640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cbb02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a427fc640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb691cd0 0x5581cba9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 -- 172.21.15.83:0/4139872139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb691cd0 msgr2=0x5581cba9a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb691cd0 0x5581cba9a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.649+0000 7f3a42ffd640 1 -- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 msgr2=0x5581cbb02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cbb02430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a42ffd640 1 -- 172.21.15.83:0/4139872139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cba9b2d0 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a42ffd640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3a3401dbf0 tx=0x7f3a3401dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc10c2db7091740c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34040060 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3401d5a0 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3a34015880 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a34005970 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a4fd9c740 10 client.15855 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.650+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5581cbb039f0 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3401d5a0 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3a3403a070 con 0x5581cb6aa490 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 10 client.15855 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 20 client.15855 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb6d3350 0x5581cbb061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 4 client.15855 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:18:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.651+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581cb9ee860 con 0x5581cb6d3350 2024-09-21T03:18:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.652+0000 7f3a427fc640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb6d3350 0x5581cbb061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.652+0000 7f3a22ffd640 10 client.15855 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.652+0000 7f3a427fc640 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb6d3350 0x5581cbb061c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3ce3c71c294dd0 server_cookie=449408a513383ba3 in_seq=0 out_seq=0 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.654+0000 7f3a22ffd640 1 -- 172.21.15.83:0/4139872139 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581cb9ee860 con 0x5581cb6d3350 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.654+0000 7f3a22ffd640 10 client.15855 handle_command_reply: tid=0 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.654+0000 7f3a4fd9c740 1 client.15855 shutdown 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.655+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 msgr2=0x5581cba9aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.655+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3a3401dbf0 tx=0x7f3a3401dc20 comp rx=0 tx=0).stop 2024-09-21T03:18:27.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 shutdown_connections 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5581cb6d3350 0x5581cbb061c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a22ffd640 0 client.15855 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581cb6b2670 0x5581cbb02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581cb6aa490 0x5581cba9aa30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 --2- 172.21.15.83:0/4139872139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581cb691cd0 0x5581cba9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 >> 172.21.15.83:0/4139872139 conn(0x5581cb6c7740 msgr2=0x5581cba7d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 shutdown_connections 2024-09-21T03:18:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.662+0000 7f3a4fd9c740 1 -- 172.21.15.83:0/4139872139 wait complete. 2024-09-21T03:18:27.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.663+0000 7f3a4fd9c740 20 client.15855 trim_cache size 0 max 16384 2024-09-21T03:18:27.667 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:18:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:18:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0a46e274-636d-4f02-882e-a6f7cd613a3d", 2024-09-21T03:18:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:18:27.667 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:18:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.667+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 msgr2=0x7f3a2c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.667+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 0x7f3a2c0b8c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005fd0 tx=0x7f3a38079bd0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.667+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 msgr2=0x7f3a4410ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.667+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a4410ce70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c710 tx=0x7f3a3800cbd0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 shutdown_connections 2024-09-21T03:18:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a2c0b6730 0x7f3a2c0b8c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a44109e10 0x7f3a4410ce70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a441094c0 0x7f3a4410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 --2- 172.21.15.83:0/552418264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a44103b80 0x7f3a44110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.673+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 >> 172.21.15.83:0/552418264 conn(0x7f3a4407b8d0 msgr2=0x7f3a44107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.674+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 shutdown_connections 2024-09-21T03:18:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:27.674+0000 7f3a49828640 1 -- 172.21.15.83:0/552418264 wait complete. 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:28 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-21T03:18:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:18:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:28 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: pgmap v379: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:18:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:28.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:28 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:29 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:18:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:29 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:29 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:29 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:18:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:29 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:29 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:29 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:18:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:29 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:29 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:18:29.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:18:30.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:18:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:30 smithi193 ceph-mon[35224]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-21T03:18:30.607 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:30 smithi083 ceph-mon[31219]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-21T03:18:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:30 smithi142 ceph-mon[34512]: pgmap v380: 97 pgs: 97 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 -- 172.21.15.83:0/431848226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 msgr2=0x7f9928104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 --2- 172.21.15.83:0/431848226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f9928104cb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9918009900 tx=0x7f991802a410 comp rx=0 tx=0).stop 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 -- 172.21.15.83:0/431848226 shutdown_connections 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 --2- 172.21.15.83:0/431848226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992810de50 0x7f9928110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 --2- 172.21.15.83:0/431848226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9928105280 0x7f992810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 --2- 172.21.15.83:0/431848226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f9928104cb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 -- 172.21.15.83:0/431848226 >> 172.21.15.83:0/431848226 conn(0x7f9928079790 msgr2=0x7f9928079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.735+0000 7f992d29c640 1 -- 172.21.15.83:0/431848226 shutdown_connections 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.736+0000 7f992d29c640 1 -- 172.21.15.83:0/431848226 wait complete. 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 Processor -- start 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 -- start start 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 0x7f992819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9928113ab0 con 0x7f99281048a0 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9928113930 con 0x7f9928105280 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f992d29c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9928113c30 con 0x7f992810de50 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f99277fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f99277fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41830/0 (socket says 172.21.15.83:41830) 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57702/0 (socket says 172.21.15.83:57702) 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f99277fe640 1 -- 172.21.15.83:0/3081366062 learned_addr learned my addr 172.21.15.83:0/3081366062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.737+0000 7f991ffff640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 0x7f992819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 -- 172.21.15.83:0/3081366062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 msgr2=0x7f992819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 -- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 msgr2=0x7f992819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 -- 172.21.15.83:0/3081366062 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99281a04d0 con 0x7f9928105280 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f99277fe640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.738+0000 7f991ffff640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 0x7f992819ab00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f992806ab20 tx=0x7f991000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.739+0000 7f9924ff9640 1 -- 172.21.15.83:0/3081366062 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991000cc40 con 0x7f9928105280 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.739+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99281a07c0 con 0x7f9928105280 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.739+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99281a6f20 con 0x7f9928105280 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.739+0000 7f9924ff9640 1 -- 172.21.15.83:0/3081366062 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99100044d0 con 0x7f9928105280 2024-09-21T03:18:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.739+0000 7f9924ff9640 1 -- 172.21.15.83:0/3081366062 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991001c070 con 0x7f9928105280 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.743+0000 7f9933810740 1 Processor -- start 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.743+0000 7f9933810740 1 -- start start 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.743+0000 7f9933810740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9933810740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9933810740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9933810740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560869820e50 con 0x5608694e2390 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9933810740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560869810280 con 0x560869503070 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9933810740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560869814090 con 0x5608694da1b0 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9924ff9640 1 -- 172.21.15.83:0/3081366062 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9910004920 con 0x7f9928105280 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f991ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55944/0 (socket says 172.21.15.83:55944) 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 -- 172.21.15.83:0/3232362700 learned_addr learned my addr 172.21.15.83:0/3232362700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 -- 172.21.15.83:0/3232362700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 msgr2=0x5608698aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 -- 172.21.15.83:0/3232362700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 msgr2=0x5608698af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f99277fe640 1 -- 172.21.15.83:0/3232362700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086956e650 con 0x560869503070 2024-09-21T03:18:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.744+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.745+0000 7f99277fe640 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9920011780 tx=0x7f9920011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8faef1c00e4bcf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.745+0000 7f9924ff9640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 0x7f99140b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.746+0000 7f9924ff9640 1 -- 172.21.15.83:0/3081366062 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f99100e73c0 con 0x7f9928105280 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.746+0000 7f991d7fa640 1 -- 172.21.15.83:0/3232362700 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992000fb40 con 0x560869503070 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.746+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 0x7f99140b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.746+0000 7f991d7fa640 1 -- 172.21.15.83:0/3232362700 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920011e90 con 0x560869503070 2024-09-21T03:18:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.746+0000 7f991d7fa640 1 -- 172.21.15.83:0/3232362700 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920012e60 con 0x560869503070 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.747+0000 7f9933810740 1 -- 172.21.15.83:0/3232362700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 msgr2=0x5608698aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.747+0000 7f9933810740 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9920011780 tx=0x7f9920011c40 comp rx=0 tx=0).stop 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 -- 172.21.15.83:0/3232362700 shutdown_connections 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 --2- 172.21.15.83:0/3232362700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698aea60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 -- 172.21.15.83:0/3232362700 >> 172.21.15.83:0/3232362700 conn(0x5608694f7460 msgr2=0x5608698ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 -- 172.21.15.83:0/3232362700 shutdown_connections 2024-09-21T03:18:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9933810740 1 -- 172.21.15.83:0/3232362700 wait complete. 2024-09-21T03:18:30.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.748+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 0x7f99140b8c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9918031440 tx=0x7f9918032ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.749+0000 7f9933810740 1 -- start start 2024-09-21T03:18:30.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.749+0000 7f9933810740 1 Processor -- start 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698bac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560869671700 con 0x5608694e2390 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608697ab110 con 0x560869503070 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5608698bd240 con 0x5608694da1b0 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f9933810740 10 client.0 _ll_register_callbacks cb 0x5608697c4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f99277fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698bac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f99277fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698bac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55946/0 (socket says 172.21.15.83:55946) 2024-09-21T03:18:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.750+0000 7f99277fe640 1 -- 172.21.15.83:0/3927005231 learned_addr learned my addr 172.21.15.83:0/3927005231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f991ffff640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 -- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 msgr2=0x5608698bac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698bac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 -- 172.21.15.83:0/3927005231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 msgr2=0x5608698ba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698ba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.751+0000 7f9926ffd640 1 -- 172.21.15.83:0/3927005231 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608697d8c40 con 0x5608694da1b0 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.752+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698ba1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9918009230 tx=0x7f9918009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b578bb033cb16d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.752+0000 7f990e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991804c030 con 0x5608694da1b0 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9918004770 con 0x5608694da1b0 2024-09-21T03:18:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920011e90 con 0x5608694da1b0 2024-09-21T03:18:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9918035040 con 0x5608694da1b0 2024-09-21T03:18:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f9933810740 10 client.25354 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.753+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608698cd420 con 0x5608694da1b0 2024-09-21T03:18:30.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920011e90 con 0x5608694da1b0 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f99180489c0 con 0x5608694da1b0 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 10 client.25354 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 20 client.25354 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5608694c19f0 0x5608698d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 4 client.25354 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.754+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560869822da0 con 0x5608694c19f0 2024-09-21T03:18:30.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.755+0000 7f991ffff640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5608694c19f0 0x5608698d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.756+0000 7f991ffff640 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5608694c19f0 0x5608698d0a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411f6a276667edab server_cookie=5a7df8491e78c52b in_seq=0 out_seq=0 2024-09-21T03:18:30.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.756+0000 7f990e7fc640 10 client.25354 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f990e7fc640 1 -- 172.21.15.83:0/3927005231 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560869822da0 con 0x5608694c19f0 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f990e7fc640 10 client.25354 handle_command_reply: tid=0 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 client.25354 shutdown 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 msgr2=0x5608698ba1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698ba1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9918009230 tx=0x7f9918009260 comp rx=0 tx=0).stop 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 shutdown_connections 2024-09-21T03:18:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5608694c19f0 0x5608698d0a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x5608698bac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694e2390 0x5608698ba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 --2- 172.21.15.83:0/3927005231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698ba1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 >> 172.21.15.83:0/3927005231 conn(0x5608694f7460 msgr2=0x5608698b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f990e7fc640 0 client.25354 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 shutdown_connections 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.757+0000 7f9933810740 1 -- 172.21.15.83:0/3927005231 wait complete. 2024-09-21T03:18:30.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.758+0000 7f9933810740 20 client.25354 trim_cache size 0 max 16384 2024-09-21T03:18:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.786+0000 7f9933810740 1 Processor -- start 2024-09-21T03:18:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.786+0000 7f9933810740 1 -- start start 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608694e2390 0x5608698b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 0x5608698b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f990800e6b0 con 0x560869503070 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9908010310 con 0x5608694e2390 2024-09-21T03:18:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9933810740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f990800edb0 con 0x5608694da1b0 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41850/0 (socket says 172.21.15.83:41850) 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 -- 172.21.15.83:0/3263482243 learned_addr learned my addr 172.21.15.83:0/3263482243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f991ffff640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608694e2390 0x5608698b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f99277fe640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 0x5608698b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 -- 172.21.15.83:0/3263482243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608694e2390 msgr2=0x5608698b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608694e2390 0x5608698b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 -- 172.21.15.83:0/3263482243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 msgr2=0x5608698b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 0x5608698b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f9926ffd640 1 -- 172.21.15.83:0/3263482243 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608698b0d50 con 0x5608694da1b0 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.787+0000 7f99277fe640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 0x5608698b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.788+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9918006010 tx=0x7f9918000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498a2f3af81e2a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.788+0000 7f991cff9640 1 -- 172.21.15.83:0/3263482243 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991804c040 con 0x5608694da1b0 2024-09-21T03:18:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.788+0000 7f991cff9640 1 -- 172.21.15.83:0/3263482243 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991805f070 con 0x5608694da1b0 2024-09-21T03:18:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.788+0000 7f991cff9640 1 -- 172.21.15.83:0/3263482243 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991803ca20 con 0x5608694da1b0 2024-09-21T03:18:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- 172.21.15.83:0/3263482243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 msgr2=0x5608698bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9918006010 tx=0x7f9918000bc0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- 172.21.15.83:0/3263482243 shutdown_connections 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560869503070 0x5608698b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608694e2390 0x5608698b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 --2- 172.21.15.83:0/3263482243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694da1b0 0x5608698bc090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- 172.21.15.83:0/3263482243 >> 172.21.15.83:0/3263482243 conn(0x5608694f7460 msgr2=0x5608698afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- 172.21.15.83:0/3263482243 shutdown_connections 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- 172.21.15.83:0/3263482243 wait complete. 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.789+0000 7f9933810740 1 -- start start 2024-09-21T03:18:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.790+0000 7f9933810740 1 Processor -- start 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.790+0000 7f9933810740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 0x5608698c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57744/0 (socket says 172.21.15.83:57744) 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9926ffd640 1 -- 172.21.15.83:0/3602110549 learned_addr learned my addr 172.21.15.83:0/3602110549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f991ffff640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 0x5608698c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x56086993a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f990800fb80 con 0x5608694da1b0 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f990800f4b0 con 0x560869503070 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f990800c690 con 0x5608694e2390 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.791+0000 7f9933810740 10 client.0 _ll_register_callbacks cb 0x7f9918009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 -- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 msgr2=0x56086993a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x56086993a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 -- 172.21.15.83:0/3602110549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 msgr2=0x5608698c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608698c9dd0 con 0x5608694e2390 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f991ffff640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 0x5608698c95e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99100e4b30 tx=0x7f99100a9190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86de24b9a903e001 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f990e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.792+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100ac9a0 con 0x5608694e2390 2024-09-21T03:18:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.793+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f99100b67c0 con 0x5608694e2390 2024-09-21T03:18:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.793+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f991805f070 con 0x5608694e2390 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.794+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99100e4fb0 con 0x5608694e2390 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.794+0000 7f9933810740 10 client.25360 fetch_fsmap learned FSMap version 32 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.794+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56086993b940 con 0x5608694e2390 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.794+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991805f070 con 0x5608694e2390 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f99100e7040 con 0x5608694e2390 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 10 client.25360 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 20 client.25360 populate_metadata read hostname 'smithi083' 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560869496da0 0x56086993e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:18:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 4 client.25360 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:18:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608698b1310 con 0x560869496da0 2024-09-21T03:18:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.795+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560869496da0 0x56086993e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:18:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.796+0000 7f9926ffd640 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560869496da0 0x56086993e170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0b30fca14fb7a1 server_cookie=69f30680fc959936 in_seq=0 out_seq=0 2024-09-21T03:18:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.796+0000 7f990e7fc640 10 client.25360 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.796+0000 7f990e7fc640 1 -- 172.21.15.83:0/3602110549 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x5608698b1310 con 0x560869496da0 2024-09-21T03:18:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.796+0000 7f990e7fc640 10 client.25360 handle_command_reply: tid=0 2024-09-21T03:18:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.796+0000 7f9933810740 1 client.25360 shutdown 2024-09-21T03:18:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.797+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 msgr2=0x5608698c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.797+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 0x5608698c95e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99100e4b30 tx=0x7f99100a9190 comp rx=0 tx=0).stop 2024-09-21T03:18:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 shutdown_connections 2024-09-21T03:18:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560869496da0 0x56086993e170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560869503070 0x56086993a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5608694e2390 0x5608698c95e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 --2- 172.21.15.83:0/3602110549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608694da1b0 0x5608698c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 >> 172.21.15.83:0/3602110549 conn(0x5608694f7460 msgr2=0x5608698bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f990e7fc640 0 client.25360 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 shutdown_connections 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 1 -- 172.21.15.83:0/3602110549 wait complete. 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.798+0000 7f9933810740 20 client.25360 trim_cache size 0 max 16384 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:18:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0a46e274-636d-4f02-882e-a6f7cd613a3d", 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:18:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:18:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 msgr2=0x7f99140b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 0x7f99140b8c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9918031440 tx=0x7f9918032ed0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 msgr2=0x7f992819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 0x7f992819ab00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f992806ab20 tx=0x7f991000ee50 comp rx=0 tx=0).stop 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 shutdown_connections 2024-09-21T03:18:30.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99140b6780 0x7f99140b8c90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f992810de50 0x7f992819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9928105280 0x7f992819ab00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 --2- 172.21.15.83:0/3081366062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99281048a0 0x7f992819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.799+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 >> 172.21.15.83:0/3081366062 conn(0x7f9928079790 msgr2=0x7f99281079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.800+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 shutdown_connections 2024-09-21T03:18:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:18:30.800+0000 7f992d29c640 1 -- 172.21.15.83:0/3081366062 wait complete. 2024-09-21T03:18:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 70.0 s, 2932.4 tps, lat 34.147 ms stddev 34.811 2024-09-21T03:18:31.538 INFO:tasks.cephfs.filesystem:scrub status for tag:0a46e274-636d-4f02-882e-a6f7cd613a3d - {'path': '/', 'tag': '0a46e274-636d-4f02-882e-a6f7cd613a3d', 'options': 'recursive,force'} 2024-09-21T03:18:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:31 smithi083 ceph-mon[31219]: pgmap v381: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-21T03:18:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:31 smithi142 ceph-mon[34512]: pgmap v381: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-21T03:18:31.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:31 smithi193 ceph-mon[35224]: pgmap v381: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-21T03:18:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:33 smithi083 ceph-mon[31219]: pgmap v382: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-21T03:18:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:33 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:18:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:33 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:18:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:33 smithi142 ceph-mon[34512]: pgmap v382: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-21T03:18:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:33 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:18:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:33 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:18:33.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:33 smithi193 ceph-mon[35224]: pgmap v382: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-21T03:18:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:33 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:18:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:33 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:18:34.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:18:34.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:18:34.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:18:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:18:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:18:34.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:18:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:18:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:18:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:18:35.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:35 smithi083 ceph-mon[31219]: pgmap v383: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 753 op/s 2024-09-21T03:18:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:35 smithi142 ceph-mon[34512]: pgmap v383: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 753 op/s 2024-09-21T03:18:35.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:35 smithi193 ceph-mon[35224]: pgmap v383: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 753 op/s 2024-09-21T03:18:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 75.0 s, 3396.8 tps, lat 29.363 ms stddev 29.621 2024-09-21T03:18:38.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:38 smithi193 ceph-mon[35224]: pgmap v384: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-21T03:18:38.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:38 smithi083 ceph-mon[31219]: pgmap v384: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-21T03:18:38.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:38 smithi142 ceph-mon[34512]: pgmap v384: 97 pgs: 97 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-21T03:18:40.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:40 smithi193 ceph-mon[35224]: pgmap v385: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.4 MiB/s wr, 835 op/s 2024-09-21T03:18:40.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:40 smithi083 ceph-mon[31219]: pgmap v385: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.4 MiB/s wr, 835 op/s 2024-09-21T03:18:40.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:40 smithi142 ceph-mon[34512]: pgmap v385: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.4 MiB/s wr, 835 op/s 2024-09-21T03:18:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 80.0 s, 3479.0 tps, lat 28.764 ms stddev 30.521 2024-09-21T03:18:42.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:42 smithi193 ceph-mon[35224]: pgmap v386: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 701 op/s 2024-09-21T03:18:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:42 smithi083 ceph-mon[31219]: pgmap v386: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 701 op/s 2024-09-21T03:18:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:42 smithi142 ceph-mon[34512]: pgmap v386: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 701 op/s 2024-09-21T03:18:44.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:44 smithi193 ceph-mon[35224]: pgmap v387: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 827 op/s 2024-09-21T03:18:44.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:44 smithi083 ceph-mon[31219]: pgmap v387: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 827 op/s 2024-09-21T03:18:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:44 smithi142 ceph-mon[34512]: pgmap v387: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 827 op/s 2024-09-21T03:18:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 85.0 s, 3351.4 tps, lat 29.905 ms stddev 31.984 2024-09-21T03:18:46.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:46 smithi193 ceph-mon[35224]: pgmap v388: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 613 op/s 2024-09-21T03:18:46.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:46 smithi083 ceph-mon[31219]: pgmap v388: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 613 op/s 2024-09-21T03:18:46.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:46 smithi142 ceph-mon[34512]: pgmap v388: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 613 op/s 2024-09-21T03:18:48.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:48 smithi193 ceph-mon[35224]: pgmap v389: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-09-21T03:18:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:48 smithi083 ceph-mon[31219]: pgmap v389: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-09-21T03:18:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:48 smithi142 ceph-mon[34512]: pgmap v389: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 763 op/s 2024-09-21T03:18:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:49 smithi193 ceph-mon[35224]: pgmap v390: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 875 op/s 2024-09-21T03:18:49.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:49 smithi083 ceph-mon[31219]: pgmap v390: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 875 op/s 2024-09-21T03:18:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:49 smithi142 ceph-mon[34512]: pgmap v390: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 875 op/s 2024-09-21T03:18:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 90.0 s, 3699.4 tps, lat 27.003 ms stddev 28.027 2024-09-21T03:18:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:51 smithi083 ceph-mon[31219]: pgmap v391: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 672 op/s 2024-09-21T03:18:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:51 smithi142 ceph-mon[34512]: pgmap v391: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 672 op/s 2024-09-21T03:18:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:51 smithi193 ceph-mon[35224]: pgmap v391: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 672 op/s 2024-09-21T03:18:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:54 smithi193 ceph-mon[35224]: pgmap v392: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-09-21T03:18:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:54 smithi083 ceph-mon[31219]: pgmap v392: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-09-21T03:18:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:54 smithi142 ceph-mon[34512]: pgmap v392: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 816 op/s 2024-09-21T03:18:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 95.0 s, 3445.4 tps, lat 29.002 ms stddev 30.164 2024-09-21T03:18:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:56 smithi193 ceph-mon[35224]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:18:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:56 smithi083 ceph-mon[31219]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:18:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:56 smithi142 ceph-mon[34512]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:18:57.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:18:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:18:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:18:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:18:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:18:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:18:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:18:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:18:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:18:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:18:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:18:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:18:58.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:58 smithi193 ceph-mon[35224]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-21T03:18:58.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:58 smithi083 ceph-mon[31219]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-21T03:18:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:58 smithi142 ceph-mon[34512]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-21T03:18:59.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:18:59 smithi193 ceph-mon[35224]: pgmap v395: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-21T03:18:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:18:59 smithi083 ceph-mon[31219]: pgmap v395: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-21T03:18:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:18:59 smithi142 ceph-mon[34512]: pgmap v395: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-21T03:19:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 100.0 s, 3502.2 tps, lat 28.417 ms stddev 29.498 2024-09-21T03:19:01.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:19:02.047 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:02.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:02 smithi083 ceph-mon[31219]: pgmap v396: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-21T03:19:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:02 smithi193 ceph-mon[35224]: pgmap v396: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-21T03:19:02.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.650+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/1143559596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d4105190 msgr2=0x7fa9d41055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.650+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/1143559596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d4105190 0x7fa9d41055a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009900 tx=0x7fa9c402a410 comp rx=0 tx=0).stop 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/1143559596 shutdown_connections 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/1143559596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/1143559596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d4105190 0x7fa9d41055a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/1143559596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d41031c0 0x7fa9d41035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/1143559596 >> 172.21.15.83:0/1143559596 conn(0x7fa9d4077c60 msgr2=0x7fa9d40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/1143559596 shutdown_connections 2024-09-21T03:19:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/1143559596 wait complete. 2024-09-21T03:19:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 Processor -- start 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.651+0000 7fa9dc0fc640 1 -- start start 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d41031c0 0x7fa9d4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 0x7fa9d41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112990 con 0x7fa9d41031c0 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112810 con 0x7fa9d4105ae0 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.652+0000 7fa9dc0fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112b10 con 0x7fa9d4105190 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.653+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 0x7fa9d41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.653+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 0x7fa9d41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45726/0 (socket says 172.21.15.83:45726) 2024-09-21T03:19:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.653+0000 7fa9d9697640 1 -- 172.21.15.83:0/3112614227 learned_addr learned my addr 172.21.15.83:0/3112614227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 -- 172.21.15.83:0/3112614227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 msgr2=0x7fa9d41995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 0x7fa9d41995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 -- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d41031c0 msgr2=0x7fa9d4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d41031c0 0x7fa9d4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.654+0000 7fa9da699640 1 -- 172.21.15.83:0/3112614227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419e350 con 0x7fa9d4105ae0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d4199af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009fd0 tx=0x7fa9cc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9caffd640 1 -- 172.21.15.83:0/3112614227 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00ce20 con 0x7fa9d4105ae0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419e640 con 0x7fa9d4105ae0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9caffd640 1 -- 172.21.15.83:0/3112614227 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc005020 con 0x7fa9d4105ae0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9caffd640 1 -- 172.21.15.83:0/3112614227 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc010690 con 0x7fa9d4105ae0 2024-09-21T03:19:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.655+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d419eb00 con 0x7fa9d4105ae0 2024-09-21T03:19:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.659+0000 7fa9e2670740 1 Processor -- start 2024-09-21T03:19:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9caffd640 1 -- 172.21.15.83:0/3112614227 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa9cc010830 con 0x7fa9d4105ae0 2024-09-21T03:19:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 -- start start 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a93070 0x55e5a1e3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1db0e50 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1da0280 con 0x55e5a1a93070 2024-09-21T03:19:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.660+0000 7fa9e2670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1da4090 con 0x55e5a1a72390 2024-09-21T03:19:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.661+0000 7fa9caffd640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 0x7fa9c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.661+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a93070 0x55e5a1e3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.661+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.662+0000 7fa9caffd640 1 -- 172.21.15.83:0/3112614227 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9cc0e7e20 con 0x7fa9d4105ae0 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.662+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 0x7fa9c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.662+0000 7fa9d9e98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.662+0000 7fa9d9e98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33330/0 (socket says 172.21.15.83:33330) 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.662+0000 7fa9d9e98640 1 -- 172.21.15.83:0/3582226652 learned_addr learned my addr 172.21.15.83:0/3582226652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 -- 172.21.15.83:0/3582226652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 msgr2=0x55e5a1e3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 -- 172.21.15.83:0/3582226652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a93070 msgr2=0x55e5a1e3a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a93070 0x55e5a1e3a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 -- 172.21.15.83:0/3582226652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a1afe650 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.663+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000a9c0 tx=0x7fa9d000ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88ab86ad4a299f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.664+0000 7fa9bffff640 1 -- 172.21.15.83:0/3582226652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000cde0 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.665+0000 7fa9bffff640 1 -- 172.21.15.83:0/3582226652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d000f900 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.665+0000 7fa9bffff640 1 -- 172.21.15.83:0/3582226652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0018900 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.666+0000 7fa9e2670740 1 -- 172.21.15.83:0/3582226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 msgr2=0x55e5a1e3a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.666+0000 7fa9e2670740 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000a9c0 tx=0x7fa9d000ae80 comp rx=0 tx=0).stop 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.668+0000 7fa9e2670740 1 -- 172.21.15.83:0/3582226652 shutdown_connections 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.668+0000 7fa9e2670740 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.668+0000 7fa9e2670740 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3a7b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.668+0000 7fa9e2670740 1 --2- 172.21.15.83:0/3582226652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a93070 0x55e5a1e3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.668+0000 7fa9e2670740 1 -- 172.21.15.83:0/3582226652 >> 172.21.15.83:0/3582226652 conn(0x55e5a1a87460 msgr2=0x55e5a1e38160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.669+0000 7fa9da699640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 0x7fa9c00b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0e55f0 tx=0x7fa9cc014000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.671+0000 7fa9e2670740 1 -- 172.21.15.83:0/3582226652 shutdown_connections 2024-09-21T03:19:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.671+0000 7fa9e2670740 1 -- 172.21.15.83:0/3582226652 wait complete. 2024-09-21T03:19:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.672+0000 7fa9e2670740 1 -- start start 2024-09-21T03:19:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.672+0000 7fa9e2670740 1 Processor -- start 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.673+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.673+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e45ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.673+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a93070 0x55e5a1e51de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.673+0000 7fa9e2670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1c01700 con 0x55e5a1a93070 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.673+0000 7fa9e2670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1d3b110 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.674+0000 7fa9e2670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5a1e48aa0 con 0x55e5a1a72390 2024-09-21T03:19:02.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.674+0000 7fa9d9e98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e45ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.674+0000 7fa9d9e98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e45ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45742/0 (socket says 172.21.15.83:45742) 2024-09-21T03:19:02.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.674+0000 7fa9d9e98640 1 -- 172.21.15.83:0/1106496527 learned_addr learned my addr 172.21.15.83:0/1106496527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:02.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.674+0000 7fa9e2670740 10 client.0 _ll_register_callbacks cb 0x55e5a1d54990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:02 smithi142 ceph-mon[34512]: pgmap v396: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-21T03:19:02.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.675+0000 7fa9d9697640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.675+0000 7fa9d9697640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a93070 0x55e5a1e51de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 -- 172.21.15.83:0/1106496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 msgr2=0x55e5a1e45ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e45ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 -- 172.21.15.83:0/1106496527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a93070 msgr2=0x55e5a1e51de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a93070 0x55e5a1e51de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 -- 172.21.15.83:0/1106496527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a1d68c40 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9d9697640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e459b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004770 tx=0x7fa9c4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aebd948885d18963 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.677+0000 7fa9bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.678+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4041070 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.678+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9c4005020 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.678+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d000f900 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.679+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c4052050 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.681+0000 7fa9e2670740 10 client.25406 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:02.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.681+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5a1e54810 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c40047a0 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa9c40314a0 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 10 client.25406 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 10 client.25406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 20 client.25406 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:02.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a519f0 0x55e5a1e57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 4 client.25406 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:02.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.682+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa9d000f900 con 0x55e5a1a519f0 2024-09-21T03:19:02.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.684+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a519f0 0x55e5a1e57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.686+0000 7fa9bd7fa640 10 client.25406 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.686+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a519f0 0x55e5a1e57d90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56884126bcbb29a2 server_cookie=1f6facdf3bbeb901 in_seq=0 out_seq=0 2024-09-21T03:19:02.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.687+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/1106496527 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa9d000f900 con 0x55e5a1a519f0 2024-09-21T03:19:02.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.687+0000 7fa9bd7fa640 10 client.25406 handle_command_reply: tid=0 2024-09-21T03:19:02.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.687+0000 7fa9e2670740 1 client.25406 shutdown 2024-09-21T03:19:02.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 msgr2=0x55e5a1e459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e459b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004770 tx=0x7fa9c4005a20 comp rx=0 tx=0).stop 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 shutdown_connections 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a519f0 0x55e5a1e57d90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9bd7fa640 0 client.25406 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a93070 0x55e5a1e51de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1e45ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 --2- 172.21.15.83:0/1106496527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e459b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 >> 172.21.15.83:0/1106496527 conn(0x55e5a1a87460 msgr2=0x55e5a1e3b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 shutdown_connections 2024-09-21T03:19:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.688+0000 7fa9e2670740 1 -- 172.21.15.83:0/1106496527 wait complete. 2024-09-21T03:19:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.689+0000 7fa9e2670740 20 client.25406 trim_cache size 0 max 16384 2024-09-21T03:19:02.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 Processor -- start 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 -- start start 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a519f0 0x55e5a1e45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b5be0 con 0x55e5a1a72390 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b7840 con 0x55e5a1a519f0 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9e2670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b62e0 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33348/0 (socket says 172.21.15.83:33348) 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9d9697640 1 -- 172.21.15.83:0/2604995965 learned_addr learned my addr 172.21.15.83:0/2604995965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.716+0000 7fa9d9697640 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a519f0 0x55e5a1e45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 -- 172.21.15.83:0/2604995965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a6a1b0 msgr2=0x55e5a1e3b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 -- 172.21.15.83:0/2604995965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a519f0 msgr2=0x55e5a1e45dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a519f0 0x55e5a1e45dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 -- 172.21.15.83:0/2604995965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a1e3c590 con 0x55e5a1a72390 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9d9697640 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003a20 tx=0x7fa9c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f4887e7089f047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9bf7fe640 1 -- 172.21.15.83:0/2604995965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4041040 con 0x55e5a1a72390 2024-09-21T03:19:02.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9bf7fe640 1 -- 172.21.15.83:0/2604995965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c4003720 con 0x55e5a1a72390 2024-09-21T03:19:02.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.717+0000 7fa9bf7fe640 1 -- 172.21.15.83:0/2604995965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4035030 con 0x55e5a1a72390 2024-09-21T03:19:02.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 -- 172.21.15.83:0/2604995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 msgr2=0x55e5a1e51ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4003a20 tx=0x7fa9c4005fd0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 -- 172.21.15.83:0/2604995965 shutdown_connections 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a72390 0x55e5a1e51ce0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e3b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 --2- 172.21.15.83:0/2604995965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a519f0 0x55e5a1e45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 -- 172.21.15.83:0/2604995965 >> 172.21.15.83:0/2604995965 conn(0x55e5a1a87460 msgr2=0x55e5a1e3b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 -- 172.21.15.83:0/2604995965 shutdown_connections 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.719+0000 7fa9e2670740 1 -- 172.21.15.83:0/2604995965 wait complete. 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.720+0000 7fa9e2670740 1 -- start start 2024-09-21T03:19:02.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.720+0000 7fa9e2670740 1 Processor -- start 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1ec1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b70b0 con 0x55e5a1a519f0 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b69e0 con 0x55e5a1a6a1b0 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b3bc0 con 0x55e5a1a72390 2024-09-21T03:19:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9e2670740 10 client.0 _ll_register_callbacks cb 0x7fa9c4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33354/0 (socket says 172.21.15.83:33354) 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 -- 172.21.15.83:0/346494866 learned_addr learned my addr 172.21.15.83:0/346494866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1ec1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 -- 172.21.15.83:0/346494866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 msgr2=0x55e5a1ec1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1ec1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 -- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 msgr2=0x55e5a1e4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 -- 172.21.15.83:0/346494866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a1ec1d50 con 0x55e5a1a519f0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9d9697640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4031b10 tx=0x7fa9c4009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24368b51e10fa280 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.721+0000 7fa9bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4068060 con 0x55e5a1a519f0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c4003720 con 0x55e5a1a519f0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9c4005430 con 0x55e5a1a519f0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c4035070 con 0x55e5a1a519f0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9e2670740 10 client.15888 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5a1ec2fe0 con 0x55e5a1a519f0 2024-09-21T03:19:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.722+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c4003720 con 0x55e5a1a519f0 2024-09-21T03:19:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.723+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa9c4063460 con 0x55e5a1a519f0 2024-09-21T03:19:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 10 client.15888 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 20 client.15888 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a93070 0x55e5a1ec57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 4 client.15888 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5a1db2da0 con 0x55e5a1a93070 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.724+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a93070 0x55e5a1ec57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9d9e98640 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a93070 0x55e5a1ec57f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c452871b7b017a51 server_cookie=d9b0e79df3ac3d75 in_seq=0 out_seq=0 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9bd7fa640 10 client.15888 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9bd7fa640 1 -- 172.21.15.83:0/346494866 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55e5a1db2da0 con 0x55e5a1a93070 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9bd7fa640 10 client.15888 handle_command_reply: tid=0 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 client.15888 shutdown 2024-09-21T03:19:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 msgr2=0x55e5a1e4a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4031b10 tx=0x7fa9c4009300 comp rx=0 tx=0).stop 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 shutdown_connections 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55e5a1a93070 0x55e5a1ec57f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a1a72390 0x55e5a1ec1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.725+0000 7fa9bd7fa640 0 client.15888 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e5a1a6a1b0 0x55e5a1e4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 1 --2- 172.21.15.83:0/346494866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a1a519f0 0x55e5a1e4a4d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 >> 172.21.15.83:0/346494866 conn(0x55e5a1a87460 msgr2=0x55e5a1e43e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 shutdown_connections 2024-09-21T03:19:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 1 -- 172.21.15.83:0/346494866 wait complete. 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.726+0000 7fa9e2670740 20 client.15888 trim_cache size 0 max 16384 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:02.731 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:02.733 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:19:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 msgr2=0x7fa9c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 0x7fa9c00b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0e55f0 tx=0x7fa9cc014000 comp rx=0 tx=0).stop 2024-09-21T03:19:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 msgr2=0x7fa9d4199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d4199af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009fd0 tx=0x7fa9cc00ef10 comp rx=0 tx=0).stop 2024-09-21T03:19:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 shutdown_connections 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9c00b6680 0x7fa9c00b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9d4105ae0 0x7fa9d4199af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d4105190 0x7fa9d41995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 --2- 172.21.15.83:0/3112614227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d41031c0 0x7fa9d4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.727+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 >> 172.21.15.83:0/3112614227 conn(0x7fa9d4077c60 msgr2=0x7fa9d41064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.728+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 shutdown_connections 2024-09-21T03:19:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:02.728+0000 7fa9dc0fc640 1 -- 172.21.15.83:0/3112614227 wait complete. 2024-09-21T03:19:03.433 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:03 smithi083 ceph-mon[31219]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 30 MiB/s wr, 1.33k op/s 2024-09-21T03:19:03.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:19:03.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:19:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:03 smithi142 ceph-mon[34512]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 30 MiB/s wr, 1.33k op/s 2024-09-21T03:19:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:03 smithi193 ceph-mon[35224]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 30 MiB/s wr, 1.33k op/s 2024-09-21T03:19:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:04.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.580+0000 7f193d0f0640 1 -- 172.21.15.83:0/2383534000 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035020 con 0x7f19381094d0 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 -- 172.21.15.83:0/2383534000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 msgr2=0x7f19381098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 --2- 172.21.15.83:0/2383534000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f19381098e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802a410 comp rx=0 tx=0).stop 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 -- 172.21.15.83:0/2383534000 shutdown_connections 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 --2- 172.21.15.83:0/2383534000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109e20 0x7f1938110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 --2- 172.21.15.83:0/2383534000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f19381098e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 --2- 172.21.15.83:0/2383534000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938103b90 0x7f1938103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.581+0000 7f1940356640 1 -- 172.21.15.83:0/2383534000 >> 172.21.15.83:0/2383534000 conn(0x7f19380ff5e0 msgr2=0x7f1938101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.583+0000 7f1940356640 1 -- 172.21.15.83:0/2383534000 shutdown_connections 2024-09-21T03:19:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.583+0000 7f1940356640 1 -- 172.21.15.83:0/2383534000 wait complete. 2024-09-21T03:19:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 Processor -- start 2024-09-21T03:19:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 -- start start 2024-09-21T03:19:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 0x7f193819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f193819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 0x7f193819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113b30 con 0x7f1938103b90 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19381139b0 con 0x7f1938109e20 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.584+0000 7f1940356640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113cb0 con 0x7f19381094d0 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193d8f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f193819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e0f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 0x7f193819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 0x7f193819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e0f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 0x7f193819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33370/0 (socket says 172.21.15.83:33370) 2024-09-21T03:19:04.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e0f2640 1 -- 172.21.15.83:0/4172842429 learned_addr learned my addr 172.21.15.83:0/4172842429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 -- 172.21.15.83:0/4172842429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 msgr2=0x7f193819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f193819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 -- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 msgr2=0x7f193819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 0x7f193819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 -- 172.21.15.83:0/4172842429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381a0400 con 0x7f1938109e20 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.585+0000 7f193e8f3640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 0x7f193819fd90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1930007db0 tx=0x7f1930010420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.586+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930010d40 con 0x7f1938109e20 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.586+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381a06f0 con 0x7f1938109e20 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.586+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19380755f0 con 0x7f1938109e20 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.586+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f193000c640 con 0x7f1938109e20 2024-09-21T03:19:04.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.586+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930017650 con 0x7f1938109e20 2024-09-21T03:19:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.587+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1938104e10 con 0x7f1938109e20 2024-09-21T03:19:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.590+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f19300177f0 con 0x7f1938109e20 2024-09-21T03:19:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.591+0000 7f192f7fe640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 0x7f19240b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.591+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f19300ebe10 con 0x7f1938109e20 2024-09-21T03:19:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.594+0000 7f193e0f2640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 0x7f19240b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:04.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.597+0000 7f193e0f2640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 0x7f19240b8bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1934005fd0 tx=0x7f193400b480 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.600+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19300b23e0 con 0x7f1938109e20 2024-09-21T03:19:04.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.935+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193810bc10 con 0x7f1938109e20 2024-09-21T03:19:04.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.937+0000 7f192f7fe640 1 -- 172.21.15.83:0/4172842429 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11991 (secure 0 0 0) 0x7f19300b67c0 con 0x7f1938109e20 2024-09-21T03:19:04.939 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:19:04.939 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"btime":"2024-09-21T03:17:56:891811+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:17:56.429534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:19:04.939 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 msgr2=0x7f19240b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 0x7f19240b8bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1934005fd0 tx=0x7f193400b480 comp rx=0 tx=0).stop 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 msgr2=0x7f193819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 0x7f193819fd90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1930007db0 tx=0x7f1930010420 comp rx=0 tx=0).stop 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 shutdown_connections 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19240b66b0 0x7f19240b8bc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938109e20 0x7f193819fd90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19381094d0 0x7f193819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 --2- 172.21.15.83:0/4172842429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938103b90 0x7f193819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 >> 172.21.15.83:0/4172842429 conn(0x7f19380ff5e0 msgr2=0x7f1938101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 shutdown_connections 2024-09-21T03:19:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:04.941+0000 7f1940356640 1 -- 172.21.15.83:0/4172842429 wait complete. 2024-09-21T03:19:05.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:04 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4172842429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:05.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:04 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4172842429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:04 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4172842429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 105.0 s, 2241.8 tps, lat 44.746 ms stddev 90.022 2024-09-21T03:19:06.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:05 smithi193 ceph-mon[35224]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 32 MiB/s wr, 1.23k op/s 2024-09-21T03:19:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:05 smithi083 ceph-mon[31219]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 32 MiB/s wr, 1.23k op/s 2024-09-21T03:19:06.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:05 smithi142 ceph-mon[34512]: pgmap v398: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 32 MiB/s wr, 1.23k op/s 2024-09-21T03:19:07.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:19:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/1737302565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 msgr2=0x7f3b5810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/1737302565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b5810f520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009900 tx=0x7f3b4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/1737302565 shutdown_connections 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/1737302565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b5810f520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/1737302565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b581087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/1737302565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 0x7f3b58102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.462+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/1737302565 >> 172.21.15.83:0/1737302565 conn(0x7f3b58079730 msgr2=0x7f3b58079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.463+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/1737302565 shutdown_connections 2024-09-21T03:19:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.463+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/1737302565 wait complete. 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 Processor -- start 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 -- start start 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 0x7f3b5819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b5819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58115c70 con 0x7f3b58108cf0 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58115af0 con 0x7f3b581083a0 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.464+0000 7f3b5ca6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58115df0 con 0x7f3b58102a60 2024-09-21T03:19:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b5819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33384/0 (socket says 172.21.15.83:33384) 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 -- 172.21.15.83:0/3533435728 learned_addr learned my addr 172.21.15.83:0/3533435728 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b5659c640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 0x7f3b5819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 -- 172.21.15.83:0/3533435728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 msgr2=0x7f3b5819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 0x7f3b5819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 -- 172.21.15.83:0/3533435728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 msgr2=0x7f3b5819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b5819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 -- 172.21.15.83:0/3533435728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b581a2510 con 0x7f3b58108cf0 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b5819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b56d9d640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005ab0 tx=0x7f3b4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.465+0000 7f3b477fe640 1 -- 172.21.15.83:0/3533435728 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c035030 con 0x7f3b58108cf0 2024-09-21T03:19:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.466+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b581a27a0 con 0x7f3b58108cf0 2024-09-21T03:19:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.466+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581a8f60 con 0x7f3b58108cf0 2024-09-21T03:19:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.466+0000 7f3b477fe640 1 -- 172.21.15.83:0/3533435728 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b4c003cb0 con 0x7f3b58108cf0 2024-09-21T03:19:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.467+0000 7f3b477fe640 1 -- 172.21.15.83:0/3533435728 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0318a0 con 0x7f3b58108cf0 2024-09-21T03:19:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.470+0000 7f3b62fde740 1 Processor -- start 2024-09-21T03:19:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.471+0000 7f3b477fe640 1 -- 172.21.15.83:0/3533435728 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3b4c03b460 con 0x7f3b58108cf0 2024-09-21T03:19:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 -- start start 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65cc070 0x55cfc69736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6974120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cfc68e9ea0 con 0x55cfc65a31b0 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cfc68d92d0 con 0x55cfc65ab390 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b62fde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cfc68dd0e0 con 0x55cfc65cc070 2024-09-21T03:19:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.472+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33394/0 (socket says 172.21.15.83:33394) 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b5659c640 1 -- 172.21.15.83:0/1805963085 learned_addr learned my addr 172.21.15.83:0/1805963085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6974120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65cc070 0x55cfc69736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b477fe640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 0x7f3b400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 -- 172.21.15.83:0/1805963085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65cc070 msgr2=0x55cfc69736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65cc070 0x55cfc69736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 -- 172.21.15.83:0/1805963085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 msgr2=0x55cfc6973be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 -- 172.21.15.83:0/1805963085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc6637650 con 0x55cfc65ab390 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b5659c640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b56d9d640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 0x7f3b400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6974120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b58079d30 tx=0x7f3b50005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105fde374a6cf4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b44ff9640 1 -- 172.21.15.83:0/1805963085 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50004030 con 0x55cfc65ab390 2024-09-21T03:19:08.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b477fe640 1 -- 172.21.15.83:0/3533435728 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3b4c107d10 con 0x7f3b58108cf0 2024-09-21T03:19:08.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b44ff9640 1 -- 172.21.15.83:0/1805963085 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b50005020 con 0x55cfc65ab390 2024-09-21T03:19:08.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.473+0000 7f3b44ff9640 1 -- 172.21.15.83:0/1805963085 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50004990 con 0x55cfc65ab390 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 -- 172.21.15.83:0/1805963085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 msgr2=0x55cfc6974120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6974120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3b58079d30 tx=0x7f3b50005f90 comp rx=0 tx=0).stop 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b56d9d640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 0x7f3b400b8c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cdb70 tx=0x7f3b4c0cddd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 -- 172.21.15.83:0/1805963085 shutdown_connections 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6974120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6973be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 --2- 172.21.15.83:0/1805963085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65cc070 0x55cfc69736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 -- 172.21.15.83:0/1805963085 >> 172.21.15.83:0/1805963085 conn(0x55cfc65c0460 msgr2=0x55cfc6971590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.475+0000 7f3b62fde740 1 -- 172.21.15.83:0/1805963085 shutdown_connections 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.476+0000 7f3b62fde740 1 -- 172.21.15.83:0/1805963085 wait complete. 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.476+0000 7f3b62fde740 1 -- start start 2024-09-21T03:19:08.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.476+0000 7f3b62fde740 1 Processor -- start 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc697f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33398/0 (socket says 172.21.15.83:33398) 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65cc070 0x55cfc698b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b55d9b640 1 -- 172.21.15.83:0/721846929 learned_addr learned my addr 172.21.15.83:0/721846929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cfc673a700 con 0x55cfc65a31b0 2024-09-21T03:19:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cfc6874160 con 0x55cfc65cc070 2024-09-21T03:19:08.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b5659c640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc697f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.477+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cfc6981e20 con 0x55cfc65ab390 2024-09-21T03:19:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65cc070 0x55cfc698b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 -- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 msgr2=0x55cfc697f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc697f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 -- 172.21.15.83:0/721846929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65cc070 msgr2=0x55cfc698b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65cc070 0x55cfc698b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc68a1c90 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005bc0 tx=0x7f3b5000b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70724d72feaf402 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b62fde740 10 client.0 _ll_register_callbacks cb 0x55cfc688d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.478+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b50005020 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.479+0000 7f3b3dffb640 1 -- 172.21.15.83:0/721846929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50018b30 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.479+0000 7f3b3dffb640 1 -- 172.21.15.83:0/721846929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3b50011040 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.479+0000 7f3b3dffb640 1 -- 172.21.15.83:0/721846929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b50018cd0 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.479+0000 7f3b62fde740 10 client.15903 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.479+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55cfc698d8b0 con 0x55cfc65a31b0 2024-09-21T03:19:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b3dffb640 1 -- 172.21.15.83:0/721846929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3b50028030 con 0x55cfc65a31b0 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 10 client.15903 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 10 client.15903 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 20 client.15903 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc658a9f0 0x55cfc6990c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 4 client.15903 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3b50005020 con 0x55cfc658a9f0 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.480+0000 7f3b5659c640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc658a9f0 0x55cfc6990c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.481+0000 7f3b3dffb640 10 client.15903 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.481+0000 7f3b5659c640 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc658a9f0 0x55cfc6990c70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d566d9b751303aa server_cookie=4c7a32eba3628f9e in_seq=0 out_seq=0 2024-09-21T03:19:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.481+0000 7f3b3dffb640 1 -- 172.21.15.83:0/721846929 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3b50005020 con 0x55cfc658a9f0 2024-09-21T03:19:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.481+0000 7f3b3dffb640 10 client.15903 handle_command_reply: tid=0 2024-09-21T03:19:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.481+0000 7f3b62fde740 1 client.15903 shutdown 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 msgr2=0x55cfc697ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005bc0 tx=0x7f3b5000b2e0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 shutdown_connections 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc658a9f0 0x55cfc6990c70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b3dffb640 0 client.15903 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65cc070 0x55cfc698b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc697f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 --2- 172.21.15.83:0/721846929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc697ee00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 >> 172.21.15.83:0/721846929 conn(0x55cfc65c0460 msgr2=0x55cfc6976830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 shutdown_connections 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.483+0000 7f3b62fde740 1 -- 172.21.15.83:0/721846929 wait complete. 2024-09-21T03:19:08.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.484+0000 7f3b62fde740 20 client.15903 trim_cache size 0 max 16384 2024-09-21T03:19:08.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.500+0000 7f3b62fde740 1 Processor -- start 2024-09-21T03:19:08.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.500+0000 7f3b62fde740 1 -- start start 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc698b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3801f6b0 con 0x55cfc65a31b0 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38021310 con 0x55cfc658a9f0 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b62fde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3801fdb0 con 0x55cfc65ab390 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33402/0 (socket says 172.21.15.83:33402) 2024-09-21T03:19:08.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58572/0 (socket says 172.21.15.83:58572) 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b5659c640 1 -- 172.21.15.83:0/3402717182 learned_addr learned my addr 172.21.15.83:0/3402717182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc698b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 -- 172.21.15.83:0/3402717182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 msgr2=0x55cfc698b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc698b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 -- 172.21.15.83:0/3402717182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 msgr2=0x55cfc6980ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b55d9b640 1 -- 172.21.15.83:0/3402717182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc6975990 con 0x55cfc658a9f0 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.501+0000 7f3b5659c640 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.502+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005f60 tx=0x7f3b50012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ed015f3239b215 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.502+0000 7f3b3ffff640 1 -- 172.21.15.83:0/3402717182 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50018b30 con 0x55cfc658a9f0 2024-09-21T03:19:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.502+0000 7f3b3ffff640 1 -- 172.21.15.83:0/3402717182 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b50003aa0 con 0x55cfc658a9f0 2024-09-21T03:19:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.502+0000 7f3b3ffff640 1 -- 172.21.15.83:0/3402717182 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b500296f0 con 0x55cfc658a9f0 2024-09-21T03:19:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- 172.21.15.83:0/3402717182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 msgr2=0x55cfc697f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005f60 tx=0x7f3b50012040 comp rx=0 tx=0).stop 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- 172.21.15.83:0/3402717182 shutdown_connections 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc65ab390 0x55cfc698b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6980ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 --2- 172.21.15.83:0/3402717182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc658a9f0 0x55cfc697f380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- 172.21.15.83:0/3402717182 >> 172.21.15.83:0/3402717182 conn(0x55cfc65c0460 msgr2=0x55cfc69745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- 172.21.15.83:0/3402717182 shutdown_connections 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- 172.21.15.83:0/3402717182 wait complete. 2024-09-21T03:19:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 -- start start 2024-09-21T03:19:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.504+0000 7f3b62fde740 1 Processor -- start 2024-09-21T03:19:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 0x55cfc6983040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6983ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38020b80 con 0x55cfc65a31b0 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b380204b0 con 0x55cfc65ab390 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3801d690 con 0x55cfc658a9f0 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b62fde740 10 client.0 _ll_register_callbacks cb 0x7f3b48009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 0x55cfc6983040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33418/0 (socket says 172.21.15.83:33418) 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b55d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 0x55cfc6983040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45830/0 (socket says 172.21.15.83:45830) 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b5659c640 1 -- 172.21.15.83:0/284487633 learned_addr learned my addr 172.21.15.83:0/284487633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.505+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6983ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 -- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 msgr2=0x55cfc6983040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 0x55cfc6983040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 -- 172.21.15.83:0/284487633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 msgr2=0x55cfc6983ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6983ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 -- 172.21.15.83:0/284487633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc69fa4d0 con 0x55cfc65a31b0 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.506+0000 7f3b5659c640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009270 tx=0x7f3b4801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f685cdabbea248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.507+0000 7f3b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.507+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48027020 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.507+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f3b480149d0 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.507+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b50003aa0 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.507+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b48014bd0 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 10 client.15909 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cfc69fb740 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b48014d70 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3b4800f040 con 0x55cfc65a31b0 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 10 client.15909 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 10 client.15909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 20 client.15909 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc65cc070 0x55cfc69fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 4 client.15909 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.508+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3b50003aa0 con 0x55cfc65cc070 2024-09-21T03:19:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.509+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc65cc070 0x55cfc69fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.511+0000 7f3b55d9b640 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc65cc070 0x55cfc69fdf10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16d4cda8fec146ec server_cookie=9f5ca179f76a8a46 in_seq=0 out_seq=0 2024-09-21T03:19:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.511+0000 7f3b3dffb640 10 client.15909 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:08.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.513+0000 7f3b3dffb640 1 -- 172.21.15.83:0/284487633 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3b50003aa0 con 0x55cfc65cc070 2024-09-21T03:19:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.513+0000 7f3b3dffb640 10 client.15909 handle_command_reply: tid=0 2024-09-21T03:19:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.513+0000 7f3b62fde740 1 client.15909 shutdown 2024-09-21T03:19:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 msgr2=0x55cfc6983580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009270 tx=0x7f3b4801eb30 comp rx=0 tx=0).stop 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 shutdown_connections 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cfc65cc070 0x55cfc69fdf10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cfc65ab390 0x55cfc6983ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cfc65a31b0 0x55cfc6983580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 --2- 172.21.15.83:0/284487633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cfc658a9f0 0x55cfc6983040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.514+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 >> 172.21.15.83:0/284487633 conn(0x55cfc65c0460 msgr2=0x55cfc6974fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.515+0000 7f3b3dffb640 0 client.15909 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.516+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 shutdown_connections 2024-09-21T03:19:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.516+0000 7f3b62fde740 1 -- 172.21.15.83:0/284487633 wait complete. 2024-09-21T03:19:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.516+0000 7f3b62fde740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T03:19:08.518 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.518+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 msgr2=0x7f3b400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.518+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 0x7f3b400b8c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cdb70 tx=0x7f3b4c0cddd0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.518+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 msgr2=0x7f3b581a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.518+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005ab0 tx=0x7f3b4c0045e0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 shutdown_connections 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3b400b6700 0x7f3b400b8c10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b58108cf0 0x7f3b581a1de0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b581083a0 0x7f3b5819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 --2- 172.21.15.83:0/3533435728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58102a60 0x7f3b5819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.519+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 >> 172.21.15.83:0/3533435728 conn(0x7f3b58079730 msgr2=0x7f3b58104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.520+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 shutdown_connections 2024-09-21T03:19:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:08.521+0000 7f3b5ca6a640 1 -- 172.21.15.83:0/3533435728 wait complete. 2024-09-21T03:19:08.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:08 smithi193 ceph-mon[35224]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 1.52k op/s 2024-09-21T03:19:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:08 smithi083 ceph-mon[31219]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 1.52k op/s 2024-09-21T03:19:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:08 smithi142 ceph-mon[34512]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 41 MiB/s wr, 1.52k op/s 2024-09-21T03:19:09.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:19:09.802 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- 172.21.15.83:0/2702351418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a60 msgr2=0x7fa7dc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2702351418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a60 0x7fa7dc102e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009ca0 tx=0x7fa7d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- 172.21.15.83:0/2702351418 shutdown_connections 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2702351418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2702351418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc1083a0 0x7fa7dc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2702351418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc102a60 0x7fa7dc102e70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- 172.21.15.83:0/2702351418 >> 172.21.15.83:0/2702351418 conn(0x7fa7dc079730 msgr2=0x7fa7dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- 172.21.15.83:0/2702351418 shutdown_connections 2024-09-21T03:19:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- 172.21.15.83:0/2702351418 wait complete. 2024-09-21T03:19:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 Processor -- start 2024-09-21T03:19:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.380+0000 7fa7e2580640 1 -- start start 2024-09-21T03:19:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 0x7fa7dc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc1083a0 0x7fa7dc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc111860 con 0x7fa7dc102a60 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1116e0 con 0x7fa7dc1083a0 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e2580640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1119e0 con 0x7fa7dc108cf0 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.381+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 0x7fa7dc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55830/0 (socket says 172.21.15.83:55830) 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2847652856 learned_addr learned my addr 172.21.15.83:0/2847652856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc1083a0 0x7fa7dc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 0x7fa7dc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37794/0 (socket says 172.21.15.83:37794) 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2847652856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc1083a0 msgr2=0x7fa7dc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc1083a0 0x7fa7dc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 msgr2=0x7fa7dc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 0x7fa7dc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.382+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2847652856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc19e1a0 con 0x7fa7dc108cf0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000f890 tx=0x7fa7d000fd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7d97fa640 1 -- 172.21.15.83:0/2847652856 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0005020 con 0x7fa7dc108cf0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7d97fa640 1 -- 172.21.15.83:0/2847652856 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d0005670 con 0x7fa7dc108cf0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc19e490 con 0x7fa7dc108cf0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7d97fa640 1 -- 172.21.15.83:0/2847652856 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004b60 con 0x7fa7dc108cf0 2024-09-21T03:19:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.383+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1a4bf0 con 0x7fa7dc108cf0 2024-09-21T03:19:10.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.387+0000 7fa7d97fa640 1 -- 172.21.15.83:0/2847652856 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa7d001b020 con 0x7fa7dc108cf0 2024-09-21T03:19:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.389+0000 7fa7d97fa640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 0x7fa7c40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.389+0000 7fa7e8af4740 1 Processor -- start 2024-09-21T03:19:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.389+0000 7fa7d97fa640 1 -- 172.21.15.83:0/2847652856 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7d00e70e0 con 0x7fa7dc108cf0 2024-09-21T03:19:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.389+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 0x7fa7c40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.389+0000 7fa7e8af4740 1 -- start start 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf419e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf4195dea0 con 0x55cf416171b0 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf4194d2d0 con 0x55cf4161f390 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf419510e0 con 0x55cf41640070 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37798/0 (socket says 172.21.15.83:37798) 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.390+0000 7fa7db7fe640 1 -- 172.21.15.83:0/3571006944 learned_addr learned my addr 172.21.15.83:0/3571006944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf419e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 -- 172.21.15.83:0/3571006944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 msgr2=0x55cf419e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf419e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 -- 172.21.15.83:0/3571006944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 msgr2=0x55cf419e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 -- 172.21.15.83:0/3571006944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf416ab650 con 0x55cf416171b0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009c60 tx=0x7fa7cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b28dfd0446d039 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.391+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 0x7fa7c40b8c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40094e0 tx=0x7fa7d4031450 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.392+0000 7fa7c27fc640 1 -- 172.21.15.83:0/3571006944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc004050 con 0x55cf416171b0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.392+0000 7fa7c27fc640 1 -- 172.21.15.83:0/3571006944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc0054d0 con 0x55cf416171b0 2024-09-21T03:19:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.392+0000 7fa7c27fc640 1 -- 172.21.15.83:0/3571006944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc004c40 con 0x55cf416171b0 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 -- 172.21.15.83:0/3571006944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 msgr2=0x55cf419e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009c60 tx=0x7fa7cc011810 comp rx=0 tx=0).stop 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 -- 172.21.15.83:0/3571006944 shutdown_connections 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419e7800 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/3571006944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf419e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.393+0000 7fa7e8af4740 1 -- 172.21.15.83:0/3571006944 >> 172.21.15.83:0/3571006944 conn(0x55cf41634460 msgr2=0x55cf419e51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.394+0000 7fa7e8af4740 1 -- 172.21.15.83:0/3571006944 shutdown_connections 2024-09-21T03:19:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.394+0000 7fa7e8af4740 1 -- 172.21.15.83:0/3571006944 wait complete. 2024-09-21T03:19:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.395+0000 7fa7e8af4740 1 -- start start 2024-09-21T03:19:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.395+0000 7fa7e8af4740 1 Processor -- start 2024-09-21T03:19:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 0x55cf419f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 0x55cf419f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 0x55cf41a0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 0x55cf419f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55848/0 (socket says 172.21.15.83:55848) 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf417ae700 con 0x55cf41640070 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf418e8160 con 0x55cf4161f390 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf41a06ab0 con 0x55cf416171b0 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45938/0 (socket says 172.21.15.83:45938) 2024-09-21T03:19:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7db7fe640 1 -- 172.21.15.83:0/2230400369 learned_addr learned my addr 172.21.15.83:0/2230400369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.396+0000 7fa7e8af4740 10 client.0 _ll_register_callbacks cb 0x55cf419019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 0x55cf41a0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 -- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 msgr2=0x55cf419f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 0x55cf419f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 -- 172.21.15.83:0/2230400369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 msgr2=0x55cf41a0fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 0x55cf41a0fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 -- 172.21.15.83:0/2230400369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf41915c90 con 0x55cf4161f390 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 0x55cf41a0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.397+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009380 tx=0x7fa7cc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9288481e1a988c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.398+0000 7fa7bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.398+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc029070 con 0x55cf4161f390 2024-09-21T03:19:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.398+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7cc003a00 con 0x55cf4161f390 2024-09-21T03:19:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.398+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc0054d0 con 0x55cf4161f390 2024-09-21T03:19:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc033050 con 0x55cf4161f390 2024-09-21T03:19:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7e8af4740 10 client.25448 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf41a12760 con 0x55cf4161f390 2024-09-21T03:19:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc009850 con 0x55cf4161f390 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa7cc02e440 con 0x55cf4161f390 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7e8af4740 10 client.25448 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7e8af4740 10 client.25448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.399+0000 7fa7e8af4740 20 client.25448 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415fe9f0 0x55cf41a15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7e8af4740 4 client.25448 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa7cc0054d0 con 0x55cf415fe9f0 2024-09-21T03:19:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415fe9f0 0x55cf41a15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7bb7fe640 10 client.25448 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.400+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415fe9f0 0x55cf41a15bd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21dc065d60ed85 server_cookie=17791ebcc3bdf35d in_seq=0 out_seq=0 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.401+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/2230400369 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa7cc0054d0 con 0x55cf415fe9f0 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.401+0000 7fa7bb7fe640 10 client.25448 handle_command_reply: tid=0 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.401+0000 7fa7e8af4740 1 client.25448 shutdown 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.401+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 msgr2=0x55cf419f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.401+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009380 tx=0x7fa7cc009f60 comp rx=0 tx=0).stop 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 shutdown_connections 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf41640070 0x55cf41a0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf4161f390 0x55cf419f2e70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415fe9f0 0x55cf41a15bd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2230400369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf416171b0 0x55cf419f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7bb7fe640 0 client.25448 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 >> 172.21.15.83:0/2230400369 conn(0x55cf41634460 msgr2=0x55cf419ea450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 shutdown_connections 2024-09-21T03:19:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2230400369 wait complete. 2024-09-21T03:19:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.402+0000 7fa7e8af4740 20 client.25448 trim_cache size 0 max 16384 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.425+0000 7fa7e8af4740 1 Processor -- start 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.425+0000 7fa7e8af4740 1 -- start start 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 0x55cf419f4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc075380 con 0x55cf416171b0 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1a6810 con 0x55cf41640070 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45954/0 (socket says 172.21.15.83:45954) 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2379596926 learned_addr learned my addr 172.21.15.83:0/2379596926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.426+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1a52b0 con 0x55cf4161f390 2024-09-21T03:19:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 0x55cf419f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2379596926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 msgr2=0x55cf419f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 0x55cf419f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2379596926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 msgr2=0x55cf419ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/2379596926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf419e05d0 con 0x55cf41640070 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.427+0000 7fa7db7fe640 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 0x55cf419f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.428+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ad3b0 tx=0x7fa7d0095040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c9ec8cc86cf10d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.428+0000 7fa7c1ffb640 1 -- 172.21.15.83:0/2379596926 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b1a30 con 0x55cf41640070 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.428+0000 7fa7c1ffb640 1 -- 172.21.15.83:0/2379596926 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d00b6970 con 0x55cf41640070 2024-09-21T03:19:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.428+0000 7fa7c1ffb640 1 -- 172.21.15.83:0/2379596926 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00ac680 con 0x55cf41640070 2024-09-21T03:19:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 msgr2=0x55cf41a0fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ad3b0 tx=0x7fa7d0095040 comp rx=0 tx=0).stop 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 shutdown_connections 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf41640070 0x55cf41a0fca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4161f390 0x55cf419f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/2379596926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf416171b0 0x55cf419ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 >> 172.21.15.83:0/2379596926 conn(0x55cf41634460 msgr2=0x55cf419f3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 shutdown_connections 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.429+0000 7fa7e8af4740 1 -- 172.21.15.83:0/2379596926 wait complete. 2024-09-21T03:19:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.430+0000 7fa7e8af4740 1 -- start start 2024-09-21T03:19:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.430+0000 7fa7e8af4740 1 Processor -- start 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf416171b0 0x55cf41a0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4161f390 0x55cf41a07750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1a6080 con 0x55cf4161f390 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1a59b0 con 0x55cf416171b0 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc073360 con 0x55cf41640070 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e8af4740 10 client.0 _ll_register_callbacks cb 0x7fa7cc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e0b1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55866/0 (socket says 172.21.15.83:55866) 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.431+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/1498352340 learned_addr learned my addr 172.21.15.83:0/1498352340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf416171b0 0x55cf41a0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/1498352340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf416171b0 msgr2=0x55cf41a0c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf416171b0 0x55cf41a0c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/1498352340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4161f390 msgr2=0x55cf41a07750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4161f390 0x55cf41a07750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 -- 172.21.15.83:0/1498352340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf41a081d0 con 0x55cf41640070 2024-09-21T03:19:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7e0b1d640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00acd00 tx=0x7fa7d000ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44345f0483fb33d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.432+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0099020 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d00b6970 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa7d005d1b0 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7d00a9a70 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 10 client.25417 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf41a800f0 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d00b6970 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa7d00b0070 con 0x55cf41640070 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 10 client.25417 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 10 client.25417 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.433+0000 7fa7e8af4740 20 client.25417 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415d3da0 0x55cf41a82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7e8af4740 4 client.25417 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf4195fdf0 con 0x55cf415d3da0 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415d3da0 0x55cf41a82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7dbfff640 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415d3da0 0x55cf41a82700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9546f518f5f108e4 server_cookie=6185668974096ae0 in_seq=0 out_seq=0 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.434+0000 7fa7bb7fe640 10 client.25417 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.435+0000 7fa7bb7fe640 1 -- 172.21.15.83:0/1498352340 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf4195fdf0 con 0x55cf415d3da0 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.435+0000 7fa7bb7fe640 10 client.25417 handle_command_reply: tid=0 2024-09-21T03:19:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.435+0000 7fa7e8af4740 1 client.25417 shutdown 2024-09-21T03:19:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 msgr2=0x55cf41a07c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00acd00 tx=0x7fa7d000ffa0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 shutdown_connections 2024-09-21T03:19:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55cf415d3da0 0x55cf41a82700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf41640070 0x55cf41a07c90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf4161f390 0x55cf41a07750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 --2- 172.21.15.83:0/1498352340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf416171b0 0x55cf41a0c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 >> 172.21.15.83:0/1498352340 conn(0x55cf41634460 msgr2=0x55cf419e82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7bb7fe640 0 client.25417 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.436+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 shutdown_connections 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e8af4740 1 -- 172.21.15.83:0/1498352340 wait complete. 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e8af4740 20 client.25417 trim_cache size 0 max 16384 2024-09-21T03:19:10.440 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 msgr2=0x7fa7c40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 0x7fa7c40b8c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40094e0 tx=0x7fa7d4031450 comp rx=0 tx=0).stop 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 msgr2=0x7fa7dc19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000f890 tx=0x7fa7d000fd50 comp rx=0 tx=0).stop 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 shutdown_connections 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7c40b6780 0x7fa7c40b8c90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc108cf0 0x7fa7dc19da70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7dc1083a0 0x7fa7dc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 --2- 172.21.15.83:0/2847652856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7dc102a60 0x7fa7dc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.437+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 >> 172.21.15.83:0/2847652856 conn(0x7fa7dc079730 msgr2=0x7fa7dc1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.438+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 shutdown_connections 2024-09-21T03:19:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:10.438+0000 7fa7e2580640 1 -- 172.21.15.83:0/2847652856 wait complete. 2024-09-21T03:19:10.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:10 smithi193 ceph-mon[35224]: pgmap v400: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 42 MiB/s wr, 1.59k op/s 2024-09-21T03:19:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:10 smithi083 ceph-mon[31219]: pgmap v400: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 42 MiB/s wr, 1.59k op/s 2024-09-21T03:19:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:10 smithi142 ceph-mon[34512]: pgmap v400: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 42 MiB/s wr, 1.59k op/s 2024-09-21T03:19:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 110.0 s, 2987.6 tps, lat 33.543 ms stddev 35.434 2024-09-21T03:19:11.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:19:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:11 smithi083 ceph-mon[31219]: pgmap v401: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 39 MiB/s wr, 1.41k op/s 2024-09-21T03:19:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:11 smithi142 ceph-mon[34512]: pgmap v401: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 39 MiB/s wr, 1.41k op/s 2024-09-21T03:19:11.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:11 smithi193 ceph-mon[35224]: pgmap v401: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 39 MiB/s wr, 1.41k op/s 2024-09-21T03:19:12.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 -- 172.21.15.83:0/2286195175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 msgr2=0x7f046c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 --2- 172.21.15.83:0/2286195175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c110470 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0464009ca0 tx=0x7f046402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 -- 172.21.15.83:0/2286195175 shutdown_connections 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 --2- 172.21.15.83:0/2286195175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c110470 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 --2- 172.21.15.83:0/2286195175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 --2- 172.21.15.83:0/2286195175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.310+0000 7f0472d08640 1 -- 172.21.15.83:0/2286195175 >> 172.21.15.83:0/2286195175 conn(0x7f046c0ff440 msgr2=0x7f046c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.311+0000 7f0472d08640 1 -- 172.21.15.83:0/2286195175 shutdown_connections 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.311+0000 7f0472d08640 1 -- 172.21.15.83:0/2286195175 wait complete. 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.311+0000 7f0472d08640 1 Processor -- start 2024-09-21T03:19:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.312+0000 7f0472d08640 1 -- start start 2024-09-21T03:19:12.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.312+0000 7f0472d08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.312+0000 7f0472d08640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.312+0000 7f0472d08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c1a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0472d08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f046c113950 con 0x7f046c109c40 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0472d08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1137d0 con 0x7f046c1039b0 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0472d08640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f046c113ad0 con 0x7f046c1092f0 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471505640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45986/0 (socket says 172.21.15.83:45986) 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471505640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c1a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55888/0 (socket says 172.21.15.83:55888) 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 -- 172.21.15.83:0/3230344310 learned_addr learned my addr 172.21.15.83:0/3230344310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0472507640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c1a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 -- 172.21.15.83:0/3230344310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 msgr2=0x7f046c1a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c1a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 -- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 msgr2=0x7f046c1a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c1a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.313+0000 7f0471d06640 1 -- 172.21.15.83:0/3230344310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c1ac970 con 0x7f046c1039b0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0472507640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c1a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0471d06640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f046800ea60 tx=0x7f046800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0462ffd640 1 -- 172.21.15.83:0/3230344310 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046800cde0 con 0x7f046c1039b0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0462ffd640 1 -- 172.21.15.83:0/3230344310 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04680035f0 con 0x7f046c1039b0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1acc60 con 0x7f046c1039b0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.314+0000 7f0462ffd640 1 -- 172.21.15.83:0/3230344310 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468010690 con 0x7f046c1039b0 2024-09-21T03:19:12.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.315+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1ad1a0 con 0x7f046c1039b0 2024-09-21T03:19:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.319+0000 7f047a4de740 1 Processor -- start 2024-09-21T03:19:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 -- start start 2024-09-21T03:19:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b43860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee277b390 0x55bee2b442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bee2ab9ea0 con 0x55bee27731b0 2024-09-21T03:19:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bee2aa92d0 con 0x55bee279c070 2024-09-21T03:19:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f047a4de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bee2aad0e0 con 0x55bee277b390 2024-09-21T03:19:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.321+0000 7f0462ffd640 1 -- 172.21.15.83:0/3230344310 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f04680037a0 con 0x7f046c1039b0 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.323+0000 7f0462ffd640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 0x7f04580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.323+0000 7f0462ffd640 1 -- 172.21.15.83:0/3230344310 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04680e7a90 con 0x7f046c1039b0 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.323+0000 7f0472507640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b43860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.324+0000 7f0471d06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.324+0000 7f0471d06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37850/0 (socket says 172.21.15.83:37850) 2024-09-21T03:19:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.324+0000 7f0471d06640 1 -- 172.21.15.83:0/1433573680 learned_addr learned my addr 172.21.15.83:0/1433573680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0471d06640 1 -- 172.21.15.83:0/1433573680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee277b390 msgr2=0x55bee2b442e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0471d06640 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee277b390 0x55bee2b442e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0471d06640 1 -- 172.21.15.83:0/1433573680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 msgr2=0x55bee2b43860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0471d06640 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b43860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0471d06640 1 -- 172.21.15.83:0/1433573680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee2807650 con 0x55bee27731b0 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.325+0000 7f0472507640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 0x7f04580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.326+0000 7f0472507640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 0x7f04580b8c10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f04640094e0 tx=0x7f046403f040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.327+0000 7f0471d06640 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f04680a9ed0 tx=0x7f04680aa070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265f65b1689de478 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.328+0000 7f0457fff640 1 -- 172.21.15.83:0/1433573680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680ad250 con 0x55bee27731b0 2024-09-21T03:19:12.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.328+0000 7f0457fff640 1 -- 172.21.15.83:0/1433573680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04680ad410 con 0x55bee27731b0 2024-09-21T03:19:12.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.328+0000 7f0457fff640 1 -- 172.21.15.83:0/1433573680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680ad070 con 0x55bee27731b0 2024-09-21T03:19:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 -- 172.21.15.83:0/1433573680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 msgr2=0x55bee2b43da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f04680a9ed0 tx=0x7f04680aa070 comp rx=0 tx=0).stop 2024-09-21T03:19:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 -- 172.21.15.83:0/1433573680 shutdown_connections 2024-09-21T03:19:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee277b390 0x55bee2b442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee27731b0 0x55bee2b43da0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 --2- 172.21.15.83:0/1433573680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b43860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.330+0000 7f047a4de740 1 -- 172.21.15.83:0/1433573680 >> 172.21.15.83:0/1433573680 conn(0x55bee2790460 msgr2=0x55bee2b41750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.331+0000 7f047a4de740 1 -- 172.21.15.83:0/1433573680 shutdown_connections 2024-09-21T03:19:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.331+0000 7f047a4de740 1 -- 172.21.15.83:0/1433573680 wait complete. 2024-09-21T03:19:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.331+0000 7f047a4de740 1 -- start start 2024-09-21T03:19:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.331+0000 7f047a4de740 1 Processor -- start 2024-09-21T03:19:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b6c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bee290a700 con 0x55bee277b390 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bee2a44160 con 0x55bee279c070 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bee2b62fa0 con 0x55bee27731b0 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f047a4de740 10 client.0 _ll_register_callbacks cb 0x55bee2a5d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0471505640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0471d06640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0471505640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37862/0 (socket says 172.21.15.83:37862) 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0471d06640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55904/0 (socket says 172.21.15.83:55904) 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0471505640 1 -- 172.21.15.83:0/558254478 learned_addr learned my addr 172.21.15.83:0/558254478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.332+0000 7f0472507640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b6c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 -- 172.21.15.83:0/558254478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 msgr2=0x55bee2b5ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 -- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 msgr2=0x55bee2b6c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b6c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 -- 172.21.15.83:0/558254478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee2a71c90 con 0x55bee277b390 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471d06640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f0471505640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f045c0116f0 tx=0x7f045c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa561eb74ea7f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f04557fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.333+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c00f920 con 0x55bee277b390 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f045c00fac0 con 0x55bee277b390 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04680ad410 con 0x55bee277b390 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f045c00fcc0 con 0x55bee277b390 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 10 client.15930 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bee2b6ed30 con 0x55bee277b390 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04680ad410 con 0x55bee277b390 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f045c018070 con 0x55bee277b390 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 10 client.15930 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 20 client.15930 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee275a9f0 0x55bee2b72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 4 client.15930 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:12.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.334+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bee2abbdf0 con 0x55bee275a9f0 2024-09-21T03:19:12.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.335+0000 7f0471d06640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee275a9f0 0x55bee2b72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.335+0000 7f04557fa640 10 client.15930 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:19:12.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.335+0000 7f0471d06640 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee275a9f0 0x55bee2b72530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7b760bd518566db server_cookie=b5adf294876ce3f9 in_seq=0 out_seq=0 2024-09-21T03:19:12.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f04557fa640 1 -- 172.21.15.83:0/558254478 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bee2abbdf0 con 0x55bee275a9f0 2024-09-21T03:19:12.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f04557fa640 10 client.15930 handle_command_reply: tid=0 2024-09-21T03:19:12.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f047a4de740 1 client.15930 shutdown 2024-09-21T03:19:12.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 msgr2=0x55bee2b60480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f045c0116f0 tx=0x7f045c011bb0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 shutdown_connections 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.336+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee275a9f0 0x55bee2b72530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee279c070 0x55bee2b6c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f04557fa640 0 client.15930 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:19:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b60480 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 --2- 172.21.15.83:0/558254478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b5ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 >> 172.21.15.83:0/558254478 conn(0x55bee2790460 msgr2=0x55bee2b45250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 shutdown_connections 2024-09-21T03:19:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 1 -- 172.21.15.83:0/558254478 wait complete. 2024-09-21T03:19:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.337+0000 7f047a4de740 20 client.15930 trim_cache size 0 max 16384 2024-09-21T03:19:12.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.358+0000 7f047a4de740 1 Processor -- start 2024-09-21T03:19:12.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.358+0000 7f047a4de740 1 -- start start 2024-09-21T03:19:12.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee27731b0 0x55bee2b60400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee279c070 0x55bee2b410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f046c075400 con 0x55bee277b390 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1aee40 con 0x55bee27731b0 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f047a4de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1ad8e0 con 0x55bee279c070 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37868/0 (socket says 172.21.15.83:37868) 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 -- 172.21.15.83:0/4113638491 learned_addr learned my addr 172.21.15.83:0/4113638491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471d06640 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee27731b0 0x55bee2b60400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0472507640 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee279c070 0x55bee2b410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 -- 172.21.15.83:0/4113638491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee279c070 msgr2=0x55bee2b410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee279c070 0x55bee2b410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 -- 172.21.15.83:0/4113638491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee27731b0 msgr2=0x55bee2b60400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee27731b0 0x55bee2b60400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.359+0000 7f0471505640 1 -- 172.21.15.83:0/4113638491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee2b3c430 con 0x55bee277b390 2024-09-21T03:19:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.360+0000 7f0471505640 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f045c000c00 tx=0x7f045c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19795bd3f7126310 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.360+0000 7f04577fe640 1 -- 172.21.15.83:0/4113638491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c00fb20 con 0x55bee277b390 2024-09-21T03:19:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.360+0000 7f04577fe640 1 -- 172.21.15.83:0/4113638491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045c01ac60 con 0x55bee277b390 2024-09-21T03:19:12.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.360+0000 7f04577fe640 1 -- 172.21.15.83:0/4113638491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c003a80 con 0x55bee277b390 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.361+0000 7f047a4de740 1 -- 172.21.15.83:0/4113638491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 msgr2=0x55bee2b6c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.361+0000 7f047a4de740 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f045c000c00 tx=0x7f045c009290 comp rx=0 tx=0).stop 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.361+0000 7f047a4de740 1 -- 172.21.15.83:0/4113638491 shutdown_connections 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.361+0000 7f047a4de740 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee279c070 0x55bee2b410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.361+0000 7f047a4de740 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee277b390 0x55bee2b6c240 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.362+0000 7f047a4de740 1 --2- 172.21.15.83:0/4113638491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee27731b0 0x55bee2b60400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.362+0000 7f047a4de740 1 -- 172.21.15.83:0/4113638491 >> 172.21.15.83:0/4113638491 conn(0x55bee2790460 msgr2=0x55bee2b5fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.362+0000 7f047a4de740 1 -- 172.21.15.83:0/4113638491 shutdown_connections 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.362+0000 7f047a4de740 1 -- 172.21.15.83:0/4113638491 wait complete. 2024-09-21T03:19:12.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.363+0000 7f047a4de740 1 -- start start 2024-09-21T03:19:12.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.363+0000 7f047a4de740 1 Processor -- start 2024-09-21T03:19:12.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.365+0000 7f047a4de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.365+0000 7f047a4de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.365+0000 7f047a4de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.365+0000 7f0471d06640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.365+0000 7f0471d06640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55918/0 (socket says 172.21.15.83:55918) 2024-09-21T03:19:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.366+0000 7f0471505640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.366+0000 7f0471505640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46020/0 (socket says 172.21.15.83:46020) 2024-09-21T03:19:12.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.366+0000 7f0472507640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.367+0000 7f0472507640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37884/0 (socket says 172.21.15.83:37884) 2024-09-21T03:19:12.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.367+0000 7f0472507640 1 -- 172.21.15.83:0/636213801 learned_addr learned my addr 172.21.15.83:0/636213801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:12.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.367+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1ae6b0 con 0x55bee279c070 2024-09-21T03:19:12.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.367+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1adfe0 con 0x55bee277b390 2024-09-21T03:19:12.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.367+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0733e0 con 0x55bee27731b0 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f047a4de740 10 client.0 _ll_register_callbacks cb 0x7f045c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 -- 172.21.15.83:0/636213801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 msgr2=0x55bee2b75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 -- 172.21.15.83:0/636213801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 msgr2=0x55bee2b767a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee2b46620 con 0x55bee277b390 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471d06640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:12.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0472507640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f0471505640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f045c00f890 tx=0x7f045c01a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29919c540716da38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f04557fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c005640 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.368+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f045c003910 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.369+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045c01ac60 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.369+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f045c00fe60 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.369+0000 7f047a4de740 10 client.25469 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.369+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bee2bdcfd0 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.370+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045c003bb0 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.370+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f045c018040 con 0x55bee277b390 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 10 client.25469 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 10 client.25469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 20 client.25469 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:12.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee272fda0 0x55bee2bdf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:12.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 4 client.25469 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:12.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f045c01ac60 con 0x55bee272fda0 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.371+0000 7f0471d06640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee272fda0 0x55bee2bdf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.372+0000 7f04557fa640 10 client.25469 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f0471d06640 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee272fda0 0x55bee2bdf670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b83bd91261640104 server_cookie=83bbc33a292f6f6f in_seq=0 out_seq=0 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f04557fa640 1 -- 172.21.15.83:0/636213801 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f045c01ac60 con 0x55bee272fda0 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f04557fa640 10 client.25469 handle_command_reply: tid=0 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 client.25469 shutdown 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 msgr2=0x55bee2b76260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f045c00f890 tx=0x7f045c01a9e0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 shutdown_connections 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55bee272fda0 0x55bee2bdf670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee279c070 0x55bee2b767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bee277b390 0x55bee2b76260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 --2- 172.21.15.83:0/636213801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bee27731b0 0x55bee2b75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 >> 172.21.15.83:0/636213801 conn(0x55bee2790460 msgr2=0x55bee2b410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.373+0000 7f04557fa640 0 client.25469 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.374+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 shutdown_connections 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.374+0000 7f047a4de740 1 -- 172.21.15.83:0/636213801 wait complete. 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.374+0000 7f047a4de740 20 client.25469 trim_cache size 0 max 16384 2024-09-21T03:19:12.376 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:12.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.375+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 msgr2=0x7f04580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.375+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 0x7f04580b8c10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f04640094e0 tx=0x7f046403f040 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 msgr2=0x7f046c1a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f046800ea60 tx=0x7f046800ef20 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 shutdown_connections 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04580b6700 0x7f04580b8c10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f046c109c40 0x7f046c1a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f046c1092f0 0x7f046c1a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 --2- 172.21.15.83:0/3230344310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f046c1039b0 0x7f046c1a1610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 >> 172.21.15.83:0/3230344310 conn(0x7f046c0ff440 msgr2=0x7f046c108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 shutdown_connections 2024-09-21T03:19:12.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:12.376+0000 7f0472d08640 1 -- 172.21.15.83:0/3230344310 wait complete. 2024-09-21T03:19:13.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:19:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:14.471 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:14 smithi083 ceph-mon[31219]: pgmap v402: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 41 MiB/s wr, 1.57k op/s 2024-09-21T03:19:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 -- 172.21.15.83:0/1999002852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 msgr2=0x7f0114109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 --2- 172.21.15.83:0/1999002852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f0114109910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0108009900 tx=0x7f010802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 -- 172.21.15.83:0/1999002852 shutdown_connections 2024-09-21T03:19:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 --2- 172.21.15.83:0/1999002852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f0114110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 --2- 172.21.15.83:0/1999002852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f0114109910 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 --2- 172.21.15.83:0/1999002852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f01141023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.512+0000 7f011a406640 1 -- 172.21.15.83:0/1999002852 >> 172.21.15.83:0/1999002852 conn(0x7f0114079f90 msgr2=0x7f011407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.513+0000 7f011a406640 1 -- 172.21.15.83:0/1999002852 shutdown_connections 2024-09-21T03:19:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.513+0000 7f011a406640 1 -- 172.21.15.83:0/1999002852 wait complete. 2024-09-21T03:19:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.513+0000 7f011a406640 1 Processor -- start 2024-09-21T03:19:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.513+0000 7f011a406640 1 -- start start 2024-09-21T03:19:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.514+0000 7f011a406640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f0114102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.514+0000 7f011a406640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f01141034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.514+0000 7f011a406640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f01141039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f0113fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f0114102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f01137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f01141034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f0113fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f0114102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55924/0 (socket says 172.21.15.83:55924) 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f0113fff640 1 -- 172.21.15.83:0/1166168833 learned_addr learned my addr 172.21.15.83:0/1166168833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0114113b60 con 0x7f0114109500 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01141139e0 con 0x7f0114109e50 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f01189a3640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f01141039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.515+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0114113ce0 con 0x7f0114101fc0 2024-09-21T03:19:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 -- 172.21.15.83:0/1166168833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 msgr2=0x7f0114102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f0114102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 -- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 msgr2=0x7f01141034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f01141034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141a8830 con 0x7f0114109e50 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01137fe640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f01141034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.516+0000 7f01189a3640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f01141039e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f010400e970 tx=0x7f010400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.517+0000 7f01117fa640 1 -- 172.21.15.83:0/1166168833 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010400ccc0 con 0x7f0114109e50 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.517+0000 7f01117fa640 1 -- 172.21.15.83:0/1166168833 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0104005020 con 0x7f0114109e50 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.517+0000 7f01117fa640 1 -- 172.21.15.83:0/1166168833 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104004a00 con 0x7f0114109e50 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.517+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a8b20 con 0x7f0114109e50 2024-09-21T03:19:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.517+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a8f80 con 0x7f0114109e50 2024-09-21T03:19:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.520+0000 7f012097a740 1 Processor -- start 2024-09-21T03:19:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.521+0000 7f01117fa640 1 -- 172.21.15.83:0/1166168833 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0104004ba0 con 0x7f0114109e50 2024-09-21T03:19:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 -- start start 2024-09-21T03:19:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01117fa640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 0x7f00fc0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 0x557f49a19eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01137fe640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 0x7f00fc0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a1a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f49987ea0 con 0x557f496411b0 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557f499772d0 con 0x557f49649390 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f012097a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f4997b0e0 con 0x557f4966a070 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01117fa640 1 -- 172.21.15.83:0/1166168833 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f01040e75e0 con 0x7f0114109e50 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37900/0 (socket says 172.21.15.83:37900) 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01137fe640 1 -- 172.21.15.83:0/4241105263 learned_addr learned my addr 172.21.15.83:0/4241105263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f0113fff640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 0x557f49a19eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.523+0000 7f01189a3640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a1a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 -- 172.21.15.83:0/4241105263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 msgr2=0x557f49a19eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 0x557f49a19eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 -- 172.21.15.83:0/4241105263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 msgr2=0x557f49a1a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a1a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 -- 172.21.15.83:0/4241105263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f496d5650 con 0x557f496411b0 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f0113fff640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 0x557f49a19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0108006010 tx=0x7f0108004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a27f0b3ffb9131b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f01137fe640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 0x7f00fc0b8c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0108047000 tx=0x7f0108048000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f00fa7fc640 1 -- 172.21.15.83:0/4241105263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01080037f0 con 0x557f496411b0 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f00fa7fc640 1 -- 172.21.15.83:0/4241105263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0108003e40 con 0x557f496411b0 2024-09-21T03:19:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.524+0000 7f00fa7fc640 1 -- 172.21.15.83:0/4241105263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108031bf0 con 0x557f496411b0 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- 172.21.15.83:0/4241105263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 msgr2=0x557f49a1a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0108006010 tx=0x7f0108004830 comp rx=0 tx=0).stop 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- 172.21.15.83:0/4241105263 shutdown_connections 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a1a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f496411b0 0x557f49a1a3f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 --2- 172.21.15.83:0/4241105263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4966a070 0x557f49a19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- 172.21.15.83:0/4241105263 >> 172.21.15.83:0/4241105263 conn(0x557f4965e460 msgr2=0x557f49a17da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- 172.21.15.83:0/4241105263 shutdown_connections 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- 172.21.15.83:0/4241105263 wait complete. 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.526+0000 7f012097a740 1 -- start start 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 Processor -- start 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f497d8700 con 0x557f49649390 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557f49912160 con 0x557f4966a070 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f49a396a0 con 0x557f496411b0 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.527+0000 7f012097a740 10 client.0 _ll_register_callbacks cb 0x557f4992b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f0113fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37910/0 (socket says 172.21.15.83:37910) 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46052/0 (socket says 172.21.15.83:46052) 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01137fe640 1 -- 172.21.15.83:0/1336767724 learned_addr learned my addr 172.21.15.83:0/1336767724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 -- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 msgr2=0x557f49a25610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a25610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 -- 172.21.15.83:0/1336767724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 msgr2=0x557f49a25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01189a3640 1 -- 172.21.15.83:0/1336767724 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f4993fc90 con 0x557f4966a070 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.528+0000 7f01137fe640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f01189a3640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f01040e4d50 tx=0x7f01040acc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b8e54a06ca8e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010401c040 con 0x557f4966a070 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f01040a7d50 con 0x557f4966a070 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0108003e40 con 0x557f4966a070 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01040e51e0 con 0x557f4966a070 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 10 client.25481 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f49a45450 con 0x557f4966a070 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01040b6e70 con 0x557f4966a070 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f01040a8f30 con 0x557f4966a070 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 10 client.25481 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 10 client.25481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 20 client.25481 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f496289f0 0x557f49a48880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 4 client.25481 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.529+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0108003e40 con 0x557f496289f0 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.530+0000 7f0113fff640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f496289f0 0x557f49a48880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.530+0000 7f00f37fe640 10 client.25481 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:19:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.530+0000 7f0113fff640 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f496289f0 0x557f49a48880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9f4a14d4f44c95f server_cookie=682f73d4e5ff2daf in_seq=0 out_seq=0 2024-09-21T03:19:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.531+0000 7f00f37fe640 1 -- 172.21.15.83:0/1336767724 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0108003e40 con 0x557f496289f0 2024-09-21T03:19:14.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.531+0000 7f00f37fe640 10 client.25481 handle_command_reply: tid=0 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 client.25481 shutdown 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 msgr2=0x557f49a42990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f01040e4d50 tx=0x7f01040acc80 comp rx=0 tx=0).stop 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 shutdown_connections 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f496289f0 0x557f49a48880 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a42990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 --2- 172.21.15.83:0/1336767724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a25610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 >> 172.21.15.83:0/1336767724 conn(0x557f4965e460 msgr2=0x557f49a1d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.533+0000 7f00f37fe640 0 client.25481 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.534+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 shutdown_connections 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.534+0000 7f012097a740 1 -- 172.21.15.83:0/1336767724 wait complete. 2024-09-21T03:19:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.534+0000 7f012097a740 20 client.25481 trim_cache size 0 max 16384 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.557+0000 7f012097a740 1 Processor -- start 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.557+0000 7f012097a740 1 -- start start 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f4966a070 0x557f49a22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0114070ef0 con 0x557f4966a070 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0114072b50 con 0x557f49649390 2024-09-21T03:19:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f012097a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01140715f0 con 0x557f496411b0 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f01137fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f01137fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46058/0 (socket says 172.21.15.83:46058) 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f01137fe640 1 -- 172.21.15.83:0/3679491856 learned_addr learned my addr 172.21.15.83:0/3679491856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f01189a3640 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f4966a070 0x557f49a22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.558+0000 7f0113fff640 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 -- 172.21.15.83:0/3679491856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 msgr2=0x557f49a1b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a1b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 -- 172.21.15.83:0/3679491856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f4966a070 msgr2=0x557f49a22800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f4966a070 0x557f49a22800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 -- 172.21.15.83:0/3679491856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f49a1c1a0 con 0x557f49649390 2024-09-21T03:19:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f01137fe640 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f010802a430 tx=0x7f01080043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2309ac9a14542d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f00f9ffb640 1 -- 172.21.15.83:0/3679491856 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108005df0 con 0x557f49649390 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f00f9ffb640 1 -- 172.21.15.83:0/3679491856 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010805a070 con 0x557f49649390 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.559+0000 7f00f9ffb640 1 -- 172.21.15.83:0/3679491856 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108031790 con 0x557f49649390 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 -- 172.21.15.83:0/3679491856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 msgr2=0x557f49a42890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f010802a430 tx=0x7f01080043f0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 -- 172.21.15.83:0/3679491856 shutdown_connections 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f4966a070 0x557f49a22800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f49649390 0x557f49a42890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 --2- 172.21.15.83:0/3679491856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a1b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.560+0000 7f012097a740 1 -- 172.21.15.83:0/3679491856 >> 172.21.15.83:0/3679491856 conn(0x557f4965e460 msgr2=0x557f49a1aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.561+0000 7f012097a740 1 -- 172.21.15.83:0/3679491856 shutdown_connections 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.561+0000 7f012097a740 1 -- 172.21.15.83:0/3679491856 wait complete. 2024-09-21T03:19:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.561+0000 7f012097a740 1 -- start start 2024-09-21T03:19:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.561+0000 7f012097a740 1 Processor -- start 2024-09-21T03:19:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a3f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a3a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01140723c0 con 0x557f49649390 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0114071cf0 con 0x557f4966a070 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f011406eed0 con 0x557f496411b0 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.562+0000 7f012097a740 10 client.0 _ll_register_callbacks cb 0x7f01040a92f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f0113fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46062/0 (socket says 172.21.15.83:46062) 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f0113fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a3f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55954/0 (socket says 172.21.15.83:55954) 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 -- 172.21.15.83:0/166738255 learned_addr learned my addr 172.21.15.83:0/166738255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01137fe640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a3a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 -- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 msgr2=0x557f49a3f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a3f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 -- 172.21.15.83:0/166738255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 msgr2=0x557f49a3a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a3a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.563+0000 7f01189a3640 1 -- 172.21.15.83:0/166738255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f49a3b310 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f01189a3640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01040ad150 tx=0x7f0104005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c41de8ee019bae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f00f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010401c070 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010805a070 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f01040a7b50 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01040acda0 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f012097a740 10 client.25487 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.564+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f49ab31b0 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010805a070 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f01040b30c0 con 0x557f4966a070 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 10 client.25487 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 10 client.25487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:19:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 20 client.25487 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f495fdda0 0x557f49ab59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 4 client.25487 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f49989df0 con 0x557f495fdda0 2024-09-21T03:19:14.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:14 smithi193 ceph-mon[35224]: pgmap v402: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 41 MiB/s wr, 1.57k op/s 2024-09-21T03:19:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.565+0000 7f0113fff640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f495fdda0 0x557f49ab59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.566+0000 7f00f37fe640 10 client.25487 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.566+0000 7f0113fff640 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f495fdda0 0x557f49ab59a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be2107125c2d5310 server_cookie=dfd3930ef76ca56a in_seq=0 out_seq=0 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.566+0000 7f00f37fe640 1 -- 172.21.15.83:0/166738255 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f49989df0 con 0x557f495fdda0 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.566+0000 7f00f37fe640 10 client.25487 handle_command_reply: tid=0 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.566+0000 7f012097a740 1 client.25487 shutdown 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 msgr2=0x557f49a3ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01040ad150 tx=0x7f0104005700 comp rx=0 tx=0).stop 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 shutdown_connections 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x557f495fdda0 0x557f49ab59a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f4966a070 0x557f49a3ab80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f49649390 0x557f49a3a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 --2- 172.21.15.83:0/166738255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f496411b0 0x557f49a3f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 >> 172.21.15.83:0/166738255 conn(0x557f4965e460 msgr2=0x557f49a1cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f00f37fe640 0 client.25487 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:19:14.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 shutdown_connections 2024-09-21T03:19:14.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 1 -- 172.21.15.83:0/166738255 wait complete. 2024-09-21T03:19:14.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.567+0000 7f012097a740 20 client.25487 trim_cache size 0 max 16384 2024-09-21T03:19:14.572 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 msgr2=0x7f00fc0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 0x7f00fc0b8c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0108047000 tx=0x7f0108048000 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 msgr2=0x7f01141039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f01141039e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f010400e970 tx=0x7f010400ee30 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 shutdown_connections 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f00fc0b6730 0x7f00fc0b8c40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0114109e50 0x7f01141039e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0114109500 0x7f01141034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 --2- 172.21.15.83:0/1166168833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0114101fc0 0x7f0114102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 >> 172.21.15.83:0/1166168833 conn(0x7f0114079f90 msgr2=0x7f01141075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 shutdown_connections 2024-09-21T03:19:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:14.568+0000 7f011a406640 1 -- 172.21.15.83:0/1166168833 wait complete. 2024-09-21T03:19:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:14 smithi142 ceph-mon[34512]: pgmap v402: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 41 MiB/s wr, 1.57k op/s 2024-09-21T03:19:15.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:19:15.673 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 115.0 s, 3239.4 tps, lat 30.709 ms stddev 34.435 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 -- 172.21.15.83:0/950015027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c108cf0 msgr2=0x7f523c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 --2- 172.21.15.83:0/950015027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c108cf0 0x7f523c10f520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5234009ca0 tx=0x7f523402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 -- 172.21.15.83:0/950015027 shutdown_connections 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 --2- 172.21.15.83:0/950015027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c108cf0 0x7f523c10f520 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 --2- 172.21.15.83:0/950015027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 0x7f523c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 --2- 172.21.15.83:0/950015027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f523c102a60 0x7f523c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 -- 172.21.15.83:0/950015027 >> 172.21.15.83:0/950015027 conn(0x7f523c079730 msgr2=0x7f523c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 -- 172.21.15.83:0/950015027 shutdown_connections 2024-09-21T03:19:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.238+0000 7f5240efe640 1 -- 172.21.15.83:0/950015027 wait complete. 2024-09-21T03:19:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 Processor -- start 2024-09-21T03:19:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 -- start start 2024-09-21T03:19:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 0x7f523c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f523c108cf0 0x7f523c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c111860 con 0x7f523c1083a0 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1116e0 con 0x7f523c102a60 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.239+0000 7f5240efe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1119e0 con 0x7f523c108cf0 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46074/0 (socket says 172.21.15.83:46074) 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 -- 172.21.15.83:0/2121678807 learned_addr learned my addr 172.21.15.83:0/2121678807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f5239d9b640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 0x7f523c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 -- 172.21.15.83:0/2121678807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f523c108cf0 msgr2=0x7f523c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f523c108cf0 0x7f523c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 -- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 msgr2=0x7f523c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 0x7f523c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 -- 172.21.15.83:0/2121678807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c19e140 con 0x7f523c102a60 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.240+0000 7f523a59c640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f522c009db0 tx=0x7f522c00f4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.241+0000 7f522b7fe640 1 -- 172.21.15.83:0/2121678807 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00b020 con 0x7f523c102a60 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.241+0000 7f522b7fe640 1 -- 172.21.15.83:0/2121678807 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c00b1c0 con 0x7f523c102a60 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.241+0000 7f522b7fe640 1 -- 172.21.15.83:0/2121678807 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c017660 con 0x7f523c102a60 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.241+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c19e430 con 0x7f523c102a60 2024-09-21T03:19:16.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.241+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a4b90 con 0x7f523c102a60 2024-09-21T03:19:16.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.246+0000 7f522b7fe640 1 -- 172.21.15.83:0/2121678807 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f522c00a980 con 0x7f523c102a60 2024-09-21T03:19:16.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.246+0000 7f5247472740 1 Processor -- start 2024-09-21T03:19:16.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.246+0000 7f5247472740 1 -- start start 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ce5544ea0 con 0x560ce5206390 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560ce55342d0 con 0x560ce51fe1b0 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5247472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ce55380e0 con 0x560ce5227070 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f523ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f523ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55d2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55978/0 (socket says 172.21.15.83:55978) 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37964/0 (socket says 172.21.15.83:37964) 2024-09-21T03:19:16.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f523ad9d640 1 -- 172.21.15.83:0/3665737787 learned_addr learned my addr 172.21.15.83:0/3665737787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f522b7fe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 0x7f52240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f523a59c640 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 -- 172.21.15.83:0/3665737787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 msgr2=0x560ce55d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 -- 172.21.15.83:0/3665737787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 msgr2=0x560ce55d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.247+0000 7f5239d9b640 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f5239d9b640 1 -- 172.21.15.83:0/3665737787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce5292650 con 0x560ce5206390 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f522b7fe640 1 -- 172.21.15.83:0/2121678807 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f522c0e93a0 con 0x7f523c102a60 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f523a59c640 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f5239d9b640 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f523c06aaa0 tx=0x7f5230011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59093fd9f85a64f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f523ad9d640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 0x7f52240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f5228ff9640 1 -- 172.21.15.83:0/3665737787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523001e070 con 0x560ce5206390 2024-09-21T03:19:16.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f5228ff9640 1 -- 172.21.15.83:0/3665737787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5230009d70 con 0x560ce5206390 2024-09-21T03:19:16.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.248+0000 7f5228ff9640 1 -- 172.21.15.83:0/3665737787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230018730 con 0x560ce5206390 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f523ad9d640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 0x7f52240b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52340097b0 tx=0x7f52340095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 -- 172.21.15.83:0/3665737787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 msgr2=0x560ce55d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f523c06aaa0 tx=0x7f5230011810 comp rx=0 tx=0).stop 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 -- 172.21.15.83:0/3665737787 shutdown_connections 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55d3530 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 --2- 172.21.15.83:0/3665737787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.250+0000 7f5247472740 1 -- 172.21.15.83:0/3665737787 >> 172.21.15.83:0/3665737787 conn(0x560ce521b460 msgr2=0x560ce55d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.252+0000 7f5247472740 1 -- 172.21.15.83:0/3665737787 shutdown_connections 2024-09-21T03:19:16.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.252+0000 7f5247472740 1 -- 172.21.15.83:0/3665737787 wait complete. 2024-09-21T03:19:16.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 -- start start 2024-09-21T03:19:16.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 Processor -- start 2024-09-21T03:19:16.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 0x560ce55cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 0x560ce55cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ce5395700 con 0x560ce51fe1b0 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560ce54cf160 con 0x560ce5206390 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ce55e5670 con 0x560ce5227070 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5247472740 10 client.0 _ll_register_callbacks cb 0x560ce54e89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f523ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f523ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55986/0 (socket says 172.21.15.83:55986) 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 0x560ce55cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.253+0000 7f523ad9d640 1 -- 172.21.15.83:0/1205552697 learned_addr learned my addr 172.21.15.83:0/1205552697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f523a59c640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 0x560ce55cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 -- 172.21.15.83:0/1205552697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 msgr2=0x560ce55ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 -- 172.21.15.83:0/1205552697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 msgr2=0x560ce55cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 0x560ce55cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 -- 172.21.15.83:0/1205552697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce54fcc90 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f523a59c640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 0x560ce55cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5239d9b640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 0x560ce55cd670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f523001c040 tx=0x7f5230005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d242066f906452e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523001e040 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f5230005460 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.254+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5230009d70 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52300044d0 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 10 client.25496 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ce55f1320 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5230009d70 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f52300036f0 con 0x560ce5206390 2024-09-21T03:19:16.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 10 client.25496 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 20 client.25496 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51e59f0 0x560ce55f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 4 client.25496 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce5546df0 con 0x560ce51e59f0 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.255+0000 7f523a59c640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51e59f0 0x560ce55f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.256+0000 7f5221ffb640 10 client.25496 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.256+0000 7f523a59c640 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51e59f0 0x560ce55f4680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6b1a31655f4263 server_cookie=fc97ebc0c191be06 in_seq=0 out_seq=0 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.257+0000 7f5221ffb640 1 -- 172.21.15.83:0/1205552697 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f522c08ff90 con 0x560ce51e59f0 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.257+0000 7f5221ffb640 10 client.25496 handle_command_reply: tid=0 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.257+0000 7f5247472740 1 client.25496 shutdown 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.257+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 msgr2=0x560ce55cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.257+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 0x560ce55cd670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f523001c040 tx=0x7f5230005270 comp rx=0 tx=0).stop 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 shutdown_connections 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51e59f0 0x560ce55f4680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce5206390 0x560ce55cd670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 --2- 172.21.15.83:0/1205552697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce51fe1b0 0x560ce55cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 >> 172.21.15.83:0/1205552697 conn(0x560ce521b460 msgr2=0x560ce55d5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5221ffb640 0 client.25496 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 shutdown_connections 2024-09-21T03:19:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 1 -- 172.21.15.83:0/1205552697 wait complete. 2024-09-21T03:19:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.258+0000 7f5247472740 20 client.25496 trim_cache size 0 max 16384 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 Processor -- start 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 -- start start 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1ad390 con 0x560ce5206390 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1aeff0 con 0x560ce51fe1b0 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5247472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1ada90 con 0x560ce5227070 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f523a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37982/0 (socket says 172.21.15.83:37982) 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.280+0000 7f5239d9b640 1 -- 172.21.15.83:0/1086936690 learned_addr learned my addr 172.21.15.83:0/1086936690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523ad9d640 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 -- 172.21.15.83:0/1086936690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 msgr2=0x560ce55ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 -- 172.21.15.83:0/1086936690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 msgr2=0x560ce55e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 -- 172.21.15.83:0/1086936690 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce55ce3f0 con 0x560ce51fe1b0 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f5239d9b640 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f523a59c640 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55cd5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f522c0e6b10 tx=0x7f522c07adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ec15be1b625049 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f5223fff640 1 -- 172.21.15.83:0/1086936690 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c073760 con 0x560ce51fe1b0 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f5223fff640 1 -- 172.21.15.83:0/1086936690 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ce5546df0 con 0x560ce51fe1b0 2024-09-21T03:19:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.281+0000 7f5223fff640 1 -- 172.21.15.83:0/1086936690 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c07c0e0 con 0x560ce51fe1b0 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.282+0000 7f5247472740 1 -- 172.21.15.83:0/1086936690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 msgr2=0x560ce55cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.282+0000 7f5247472740 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55cd5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f522c0e6b10 tx=0x7f522c07adc0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 -- 172.21.15.83:0/1086936690 shutdown_connections 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 --2- 172.21.15.83:0/1086936690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55cd5f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 -- 172.21.15.83:0/1086936690 >> 172.21.15.83:0/1086936690 conn(0x560ce521b460 msgr2=0x560ce55ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 -- 172.21.15.83:0/1086936690 shutdown_connections 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.283+0000 7f5247472740 1 -- 172.21.15.83:0/1086936690 wait complete. 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.284+0000 7f5247472740 1 -- start start 2024-09-21T03:19:16.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.284+0000 7f5247472740 1 Processor -- start 2024-09-21T03:19:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55e7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1ae860 con 0x560ce5206390 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1ae190 con 0x560ce51fe1b0 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1ab370 con 0x560ce5227070 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37984/0 (socket says 172.21.15.83:37984) 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 -- 172.21.15.83:0/2785285721 learned_addr learned my addr 172.21.15.83:0/2785285721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f523a59c640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5247472740 10 client.0 _ll_register_callbacks cb 0x7f523001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f523ad9d640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 -- 172.21.15.83:0/2785285721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 msgr2=0x560ce55e7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55e7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 -- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 msgr2=0x560ce55ea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55ea8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.285+0000 7f5239d9b640 1 -- 172.21.15.83:0/2785285721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce55eb470 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5239d9b640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5230003b40 tx=0x7f5230009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e06747c5bb843c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523003c060 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f52300037f0 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ce5546df0 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52300052e0 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5247472740 10 client.15969 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.286+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ce565f380 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ce5546df0 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5230037470 con 0x560ce5206390 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 20 client.15969 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51bada0 0x560ce5661ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 4 client.15969 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ce55428a0 con 0x560ce51bada0 2024-09-21T03:19:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.287+0000 7f523a59c640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51bada0 0x560ce5661ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.288+0000 7f523a59c640 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51bada0 0x560ce5661ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528ed14c726400e5 server_cookie=68f220613c90b2ee in_seq=0 out_seq=0 2024-09-21T03:19:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.288+0000 7f5221ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:16.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.289+0000 7f5221ffb640 1 -- 172.21.15.83:0/2785285721 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ce55428a0 con 0x560ce51bada0 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.289+0000 7f5221ffb640 10 client.15969 handle_command_reply: tid=0 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.289+0000 7f5247472740 1 client.15969 shutdown 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 msgr2=0x560ce55eae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5230003b40 tx=0x7f5230009b00 comp rx=0 tx=0).stop 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 shutdown_connections 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560ce51bada0 0x560ce5661ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ce5227070 0x560ce55e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ce5206390 0x560ce55eae00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 --2- 172.21.15.83:0/2785285721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560ce51fe1b0 0x560ce55ea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 >> 172.21.15.83:0/2785285721 conn(0x560ce521b460 msgr2=0x560ce55ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5221ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.290+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 shutdown_connections 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.291+0000 7f5247472740 1 -- 172.21.15.83:0/2785285721 wait complete. 2024-09-21T03:19:16.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.292+0000 7f5247472740 20 client.15969 trim_cache size 0 max 16384 2024-09-21T03:19:16.295 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 msgr2=0x7f52240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 0x7f52240b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52340097b0 tx=0x7f52340095c0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 msgr2=0x7f523c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f522c009db0 tx=0x7f522c00f4a0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 shutdown_connections 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52240b6700 0x7f52240b8c10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f523c108cf0 0x7f523c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1083a0 0x7f523c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 --2- 172.21.15.83:0/2121678807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f523c102a60 0x7f523c1981d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 >> 172.21.15.83:0/2121678807 conn(0x7f523c079730 msgr2=0x7f523c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 shutdown_connections 2024-09-21T03:19:16.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:16.293+0000 7f5240efe640 1 -- 172.21.15.83:0/2121678807 wait complete. 2024-09-21T03:19:16.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:16 smithi083 ceph-mon[31219]: pgmap v403: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 962 op/s 2024-09-21T03:19:16.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:16 smithi193 ceph-mon[35224]: pgmap v403: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 962 op/s 2024-09-21T03:19:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:16 smithi142 ceph-mon[34512]: pgmap v403: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 962 op/s 2024-09-21T03:19:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:17 smithi083 ceph-mon[31219]: pgmap v404: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 926 op/s 2024-09-21T03:19:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:17 smithi142 ceph-mon[34512]: pgmap v404: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 926 op/s 2024-09-21T03:19:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:17 smithi193 ceph-mon[35224]: pgmap v404: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 926 op/s 2024-09-21T03:19:18.193 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:19:18.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:19:18.557 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:19.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.146+0000 7ff9f8108640 1 -- 172.21.15.83:0/3750440566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 msgr2=0x7ff9f0107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.146+0000 7ff9f8108640 1 --2- 172.21.15.83:0/3750440566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f0107620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009930 tx=0x7ff9e002a440 comp rx=0 tx=0).stop 2024-09-21T03:19:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 -- 172.21.15.83:0/3750440566 shutdown_connections 2024-09-21T03:19:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 --2- 172.21.15.83:0/3750440566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 0x7ff9f01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 --2- 172.21.15.83:0/3750440566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f0107620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 --2- 172.21.15.83:0/3750440566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 -- 172.21.15.83:0/3750440566 >> 172.21.15.83:0/3750440566 conn(0x7ff9f0079730 msgr2=0x7ff9f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 -- 172.21.15.83:0/3750440566 shutdown_connections 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.148+0000 7ff9f8108640 1 -- 172.21.15.83:0/3750440566 wait complete. 2024-09-21T03:19:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 Processor -- start 2024-09-21T03:19:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 -- start start 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 0x7ff9f01a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f011a090 con 0x7ff9f01027a0 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0119f10 con 0x7ff9f01030f0 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f8108640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f011a210 con 0x7ff9f0107be0 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f56a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f56a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46150/0 (socket says 172.21.15.83:46150) 2024-09-21T03:19:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.149+0000 7ff9f56a3640 1 -- 172.21.15.83:0/1778225112 learned_addr learned my addr 172.21.15.83:0/1778225112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 0x7ff9f01a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 -- 172.21.15.83:0/1778225112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 msgr2=0x7ff9f01a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 0x7ff9f01a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 -- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 msgr2=0x7ff9f01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 -- 172.21.15.83:0/1778225112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01a6a70 con 0x7ff9f01030f0 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0031440 tx=0x7ff9e0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9e6ffd640 1 -- 172.21.15.83:0/1778225112 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e003f070 con 0x7ff9f01030f0 2024-09-21T03:19:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9e6ffd640 1 -- 172.21.15.83:0/1778225112 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e0031b50 con 0x7ff9f01030f0 2024-09-21T03:19:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f01a6d00 con 0x7ff9f01030f0 2024-09-21T03:19:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9e6ffd640 1 -- 172.21.15.83:0/1778225112 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0035030 con 0x7ff9f01030f0 2024-09-21T03:19:19.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.150+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01ad4c0 con 0x7ff9f01030f0 2024-09-21T03:19:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.154+0000 7ff9fe67c740 1 Processor -- start 2024-09-21T03:19:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 -- start start 2024-09-21T03:19:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x5557874f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578712a670 0x5557874f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555787468e60 con 0x555787122490 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555787458290 con 0x55578714b350 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55578745c0a0 con 0x55578712a670 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9f66a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x5557874f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9f66a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46152/0 (socket says 172.21.15.83:46152) 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9f66a5640 1 -- 172.21.15.83:0/2280307112 learned_addr learned my addr 172.21.15.83:0/2280307112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.155+0000 7ff9e6ffd640 1 -- 172.21.15.83:0/1778225112 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff9e0042070 con 0x7ff9f01030f0 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 -- 172.21.15.83:0/2280307112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578712a670 msgr2=0x5557874f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578712a670 0x5557874f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 -- 172.21.15.83:0/2280307112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 msgr2=0x5557874f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x5557874f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 -- 172.21.15.83:0/2280307112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557871b6930 con 0x55578714b350 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x5557874f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009f50 tx=0x7ff9e80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7722430e78399023 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9dbfff640 1 -- 172.21.15.83:0/2280307112 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8004d20 con 0x55578714b350 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9dbfff640 1 -- 172.21.15.83:0/2280307112 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e8004050 con 0x55578714b350 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.156+0000 7ff9dbfff640 1 -- 172.21.15.83:0/2280307112 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8005280 con 0x55578714b350 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9e6ffd640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 0x7ff9dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 0x7ff9dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9e6ffd640 1 -- 172.21.15.83:0/1778225112 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7ff9e0107ce0 con 0x7ff9f01030f0 2024-09-21T03:19:19.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 -- 172.21.15.83:0/2280307112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 msgr2=0x5557874f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009f50 tx=0x7ff9e80118d0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 0x7ff9dc0b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5557874f10b0 tx=0x7ff9ec00b5e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 -- 172.21.15.83:0/2280307112 shutdown_connections 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578712a670 0x5557874f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x5557874f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.157+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/2280307112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x5557874f2430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.158+0000 7ff9fe67c740 1 -- 172.21.15.83:0/2280307112 >> 172.21.15.83:0/2280307112 conn(0x55578713f740 msgr2=0x5557874f0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.158+0000 7ff9fe67c740 1 -- 172.21.15.83:0/2280307112 shutdown_connections 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.158+0000 7ff9fe67c740 1 -- 172.21.15.83:0/2280307112 wait complete. 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.158+0000 7ff9fe67c740 1 -- start start 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.159+0000 7ff9fe67c740 1 Processor -- start 2024-09-21T03:19:19.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x55578750f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x55578750f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x55578750f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x55578750f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38020/0 (socket says 172.21.15.83:38020) 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9f5ea4640 1 -- 172.21.15.83:0/3720098569 learned_addr learned my addr 172.21.15.83:0/3720098569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x55578750f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 0x55578751ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557872b9890 con 0x555787122490 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5557873f2d40 con 0x55578712a670 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.160+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555787511ab0 con 0x55578714b350 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9fe67c740 10 client.0 _ll_register_callbacks cb 0x555787456c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:19.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 0x55578751ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 -- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 msgr2=0x55578750f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x55578750f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 -- 172.21.15.83:0/3720098569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 msgr2=0x55578750f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x55578750f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578751b430 con 0x55578714b350 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x55578750f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 0x55578751ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8012fd0 tx=0x7ff9e8004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=485cbf2ce0a49ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.161+0000 7ff9d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8027070 con 0x55578714b350 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7ff9e800f480 con 0x55578714b350 2024-09-21T03:19:19.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e8004050 con 0x55578714b350 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff9e8033050 con 0x55578714b350 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 10 client.25474 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55578751dd20 con 0x55578714b350 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e8004050 con 0x55578714b350 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff9e80194c0 con 0x55578714b350 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 10 client.25474 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 20 client.25474 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555787109cd0 0x5557875210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 4 client.25474 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.162+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578746adb0 con 0x555787109cd0 2024-09-21T03:19:19.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.163+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555787109cd0 0x5557875210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.163+0000 7ff9d97fa640 10 client.25474 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.163+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555787109cd0 0x5557875210e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece1c98d036b2ee0 server_cookie=296cfe833fbf48a8 in_seq=0 out_seq=0 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.164+0000 7ff9d97fa640 1 -- 172.21.15.83:0/3720098569 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55578746adb0 con 0x555787109cd0 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.164+0000 7ff9d97fa640 10 client.25474 handle_command_reply: tid=0 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.164+0000 7ff9fe67c740 1 client.25474 shutdown 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 msgr2=0x55578751ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 0x55578751ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8012fd0 tx=0x7ff9e8004430 comp rx=0 tx=0).stop 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 shutdown_connections 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555787109cd0 0x5557875210e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55578714b350 0x55578751ae40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x55578750f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3720098569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555787122490 0x55578750f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 >> 172.21.15.83:0/3720098569 conn(0x55578713f740 msgr2=0x5557874f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9d97fa640 0 client.25474 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:19.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 shutdown_connections 2024-09-21T03:19:19.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.165+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3720098569 wait complete. 2024-09-21T03:19:19.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.166+0000 7ff9fe67c740 20 client.25474 trim_cache size 0 max 16384 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 Processor -- start 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 -- start start 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x5557874f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x55578750e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01af8d0 con 0x55578712a670 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01ad690 con 0x55578714b350 2024-09-21T03:19:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.194+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01af200 con 0x555787122490 2024-09-21T03:19:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.196+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x5557874f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.196+0000 7ff9f56a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.196+0000 7ff9f56a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49372/0 (socket says 172.21.15.83:49372) 2024-09-21T03:19:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.196+0000 7ff9f56a3640 1 -- 172.21.15.83:0/3301304700 learned_addr learned my addr 172.21.15.83:0/3301304700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x55578750e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f56a3640 1 -- 172.21.15.83:0/3301304700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 msgr2=0x5557874f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x5557874f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f56a3640 1 -- 172.21.15.83:0/3301304700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 msgr2=0x55578750e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x55578750e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f56a3640 1 -- 172.21.15.83:0/3301304700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557874f3220 con 0x55578712a670 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.197+0000 7ff9f66a5640 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x55578750e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.198+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00cae20 tx=0x7ff9e0031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa6608ef781c562 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.198+0000 7ff9db7fe640 1 -- 172.21.15.83:0/3301304700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0049040 con 0x55578712a670 2024-09-21T03:19:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.198+0000 7ff9db7fe640 1 -- 172.21.15.83:0/3301304700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e00cd6c0 con 0x55578712a670 2024-09-21T03:19:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.198+0000 7ff9db7fe640 1 -- 172.21.15.83:0/3301304700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00c9e20 con 0x55578712a670 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3301304700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 msgr2=0x5557874f0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00cae20 tx=0x7ff9e0031f70 comp rx=0 tx=0).stop 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3301304700 shutdown_connections 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578714b350 0x55578750e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578712a670 0x5557874f0460 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/3301304700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x5557874f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3301304700 >> 172.21.15.83:0/3301304700 conn(0x55578713f740 msgr2=0x5557874ee850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3301304700 shutdown_connections 2024-09-21T03:19:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.199+0000 7ff9fe67c740 1 -- 172.21.15.83:0/3301304700 wait complete. 2024-09-21T03:19:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.200+0000 7ff9fe67c740 1 -- start start 2024-09-21T03:19:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.200+0000 7ff9fe67c740 1 Processor -- start 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.200+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.200+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x5557875139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.200+0000 7ff9fe67c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578714b350 0x55578758aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01ae3a0 con 0x55578714b350 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01adca0 con 0x55578712a670 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43206/0 (socket says 172.21.15.83:43206) 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 -- 172.21.15.83:0/30860358 learned_addr learned my addr 172.21.15.83:0/30860358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9fe67c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01aea70 con 0x555787122490 2024-09-21T03:19:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9fe67c740 10 client.0 _ll_register_callbacks cb 0x7ff9ec00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 -- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 msgr2=0x5557875139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x5557875139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 -- 172.21.15.83:0/30860358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578714b350 msgr2=0x55578758aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578714b350 0x55578758aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.201+0000 7ff9f5ea4640 1 -- 172.21.15.83:0/30860358 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578758afb0 con 0x555787122490 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.202+0000 7ff9f5ea4640 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00bde0 tx=0x7ff9ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9594efde7402e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.202+0000 7ff9d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9d97fa640 1 -- 172.21.15.83:0/30860358 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec02d070 con 0x555787122490 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e00cd6c0 con 0x555787122490 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9d97fa640 1 -- 172.21.15.83:0/30860358 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7ff9ec01f8c0 con 0x555787122490 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9d97fa640 1 -- 172.21.15.83:0/30860358 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff9ec01fac0 con 0x555787122490 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9fe67c740 10 client.25480 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.203+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55578758c190 con 0x555787122490 2024-09-21T03:19:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9d97fa640 1 -- 172.21.15.83:0/30860358 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff9ec01fc60 con 0x555787122490 2024-09-21T03:19:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 10 client.25480 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 20 client.25480 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5557870df080 0x55578758e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 4 client.25480 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.204+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff9e00cd6c0 con 0x5557870df080 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.205+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5557870df080 0x55578758e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.205+0000 7ff9f56a3640 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5557870df080 0x55578758e7f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571295ef2e1a45a5 server_cookie=509f9958f99c3806 in_seq=0 out_seq=0 2024-09-21T03:19:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.205+0000 7ff9d97fa640 10 client.25480 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.207+0000 7ff9d97fa640 1 -- 172.21.15.83:0/30860358 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff9e00cd6c0 con 0x5557870df080 2024-09-21T03:19:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.207+0000 7ff9d97fa640 10 client.25480 handle_command_reply: tid=0 2024-09-21T03:19:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.207+0000 7ff9fe67c740 1 client.25480 shutdown 2024-09-21T03:19:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.207+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 msgr2=0x555787513480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.207+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00bde0 tx=0x7ff9ec0046c0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 shutdown_connections 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5557870df080 0x55578758e7f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55578714b350 0x55578758aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9d97fa640 0 client.25480 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55578712a670 0x5557875139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 --2- 172.21.15.83:0/30860358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555787122490 0x555787513480 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 >> 172.21.15.83:0/30860358 conn(0x55578713f740 msgr2=0x5557874f0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 shutdown_connections 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 1 -- 172.21.15.83:0/30860358 wait complete. 2024-09-21T03:19:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.208+0000 7ff9fe67c740 20 client.25480 trim_cache size 0 max 16384 2024-09-21T03:19:19.211 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:19:19.211 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:19:19.211 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09", 2024-09-21T03:19:19.211 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:19:19.211 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:19:19.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.210+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 msgr2=0x7ff9dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.210+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 0x7ff9dc0b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5557874f10b0 tx=0x7ff9ec00b5e0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 msgr2=0x7ff9f01a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0031440 tx=0x7ff9e0003b90 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 shutdown_connections 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff9dc0b6700 0x7ff9dc0b8c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f0107be0 0x7ff9f01a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9f01030f0 0x7ff9f01a10a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 --2- 172.21.15.83:0/1778225112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9f01027a0 0x7ff9f01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 >> 172.21.15.83:0/1778225112 conn(0x7ff9f0079730 msgr2=0x7ff9f01055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 shutdown_connections 2024-09-21T03:19:19.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:19.211+0000 7ff9f8108640 1 -- 172.21.15.83:0/1778225112 wait complete. 2024-09-21T03:19:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:19 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:19:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:19 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:19 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:19 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:19 smithi083 ceph-mon[31219]: pgmap v405: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-09-21T03:19:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:19 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:19:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:19 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:19 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:19 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:19 smithi142 ceph-mon[34512]: pgmap v405: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-09-21T03:19:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:19 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:19:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:19 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:19 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:19 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:19 smithi193 ceph-mon[35224]: pgmap v405: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-09-21T03:19:20.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:19:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:19:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 120.0 s, 3194.8 tps, lat 31.455 ms stddev 33.957 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.055+0000 7fe82b0b8640 1 -- 172.21.15.83:0/4004958247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8241049e0 msgr2=0x7fe824104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.055+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/4004958247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8241049e0 0x7fe824104df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe818009860 tx=0x7fe81802a340 comp rx=0 tx=0).stop 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 -- 172.21.15.83:0/4004958247 shutdown_connections 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/4004958247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82410de10 0x7fe824110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/4004958247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe824105330 0x7fe82410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/4004958247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8241049e0 0x7fe824104df0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 -- 172.21.15.83:0/4004958247 >> 172.21.15.83:0/4004958247 conn(0x7fe824079730 msgr2=0x7fe824079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 -- 172.21.15.83:0/4004958247 shutdown_connections 2024-09-21T03:19:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.056+0000 7fe82b0b8640 1 -- 172.21.15.83:0/4004958247 wait complete. 2024-09-21T03:19:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.057+0000 7fe82b0b8640 1 Processor -- start 2024-09-21T03:19:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.057+0000 7fe82b0b8640 1 -- start start 2024-09-21T03:19:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 0x7fe824104130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe824105330 0x7fe824102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 0x7fe824102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe828e54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 0x7fe824104130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe828e54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 0x7fe824104130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43212/0 (socket says 172.21.15.83:43212) 2024-09-21T03:19:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe828e54640 1 -- 172.21.15.83:0/623697811 learned_addr learned my addr 172.21.15.83:0/623697811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe824113b00 con 0x7fe82410de10 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe829655640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 0x7fe824102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe824113980 con 0x7fe824105330 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.058+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe824113c80 con 0x7fe8241049e0 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 -- 172.21.15.83:0/623697811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 msgr2=0x7fe824104130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 0x7fe824104130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 -- 172.21.15.83:0/623697811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe824105330 msgr2=0x7fe824102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe824105330 0x7fe824102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe824103480 con 0x7fe82410de10 2024-09-21T03:19:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.059+0000 7fe829655640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 0x7fe824102c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009fd0 tx=0x7fe81c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.060+0000 7fe821ffb640 1 -- 172.21.15.83:0/623697811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00ce20 con 0x7fe82410de10 2024-09-21T03:19:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.060+0000 7fe821ffb640 1 -- 172.21.15.83:0/623697811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81c005020 con 0x7fe82410de10 2024-09-21T03:19:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.060+0000 7fe821ffb640 1 -- 172.21.15.83:0/623697811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c010690 con 0x7fe82410de10 2024-09-21T03:19:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.061+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8241acd10 con 0x7fe82410de10 2024-09-21T03:19:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.061+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8241ad1a0 con 0x7fe82410de10 2024-09-21T03:19:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 Processor -- start 2024-09-21T03:19:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe821ffb640 1 -- 172.21.15.83:0/623697811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe81c0054d0 con 0x7fe82410de10 2024-09-21T03:19:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 -- start start 2024-09-21T03:19:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfaae390 0x55a7dfe770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfdece50 con 0x55a7dfaae390 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfddc280 con 0x55a7dfaa61b0 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.064+0000 7fe83162c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfde0090 con 0x55a7dfacf070 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe823fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe823fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43214/0 (socket says 172.21.15.83:43214) 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe823fff640 1 -- 172.21.15.83:0/2541523250 learned_addr learned my addr 172.21.15.83:0/2541523250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe823fff640 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfaae390 0x55a7dfe770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 -- 172.21.15.83:0/2541523250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 msgr2=0x55a7dfe76650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 -- 172.21.15.83:0/2541523250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfaae390 msgr2=0x55a7dfe770d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfaae390 0x55a7dfe770d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe828e54640 1 -- 172.21.15.83:0/2541523250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dfb3a650 con 0x55a7dfaa61b0 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe821ffb640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 0x7fe8140b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe823fff640 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.065+0000 7fe821ffb640 1 -- 172.21.15.83:0/623697811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe81c0e7d90 con 0x7fe82410de10 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.066+0000 7fe823fff640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 0x7fe8140b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.067+0000 7fe828e54640 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe76b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe82419e220 tx=0x7fe818005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1402d1413a106c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.067+0000 7fe80effd640 1 -- 172.21.15.83:0/2541523250 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818042070 con 0x55a7dfaa61b0 2024-09-21T03:19:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.067+0000 7fe80effd640 1 -- 172.21.15.83:0/2541523250 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81800b0b0 con 0x55a7dfaa61b0 2024-09-21T03:19:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.067+0000 7fe80effd640 1 -- 172.21.15.83:0/2541523250 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818037040 con 0x55a7dfaa61b0 2024-09-21T03:19:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.067+0000 7fe823fff640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 0x7fe8140b8d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe810005fd0 tx=0x7fe810005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.068+0000 7fe83162c740 1 -- 172.21.15.83:0/2541523250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 msgr2=0x55a7dfe76b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.068+0000 7fe83162c740 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe76b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe82419e220 tx=0x7fe818005c90 comp rx=0 tx=0).stop 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 -- 172.21.15.83:0/2541523250 shutdown_connections 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfaae390 0x55a7dfe770d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe76b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 --2- 172.21.15.83:0/2541523250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfacf070 0x55a7dfe76650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 -- 172.21.15.83:0/2541523250 >> 172.21.15.83:0/2541523250 conn(0x55a7dfac3460 msgr2=0x55a7dfe74540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 -- 172.21.15.83:0/2541523250 shutdown_connections 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.069+0000 7fe83162c740 1 -- 172.21.15.83:0/2541523250 wait complete. 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.070+0000 7fe83162c740 1 -- start start 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.070+0000 7fe83162c740 1 Processor -- start 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaae390 0x55a7dfe82260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfacf070 0x55a7dfe8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfc3d700 con 0x55a7dfacf070 2024-09-21T03:19:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfd77110 con 0x55a7dfaae390 2024-09-21T03:19:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7dfe84dd0 con 0x55a7dfaa61b0 2024-09-21T03:19:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe83162c740 10 client.0 _ll_register_callbacks cb 0x55a7dfd90990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe828e54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe828e54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43230/0 (socket says 172.21.15.83:43230) 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe828e54640 1 -- 172.21.15.83:0/1966240357 learned_addr learned my addr 172.21.15.83:0/1966240357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe823fff640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaae390 0x55a7dfe82260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe829655640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfacf070 0x55a7dfe8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.071+0000 7fe828e54640 1 -- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaae390 msgr2=0x55a7dfe82260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe828e54640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaae390 0x55a7dfe82260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe828e54640 1 -- 172.21.15.83:0/1966240357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfacf070 msgr2=0x55a7dfe8e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe828e54640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfacf070 0x55a7dfe8e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe828e54640 1 -- 172.21.15.83:0/1966240357 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dfda4c40 con 0x55a7dfaa61b0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe828e54640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe81802a380 tx=0x7fe818005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e932946a4b5706 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe803fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81804d060 con 0x55a7dfaa61b0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe818005450 con 0x55a7dfaa61b0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe81800b0b0 con 0x55a7dfaa61b0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe818042070 con 0x55a7dfaa61b0 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe83162c740 10 client.25492 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.072+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7dfe90b20 con 0x55a7dfaa61b0 2024-09-21T03:19:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.073+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe818030450 con 0x55a7dfaa61b0 2024-09-21T03:19:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe81803d680 con 0x55a7dfaa61b0 2024-09-21T03:19:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 10 client.25492 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 20 client.25492 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfa8d9f0 0x55a7dfe93e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 4 client.25492 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe81800b0b0 con 0x55a7dfa8d9f0 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.074+0000 7fe823fff640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfa8d9f0 0x55a7dfe93e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.075+0000 7fe803fff640 10 client.25492 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:21.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.075+0000 7fe823fff640 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfa8d9f0 0x55a7dfe93e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ec649c19d2355d server_cookie=b90b0757a1f4f270 in_seq=0 out_seq=0 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.075+0000 7fe803fff640 1 -- 172.21.15.83:0/1966240357 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe81800b0b0 con 0x55a7dfa8d9f0 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.075+0000 7fe803fff640 10 client.25492 handle_command_reply: tid=0 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.075+0000 7fe83162c740 1 client.25492 shutdown 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 msgr2=0x55a7dfe81d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe81802a380 tx=0x7fe818005fd0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 shutdown_connections 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfa8d9f0 0x55a7dfe93e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfacf070 0x55a7dfe8e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaae390 0x55a7dfe82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 --2- 172.21.15.83:0/1966240357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe81d20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe803fff640 0 client.25492 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 >> 172.21.15.83:0/1966240357 conn(0x55a7dfac3460 msgr2=0x55a7dfe797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 shutdown_connections 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 1 -- 172.21.15.83:0/1966240357 wait complete. 2024-09-21T03:19:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.076+0000 7fe83162c740 20 client.25492 trim_cache size 0 max 16384 2024-09-21T03:19:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.101+0000 7fe83162c740 1 Processor -- start 2024-09-21T03:19:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 -- start start 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe77d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dfe740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b5ad0 con 0x55a7dfa8d9f0 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b7730 con 0x55a7dfaa61b0 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe83162c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b61d0 con 0x55a7dfaae390 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe823fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe823fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52502/0 (socket says 172.21.15.83:52502) 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe823fff640 1 -- 172.21.15.83:0/423153484 learned_addr learned my addr 172.21.15.83:0/423153484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe828e54640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe77d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.102+0000 7fe829655640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dfe740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 -- 172.21.15.83:0/423153484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 msgr2=0x55a7dfe740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dfe740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 -- 172.21.15.83:0/423153484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 msgr2=0x55a7dfe77d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe77d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 -- 172.21.15.83:0/423153484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dfe80220 con 0x55a7dfaa61b0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe828e54640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe77d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe823fff640 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe810010f80 tx=0x7fe8100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f0809d77571fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe80e7fc640 1 -- 172.21.15.83:0/423153484 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810013b70 con 0x55a7dfaa61b0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe80e7fc640 1 -- 172.21.15.83:0/423153484 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810013d10 con 0x55a7dfaa61b0 2024-09-21T03:19:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.103+0000 7fe80e7fc640 1 -- 172.21.15.83:0/423153484 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810012690 con 0x55a7dfaa61b0 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- 172.21.15.83:0/423153484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 msgr2=0x55a7dfe73b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe810010f80 tx=0x7fe8100046c0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- 172.21.15.83:0/423153484 shutdown_connections 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dfe740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe73b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 --2- 172.21.15.83:0/423153484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe77d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- 172.21.15.83:0/423153484 >> 172.21.15.83:0/423153484 conn(0x55a7dfac3460 msgr2=0x55a7dfe77680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- 172.21.15.83:0/423153484 shutdown_connections 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- 172.21.15.83:0/423153484 wait complete. 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.104+0000 7fe83162c740 1 -- start start 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 Processor -- start 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dff0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b6fa0 con 0x55a7dfa8d9f0 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b68d0 con 0x55a7dfaa61b0 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe83162c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8241b3ab0 con 0x55a7dfaae390 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.105+0000 7fe828e54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe828e54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe95270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49430/0 (socket says 172.21.15.83:49430) 2024-09-21T03:19:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe828e54640 1 -- 172.21.15.83:0/80093266 learned_addr learned my addr 172.21.15.83:0/80093266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe83162c740 10 client.0 _ll_register_callbacks cb 0x7fe818040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dff0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 -- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 msgr2=0x55a7dfe957b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe957b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 -- 172.21.15.83:0/80093266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 msgr2=0x55a7dfe95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 -- 172.21.15.83:0/80093266 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dff0eb50 con 0x55a7dfaae390 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe829655640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dff0e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0e5500 tx=0x7fe81c0e5a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91632cd0469c0672 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.106+0000 7fe803fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0d12c0 con 0x55a7dfaae390 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe81c0ad060 con 0x55a7dfaae390 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe810013d10 con 0x55a7dfaae390 2024-09-21T03:19:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe81c0ad710 con 0x55a7dfaae390 2024-09-21T03:19:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe83162c740 10 client.25498 fetch_fsmap learned FSMap version 32 2024-09-21T03:19:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7dff0fc40 con 0x55a7dfaae390 2024-09-21T03:19:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81c0ae300 con 0x55a7dfaae390 2024-09-21T03:19:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.107+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe81c0b3870 con 0x55a7dfaae390 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 10 client.25498 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 20 client.25498 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfacf070 0x55a7dff12420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 4 client.25498 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe810013d10 con 0x55a7dfacf070 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.109+0000 7fe828e54640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfacf070 0x55a7dff12420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe803fff640 10 client.25498 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe828e54640 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfacf070 0x55a7dff12420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62822f5b1841aa server_cookie=d1e0da203aa116ab in_seq=0 out_seq=0 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe803fff640 1 -- 172.21.15.83:0/80093266 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x7fe810013d10 con 0x55a7dfacf070 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe803fff640 10 client.25498 handle_command_reply: tid=0 2024-09-21T03:19:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe83162c740 1 client.25498 shutdown 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.110+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 msgr2=0x55a7dff0e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dff0e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0e5500 tx=0x7fe81c0e5a00 comp rx=0 tx=0).stop 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 shutdown_connections 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a7dfacf070 0x55a7dff12420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7dfaae390 0x55a7dff0e610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dfaa61b0 0x55a7dfe957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 --2- 172.21.15.83:0/80093266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7dfa8d9f0 0x55a7dfe95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe803fff640 0 client.25498 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 >> 172.21.15.83:0/80093266 conn(0x55a7dfac3460 msgr2=0x55a7dfe77680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 shutdown_connections 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 1 -- 172.21.15.83:0/80093266 wait complete. 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.111+0000 7fe83162c740 20 client.25498 trim_cache size 0 max 16384 2024-09-21T03:19:21.115 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09", 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:21.116 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:21.117 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:21.118 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:19:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 msgr2=0x7fe8140b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 0x7fe8140b8d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe810005fd0 tx=0x7fe810005850 comp rx=0 tx=0).stop 2024-09-21T03:19:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 msgr2=0x7fe824102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 0x7fe824102c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009fd0 tx=0x7fe81c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 shutdown_connections 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8140b6880 0x7fe8140b8d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe82410de10 0x7fe824102c60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe824105330 0x7fe824102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 --2- 172.21.15.83:0/623697811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8241049e0 0x7fe824104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 >> 172.21.15.83:0/623697811 conn(0x7fe824079730 msgr2=0x7fe824105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 shutdown_connections 2024-09-21T03:19:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:21.112+0000 7fe82b0b8640 1 -- 172.21.15.83:0/623697811 wait complete. 2024-09-21T03:19:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:21 smithi083 ceph-mon[31219]: pgmap v406: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T03:19:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:21 smithi142 ceph-mon[34512]: pgmap v406: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T03:19:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:21 smithi193 ceph-mon[35224]: pgmap v406: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-09-21T03:19:21.940 INFO:tasks.cephfs.filesystem:scrub status for tag:1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09 - {'path': '/', 'tag': '1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09', 'options': 'recursive,force'} 2024-09-21T03:19:24.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:24 smithi193 ceph-mon[35224]: pgmap v407: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.6 MiB/s wr, 840 op/s 2024-09-21T03:19:24.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:24 smithi083 ceph-mon[31219]: pgmap v407: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.6 MiB/s wr, 840 op/s 2024-09-21T03:19:24.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:24 smithi142 ceph-mon[34512]: pgmap v407: 97 pgs: 97 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.6 MiB/s wr, 840 op/s 2024-09-21T03:19:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 125.0 s, 3431.8 tps, lat 29.189 ms stddev 32.052 2024-09-21T03:19:26.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:26 smithi083 ceph-mon[31219]: pgmap v408: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 626 op/s 2024-09-21T03:19:26.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:26 smithi142 ceph-mon[34512]: pgmap v408: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 626 op/s 2024-09-21T03:19:26.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:26 smithi193 ceph-mon[35224]: pgmap v408: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 626 op/s 2024-09-21T03:19:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:27 smithi193 ceph-mon[35224]: pgmap v409: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-21T03:19:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:27 smithi083 ceph-mon[31219]: pgmap v409: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-21T03:19:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:27 smithi142 ceph-mon[34512]: pgmap v409: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-21T03:19:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:28 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:19:28.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:28 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:19:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:28 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:19:28.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:28 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:19:28.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:28 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:19:28.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:28 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:19:29.582 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:19:29.582 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:19:29.582 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:19:29.582 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:29 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:29.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:29 smithi083 ceph-mon[31219]: pgmap v410: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T03:19:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:19:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:19:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:19:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:29 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:29 smithi142 ceph-mon[34512]: pgmap v410: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T03:19:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:19:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:19:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:19:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:29 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:29 smithi193 ceph-mon[35224]: pgmap v410: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T03:19:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 130.0 s, 3348.2 tps, lat 29.853 ms stddev 29.112 2024-09-21T03:19:32.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:32 smithi193 ceph-mon[35224]: pgmap v411: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-21T03:19:32.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:32 smithi083 ceph-mon[31219]: pgmap v411: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-21T03:19:32.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:32 smithi142 ceph-mon[34512]: pgmap v411: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-21T03:19:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:33 smithi083 ceph-mon[31219]: pgmap v412: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 980 op/s 2024-09-21T03:19:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:33 smithi142 ceph-mon[34512]: pgmap v412: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 980 op/s 2024-09-21T03:19:33.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:33 smithi193 ceph-mon[35224]: pgmap v412: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 980 op/s 2024-09-21T03:19:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 135.0 s, 3300.2 tps, lat 30.266 ms stddev 30.545 2024-09-21T03:19:36.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:36 smithi193 ceph-mon[35224]: pgmap v413: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 773 op/s 2024-09-21T03:19:36.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:36 smithi083 ceph-mon[31219]: pgmap v413: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 773 op/s 2024-09-21T03:19:36.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:36 smithi142 ceph-mon[34512]: pgmap v413: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 773 op/s 2024-09-21T03:19:37.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:37 smithi083 ceph-mon[31219]: pgmap v414: 97 pgs: 97 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 865 op/s 2024-09-21T03:19:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:37 smithi142 ceph-mon[34512]: pgmap v414: 97 pgs: 97 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 865 op/s 2024-09-21T03:19:37.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:37 smithi193 ceph-mon[35224]: pgmap v414: 97 pgs: 97 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 865 op/s 2024-09-21T03:19:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:39 smithi083 ceph-mon[31219]: pgmap v415: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 805 op/s 2024-09-21T03:19:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:39 smithi142 ceph-mon[34512]: pgmap v415: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 805 op/s 2024-09-21T03:19:39.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:39 smithi193 ceph-mon[35224]: pgmap v415: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 805 op/s 2024-09-21T03:19:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 140.0 s, 3167.8 tps, lat 31.488 ms stddev 32.135 2024-09-21T03:19:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:41 smithi083 ceph-mon[31219]: pgmap v416: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 650 op/s 2024-09-21T03:19:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:41 smithi142 ceph-mon[34512]: pgmap v416: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 650 op/s 2024-09-21T03:19:41.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:41 smithi193 ceph-mon[35224]: pgmap v416: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 650 op/s 2024-09-21T03:19:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:43 smithi083 ceph-mon[31219]: pgmap v417: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.7 MiB/s wr, 852 op/s 2024-09-21T03:19:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:43 smithi142 ceph-mon[34512]: pgmap v417: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.7 MiB/s wr, 852 op/s 2024-09-21T03:19:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:43 smithi193 ceph-mon[35224]: pgmap v417: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.7 MiB/s wr, 852 op/s 2024-09-21T03:19:45.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:45 smithi193 ceph-mon[35224]: pgmap v418: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:45 smithi083 ceph-mon[31219]: pgmap v418: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:45 smithi142 ceph-mon[34512]: pgmap v418: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 145.0 s, 3433.0 tps, lat 29.184 ms stddev 31.497 2024-09-21T03:19:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:47 smithi083 ceph-mon[31219]: pgmap v419: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 723 op/s 2024-09-21T03:19:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:47 smithi142 ceph-mon[34512]: pgmap v419: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 723 op/s 2024-09-21T03:19:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:47 smithi193 ceph-mon[35224]: pgmap v419: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 723 op/s 2024-09-21T03:19:49.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:49 smithi083 ceph-mon[31219]: pgmap v420: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T03:19:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:49 smithi142 ceph-mon[34512]: pgmap v420: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T03:19:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:49 smithi193 ceph-mon[35224]: pgmap v420: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T03:19:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 150.0 s, 3224.0 tps, lat 30.910 ms stddev 32.632 2024-09-21T03:19:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:51 smithi142 ceph-mon[34512]: pgmap v421: 97 pgs: 97 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-09-21T03:19:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:51 smithi193 ceph-mon[35224]: pgmap v421: 97 pgs: 97 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-09-21T03:19:51.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:51 smithi083 ceph-mon[31219]: pgmap v421: 97 pgs: 97 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 653 op/s 2024-09-21T03:19:51.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:19:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.986+0000 7f6cab333640 1 -- 172.21.15.83:0/1806244180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca410b6f0 msgr2=0x7f6ca410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.986+0000 7f6cab333640 1 --2- 172.21.15.83:0/1806244180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca410b6f0 0x7f6ca410bb00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009900 tx=0x7f6c9402a410 comp rx=0 tx=0).stop 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 -- 172.21.15.83:0/1806244180 shutdown_connections 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 --2- 172.21.15.83:0/1806244180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410c040 0x7f6ca4112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 --2- 172.21.15.83:0/1806244180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca410b6f0 0x7f6ca410bb00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 --2- 172.21.15.83:0/1806244180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca4105db0 0x7f6ca41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 -- 172.21.15.83:0/1806244180 >> 172.21.15.83:0/1806244180 conn(0x7f6ca41017e0 msgr2=0x7f6ca4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 -- 172.21.15.83:0/1806244180 shutdown_connections 2024-09-21T03:19:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.988+0000 7f6cab333640 1 -- 172.21.15.83:0/1806244180 wait complete. 2024-09-21T03:19:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.989+0000 7f6cab333640 1 Processor -- start 2024-09-21T03:19:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.989+0000 7f6cab333640 1 -- start start 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4105db0 0x7f6ca41124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca410c040 0x7f6ca4110d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4077b90 con 0x7f6ca4105db0 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4077a10 con 0x7f6ca410c040 2024-09-21T03:19:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6cab333640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4077d10 con 0x7f6ca410b6f0 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6ca88ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6ca88ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59678/0 (socket says 172.21.15.83:59678) 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6ca88ce640 1 -- 172.21.15.83:0/2224256995 learned_addr learned my addr 172.21.15.83:0/2224256995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.990+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4105db0 0x7f6ca41124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 -- 172.21.15.83:0/2224256995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca410c040 msgr2=0x7f6ca4110d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca410c040 0x7f6ca4110d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca410c040 0x7f6ca4110d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 -- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4105db0 msgr2=0x7f6ca41124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4105db0 0x7f6ca41124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 -- 172.21.15.83:0/2224256995 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca4111420 con 0x7f6ca410b6f0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c94005ab0 tx=0x7f6c94003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.991+0000 7f6c9a7fc640 1 -- 172.21.15.83:0/2224256995 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9403f070 con 0x7f6ca410b6f0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.992+0000 7f6c9a7fc640 1 -- 172.21.15.83:0/2224256995 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c94031b50 con 0x7f6ca410b6f0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.992+0000 7f6c9a7fc640 1 -- 172.21.15.83:0/2224256995 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94035030 con 0x7f6ca410b6f0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.992+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca41acf50 con 0x7f6ca410b6f0 2024-09-21T03:19:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.992+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca41ad2f0 con 0x7f6ca410b6f0 2024-09-21T03:19:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.996+0000 7f6c9a7fc640 1 -- 172.21.15.83:0/2224256995 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6c94042070 con 0x7f6ca410b6f0 2024-09-21T03:19:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.996+0000 7f6cb18a7740 1 Processor -- start 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.996+0000 7f6cb18a7740 1 -- start start 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.996+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5620c6621e50 con 0x5620c62db1b0 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5620c6611280 con 0x5620c62e3390 2024-09-21T03:19:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620c6615090 con 0x5620c6304070 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59686/0 (socket says 172.21.15.83:59686) 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 -- 172.21.15.83:0/4094100703 learned_addr learned my addr 172.21.15.83:0/4094100703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 -- 172.21.15.83:0/4094100703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 msgr2=0x5620c66ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 -- 172.21.15.83:0/4094100703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 msgr2=0x5620c66abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.997+0000 7f6ca98d0640 1 -- 172.21.15.83:0/4094100703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c636f650 con 0x5620c6304070 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009eb0 tx=0x7f6c9c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b877bbd197bb4977 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6c9a7fc640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 0x7f6c900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6c8f7fe640 1 -- 172.21.15.83:0/4094100703 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011640 con 0x5620c6304070 2024-09-21T03:19:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6c8f7fe640 1 -- 172.21.15.83:0/4094100703 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c011c90 con 0x5620c6304070 2024-09-21T03:19:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6c8f7fe640 1 -- 172.21.15.83:0/4094100703 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01bc30 con 0x5620c6304070 2024-09-21T03:19:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 0x7f6c900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.998+0000 7f6c9a7fc640 1 -- 172.21.15.83:0/2224256995 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6c94107c70 con 0x7f6ca410b6f0 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.999+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 0x7f6c900b8c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5620c66aa2d0 tx=0x7f6ca0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.999+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4094100703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 msgr2=0x5620c66ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:52.999+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009eb0 tx=0x7f6c9c00f730 comp rx=0 tx=0).stop 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4094100703 shutdown_connections 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4094100703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c6304070 0x5620c66ab650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4094100703 >> 172.21.15.83:0/4094100703 conn(0x5620c62f8460 msgr2=0x5620c66a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4094100703 shutdown_connections 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4094100703 wait complete. 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 -- start start 2024-09-21T03:19:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.000+0000 7f6cb18a7740 1 Processor -- start 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.001+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.001+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.001+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c6304070 0x5620c66c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59700/0 (socket says 172.21.15.83:59700) 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5620c6472700 con 0x5620c6304070 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5620c65ac110 con 0x5620c62e3390 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620c66b9e30 con 0x5620c62db1b0 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1502254761 learned_addr learned my addr 172.21.15.83:0/1502254761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6cb18a7740 10 client.0 _ll_register_callbacks cb 0x5620c65c5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 msgr2=0x5620c66b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.002+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.003+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1502254761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c6304070 msgr2=0x5620c66c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.003+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c6304070 0x5620c66c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.003+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1502254761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c65d9c40 con 0x5620c62db1b0 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.003+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00058a0 tx=0x7f6ca0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca46136a5460d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.003+0000 7f6c8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.006+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca001f640 con 0x5620c62db1b0 2024-09-21T03:19:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.006+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c011c90 con 0x5620c62db1b0 2024-09-21T03:19:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.006+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6ca001f7e0 con 0x5620c62db1b0 2024-09-21T03:19:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.007+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ca001f9e0 con 0x5620c62db1b0 2024-09-21T03:19:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.007+0000 7f6cb18a7740 10 client.25510 fetch_fsmap learned FSMap version 33 2024-09-21T03:19:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.007+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620c66c5bf0 con 0x5620c62db1b0 2024-09-21T03:19:53.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.010+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c011c90 con 0x5620c62db1b0 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f6ca0027070 con 0x5620c62db1b0 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6cb18a7740 10 client.25510 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6cb18a7740 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6cb18a7740 20 client.25510 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c62c29f0 0x5620c66c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.012+0000 7f6cb18a7740 4 client.25510 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.013+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c6623da0 con 0x5620c62c29f0 2024-09-21T03:19:53.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.013+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c62c29f0 0x5620c66c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.014+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c62c29f0 0x5620c66c9570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eaf6e10640a0c76 server_cookie=598e6a45c38decb2 in_seq=0 out_seq=0 2024-09-21T03:19:53.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.014+0000 7f6c8cff9640 10 client.25510 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.015+0000 7f6c8cff9640 1 -- 172.21.15.83:0/1502254761 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620c6623da0 con 0x5620c62c29f0 2024-09-21T03:19:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.015+0000 7f6c8cff9640 10 client.25510 handle_command_reply: tid=0 2024-09-21T03:19:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.015+0000 7f6cb18a7740 1 client.25510 shutdown 2024-09-21T03:19:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.017+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 msgr2=0x5620c66b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.017+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00058a0 tx=0x7f6ca0004730 comp rx=0 tx=0).stop 2024-09-21T03:19:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 shutdown_connections 2024-09-21T03:19:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c62c29f0 0x5620c66c9570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c6304070 0x5620c66c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c62e3390 0x5620c66b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1502254761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62db1b0 0x5620c66b6d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.018+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 >> 172.21.15.83:0/1502254761 conn(0x5620c62f8460 msgr2=0x5620c66ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.019+0000 7f6c8cff9640 0 client.25510 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.020+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 shutdown_connections 2024-09-21T03:19:53.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.020+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1502254761 wait complete. 2024-09-21T03:19:53.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.020+0000 7f6cb18a7740 20 client.25510 trim_cache size 0 max 16384 2024-09-21T03:19:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.045+0000 7f6cb18a7740 1 Processor -- start 2024-09-21T03:19:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.045+0000 7f6cb18a7740 1 -- start start 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b5c20 con 0x5620c62db1b0 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51888/0 (socket says 172.21.15.83:51888) 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1526098755 learned_addr learned my addr 172.21.15.83:0/1526098755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b7850 con 0x5620c6304070 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.046+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b6320 con 0x5620c62e3390 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1526098755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 msgr2=0x5620c66b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1526098755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 msgr2=0x5620c66b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 -- 172.21.15.83:0/1526098755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c66c5a00 con 0x5620c62db1b0 2024-09-21T03:19:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.047+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0033dc0 tx=0x7f6ca0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5a89a6daa01650 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.048+0000 7f6c8effd640 1 -- 172.21.15.83:0/1526098755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0038030 con 0x5620c62db1b0 2024-09-21T03:19:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.048+0000 7f6c8effd640 1 -- 172.21.15.83:0/1526098755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca0029580 con 0x5620c62db1b0 2024-09-21T03:19:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.048+0000 7f6c8effd640 1 -- 172.21.15.83:0/1526098755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca001fe00 con 0x5620c62db1b0 2024-09-21T03:19:53.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.049+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 msgr2=0x5620c66c3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.049+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0033dc0 tx=0x7f6ca0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 shutdown_connections 2024-09-21T03:19:53.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/1526098755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66c3050 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 >> 172.21.15.83:0/1526098755 conn(0x5620c62f8460 msgr2=0x5620c66b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:53.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 shutdown_connections 2024-09-21T03:19:53.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 -- 172.21.15.83:0/1526098755 wait complete. 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 -- start start 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.050+0000 7f6cb18a7740 1 Processor -- start 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66bb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b70f0 con 0x5620c62db1b0 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b6a20 con 0x5620c6304070 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41b3c00 con 0x5620c62e3390 2024-09-21T03:19:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6ca90cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51904/0 (socket says 172.21.15.83:51904) 2024-09-21T03:19:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6ca90cf640 1 -- 172.21.15.83:0/4117071592 learned_addr learned my addr 172.21.15.83:0/4117071592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6cb18a7740 10 client.0 _ll_register_callbacks cb 0x7f6ca0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.051+0000 7f6ca98d0640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66bb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 -- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 msgr2=0x5620c66bb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66bb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 -- 172.21.15.83:0/4117071592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 msgr2=0x5620c66bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 -- 172.21.15.83:0/4117071592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c66bbc80 con 0x5620c62e3390 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6ca88ce640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66bae70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c94105440 tx=0x7f6c94003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1136d9bf8b48432a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6c8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94049040 con 0x5620c62e3390 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f6c940f1070 con 0x5620c62e3390 2024-09-21T03:19:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca0029580 con 0x5620c62e3390 2024-09-21T03:19:53.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.052+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6c940cd910 con 0x5620c62e3390 2024-09-21T03:19:53.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.053+0000 7f6cb18a7740 10 client.25516 fetch_fsmap learned FSMap version 33 2024-09-21T03:19:53.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.053+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620c6734020 con 0x5620c62e3390 2024-09-21T03:19:53.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.053+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca0029580 con 0x5620c62e3390 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f6c940c9db0 con 0x5620c62e3390 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 10 client.25516 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 20 client.25516 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c6297da0 0x5620c6736870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 4 client.25516 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620c661f850 con 0x5620c6297da0 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.054+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c6297da0 0x5620c6736870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6ca90cf640 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c6297da0 0x5620c6736870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a61ff930640bf8 server_cookie=d6bc2d35dd8b205d in_seq=0 out_seq=0 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6c8cff9640 10 client.25516 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6c8cff9640 1 -- 172.21.15.83:0/4117071592 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5620c661f850 con 0x5620c6297da0 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6c8cff9640 10 client.25516 handle_command_reply: tid=0 2024-09-21T03:19:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6cb18a7740 1 client.25516 shutdown 2024-09-21T03:19:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 msgr2=0x5620c66bae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.055+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66bae70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c94105440 tx=0x7f6c94003f90 comp rx=0 tx=0).stop 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 shutdown_connections 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5620c6297da0 0x5620c6736870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6c8cff9640 0 client.25516 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5620c6304070 0x5620c66bb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620c62e3390 0x5620c66bae70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 --2- 172.21.15.83:0/4117071592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620c62db1b0 0x5620c66bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 >> 172.21.15.83:0/4117071592 conn(0x5620c62f8460 msgr2=0x5620c66ac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 shutdown_connections 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 1 -- 172.21.15.83:0/4117071592 wait complete. 2024-09-21T03:19:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.058+0000 7f6cb18a7740 20 client.25516 trim_cache size 0 max 16384 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:53.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:53.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:19:53.063 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 msgr2=0x7f6c900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 0x7f6c900b8c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5620c66aa2d0 tx=0x7f6ca0005f90 comp rx=0 tx=0).stop 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 msgr2=0x7f6ca41107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c94005ab0 tx=0x7f6c94003b90 comp rx=0 tx=0).stop 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 shutdown_connections 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6c900b6700 0x7f6c900b8c10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca410c040 0x7f6ca4110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ca410b6f0 0x7f6ca41107e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 --2- 172.21.15.83:0/2224256995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4105db0 0x7f6ca41124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 >> 172.21.15.83:0/2224256995 conn(0x7f6ca41017e0 msgr2=0x7f6ca410ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 shutdown_connections 2024-09-21T03:19:53.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:53.059+0000 7f6cab333640 1 -- 172.21.15.83:0/2224256995 wait complete. 2024-09-21T03:19:53.967 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:19:53.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:19:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:54 smithi083 ceph-mon[31219]: pgmap v422: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 876 op/s 2024-09-21T03:19:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:54 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:54 smithi193 ceph-mon[35224]: pgmap v422: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 876 op/s 2024-09-21T03:19:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:54 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:54 smithi142 ceph-mon[34512]: pgmap v422: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 876 op/s 2024-09-21T03:19:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:54 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 -- 172.21.15.83:0/4135886700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 msgr2=0x7fa0fc1174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 --2- 172.21.15.83:0/4135886700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1174d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000a970 tx=0x7fa0f002b670 comp rx=0 tx=0).stop 2024-09-21T03:19:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 -- 172.21.15.83:0/4135886700 shutdown_connections 2024-09-21T03:19:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 --2- 172.21.15.83:0/4135886700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1174d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 --2- 172.21.15.83:0/4135886700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 0x7fa0fc10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 --2- 172.21.15.83:0/4135886700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc10b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.891+0000 7fa1020f2640 1 -- 172.21.15.83:0/4135886700 >> 172.21.15.83:0/4135886700 conn(0x7fa0fc0712f0 msgr2=0x7fa0fc073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.892+0000 7fa1020f2640 1 -- 172.21.15.83:0/4135886700 shutdown_connections 2024-09-21T03:19:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.892+0000 7fa1020f2640 1 -- 172.21.15.83:0/4135886700 wait complete. 2024-09-21T03:19:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.893+0000 7fa1020f2640 1 Processor -- start 2024-09-21T03:19:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.893+0000 7fa1020f2640 1 -- start start 2024-09-21T03:19:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 0x7fa0fc1a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa0faffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 0x7fa0fc1a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa0fb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc11a0a0 con 0x7fa0fc10e180 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa0fb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc1a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:37564/0 (socket says 172.21.15.83:37564) 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc119f20 con 0x7fa0fc10aee0 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa0fb7fe640 1 -- 172.21.15.83:0/914758407 learned_addr learned my addr 172.21.15.83:0/914758407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc11a220 con 0x7fa0fc10ead0 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.894+0000 7fa0fbfff640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.895+0000 7fa0fbfff640 1 -- 172.21.15.83:0/914758407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 msgr2=0x7fa0fc1a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.895+0000 7fa0fbfff640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc1a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.895+0000 7fa0fbfff640 1 -- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 msgr2=0x7fa0fc1a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.895+0000 7fa0fbfff640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 0x7fa0fc1a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.895+0000 7fa0fbfff640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1a8be0 con 0x7fa0fc10ead0 2024-09-21T03:19:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.896+0000 7fa0fbfff640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1a84b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0031040 tx=0x7fa0f00329a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.897+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0005460 con 0x7fa0fc10ead0 2024-09-21T03:19:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.897+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f0005600 con 0x7fa0fc10ead0 2024-09-21T03:19:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.897+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0004de0 con 0x7fa0fc10ead0 2024-09-21T03:19:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.897+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1a8e70 con 0x7fa0fc10ead0 2024-09-21T03:19:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.897+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1a9240 con 0x7fa0fc10ead0 2024-09-21T03:19:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.899+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e80052c0 con 0x7fa0fc10ead0 2024-09-21T03:19:54.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.900+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa0f0030510 con 0x7fa0fc10ead0 2024-09-21T03:19:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.901+0000 7fa0f8ff9640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 0x7fa0e40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.901+0000 7fa0fb7fe640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 0x7fa0e40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.901+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa0f0108570 con 0x7fa0fc10ead0 2024-09-21T03:19:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.902+0000 7fa0fb7fe640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 0x7fa0e40b8d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009fd0 tx=0x7fa0f4010040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:54.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:54.907+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa0f0040030 con 0x7fa0fc10ead0 2024-09-21T03:19:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.265+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0e8005880 con 0x7fa0fc10ead0 2024-09-21T03:19:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.265+0000 7fa0f8ff9640 1 -- 172.21.15.83:0/914758407 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11987 (secure 0 0 0) 0x7fa0f00ceae0 con 0x7fa0fc10ead0 2024-09-21T03:19:55.267 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:19:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"btime":"2024-09-21T03:19:53:293616+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:19:53.243462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:19:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 msgr2=0x7fa0e40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 0x7fa0e40b8d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009fd0 tx=0x7fa0f4010040 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 msgr2=0x7fa0fc1a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1a84b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0031040 tx=0x7fa0f00329a0 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 shutdown_connections 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa0e40b6850 0x7fa0e40b8d60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0fc10ead0 0x7fa0fc1a84b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10e180 0x7fa0fc1a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 --2- 172.21.15.83:0/914758407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc10aee0 0x7fa0fc1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 >> 172.21.15.83:0/914758407 conn(0x7fa0fc0712f0 msgr2=0x7fa0fc10d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 shutdown_connections 2024-09-21T03:19:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:55.269+0000 7fa1020f2640 1 -- 172.21.15.83:0/914758407 wait complete. 2024-09-21T03:19:55.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:55 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/914758407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:55 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/914758407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:55 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/914758407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:19:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 155.0 s, 2745.2 tps, lat 36.487 ms stddev 36.763 2024-09-21T03:19:56.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:56 smithi083 ceph-mon[31219]: pgmap v423: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:56 smithi193 ceph-mon[35224]: pgmap v423: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:56.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:19:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:56 smithi142 ceph-mon[34512]: pgmap v423: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 666 op/s 2024-09-21T03:19:56.941 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:57.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:57 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:57.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.511+0000 7fd92a1bc640 1 -- 172.21.15.83:0/2710481512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 msgr2=0x7fd91c09ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.511+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/2710481512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c09ad70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd914009860 tx=0x7fd91402a340 comp rx=0 tx=0).stop 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.512+0000 7fd92a1bc640 1 -- 172.21.15.83:0/2710481512 shutdown_connections 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.512+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/2710481512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.512+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/2710481512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c09ad70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.512+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/2710481512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 0x7fd91c095430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.512+0000 7fd92a1bc640 1 -- 172.21.15.83:0/2710481512 >> 172.21.15.83:0/2710481512 conn(0x7fd91c00cd70 msgr2=0x7fd91c00d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.513+0000 7fd92a1bc640 1 -- 172.21.15.83:0/2710481512 shutdown_connections 2024-09-21T03:19:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.514+0000 7fd92a1bc640 1 -- 172.21.15.83:0/2710481512 wait complete. 2024-09-21T03:19:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 Processor -- start 2024-09-21T03:19:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 -- start start 2024-09-21T03:19:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 0x7fd91c12a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c130040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c0a3e70 con 0x7fd91c09a960 2024-09-21T03:19:57.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c0a3cf0 con 0x7fd91c095020 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd92a1bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c0a3ff0 con 0x7fd91c09b2b0 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd9289b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd9289b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51950/0 (socket says 172.21.15.83:51950) 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd9289b9640 1 -- 172.21.15.83:0/1909550129 learned_addr learned my addr 172.21.15.83:0/1909550129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd9291ba640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 0x7fd91c12a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.515+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 -- 172.21.15.83:0/1909550129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 msgr2=0x7fd91c130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c130040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 -- 172.21.15.83:0/1909550129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 msgr2=0x7fd91c12a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 0x7fd91c12a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 -- 172.21.15.83:0/1909550129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c130770 con 0x7fd91c09a960 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:19:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd9289b9640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd914000c00 tx=0x7fd9140052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd91a7fc640 1 -- 172.21.15.83:0/1909550129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914042070 con 0x7fd91c09a960 2024-09-21T03:19:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd91a7fc640 1 -- 172.21.15.83:0/1909550129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd914038040 con 0x7fd91c09a960 2024-09-21T03:19:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c130a00 con 0x7fd91c09a960 2024-09-21T03:19:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd91a7fc640 1 -- 172.21.15.83:0/1909550129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914037070 con 0x7fd91c09a960 2024-09-21T03:19:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.516+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1371c0 con 0x7fd91c09a960 2024-09-21T03:19:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 Processor -- start 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 -- start start 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf689840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf69bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cddf601ea0 con 0x55cddf2bb1b0 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cddf5f12d0 con 0x55cddf2e4070 2024-09-21T03:19:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.520+0000 7fd931992740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cddf5f50e0 con 0x55cddf2c3390 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf689840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf689840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51952/0 (socket says 172.21.15.83:51952) 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9291ba640 1 -- 172.21.15.83:0/1941914033 learned_addr learned my addr 172.21.15.83:0/1941914033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 -- 172.21.15.83:0/1941914033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 msgr2=0x55cddf69bc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:19:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf69bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 -- 172.21.15.83:0/1941914033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 msgr2=0x55cddf689840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf689840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.521+0000 7fd9299bb640 1 -- 172.21.15.83:0/1941914033 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddf34f650 con 0x55cddf2e4070 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.522+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6892c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd924069e80 tx=0x7fd924064940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78914125ae550ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.522+0000 7fd90f7fe640 1 -- 172.21.15.83:0/1941914033 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924081640 con 0x55cddf2e4070 2024-09-21T03:19:57.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.522+0000 7fd90f7fe640 1 -- 172.21.15.83:0/1941914033 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd924081c90 con 0x55cddf2e4070 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.522+0000 7fd91a7fc640 1 -- 172.21.15.83:0/1909550129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd914045070 con 0x7fd91c09a960 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.522+0000 7fd90f7fe640 1 -- 172.21.15.83:0/1941914033 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924080bc0 con 0x55cddf2e4070 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- 172.21.15.83:0/1941914033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 msgr2=0x55cddf6892c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6892c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd924069e80 tx=0x7fd924064940 comp rx=0 tx=0).stop 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- 172.21.15.83:0/1941914033 shutdown_connections 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf69bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf689840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 --2- 172.21.15.83:0/1941914033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6892c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- 172.21.15.83:0/1941914033 >> 172.21.15.83:0/1941914033 conn(0x55cddf2d8460 msgr2=0x55cddf68b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- 172.21.15.83:0/1941914033 shutdown_connections 2024-09-21T03:19:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- 172.21.15.83:0/1941914033 wait complete. 2024-09-21T03:19:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.523+0000 7fd931992740 1 -- start start 2024-09-21T03:19:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.524+0000 7fd931992740 1 Processor -- start 2024-09-21T03:19:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf69fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf6a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cddf452700 con 0x55cddf2bb1b0 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cddf58c160 con 0x55cddf2e4070 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd931992740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cddf69c710 con 0x55cddf2c3390 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd91a7fc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 0x7fd9100b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.525+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf6a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd9289b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf6a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59758/0 (socket says 172.21.15.83:59758) 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd9291ba640 1 -- 172.21.15.83:0/147769958 learned_addr learned my addr 172.21.15.83:0/147769958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd91a7fc640 1 -- 172.21.15.83:0/1909550129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd914106bd0 con 0x7fd91c09a960 2024-09-21T03:19:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd931992740 10 client.0 _ll_register_callbacks cb 0x55cddf5a59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.526+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 0x7fd9100b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 -- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 msgr2=0x55cddf6a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf6a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 -- 172.21.15.83:0/147769958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 msgr2=0x55cddf69fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf69fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 -- 172.21.15.83:0/147769958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddf5b9c90 con 0x55cddf2e4070 2024-09-21T03:19:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd9289b9640 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6a55f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd914104340 tx=0x7fd91402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f0d97a58eaff97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:57.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140cc1b0 con 0x55cddf2e4070 2024-09-21T03:19:57.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.527+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd9141045c0 con 0x55cddf2e4070 2024-09-21T03:19:57.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd924081c90 con 0x55cddf2e4070 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd9140cc550 con 0x55cddf2e4070 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 10 client.25559 fetch_fsmap learned FSMap version 34 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cddf6ae520 con 0x55cddf2e4070 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd9299bb640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 0x7fd9100b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9240514a0 tx=0x7fd924068e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140b5030 con 0x55cddf2e4070 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fd9140a80e0 con 0x55cddf2e4070 2024-09-21T03:19:57.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 10 client.25559 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 10 client.25559 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.528+0000 7fd931992740 20 client.25559 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf2a29f0 0x55cddf6b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd931992740 4 client.25559 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cddf603df0 con 0x55cddf2a29f0 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd9291ba640 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf2a29f0 0x55cddf6b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd90cff9640 10 client.25559 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.529+0000 7fd9291ba640 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf2a29f0 0x55cddf6b1950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e167a89b3346a68 server_cookie=cb25dc1b98971351 in_seq=0 out_seq=0 2024-09-21T03:19:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.530+0000 7fd90cff9640 1 -- 172.21.15.83:0/147769958 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cddf603df0 con 0x55cddf2a29f0 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.530+0000 7fd90cff9640 10 client.25559 handle_command_reply: tid=0 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.530+0000 7fd931992740 1 client.25559 shutdown 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 msgr2=0x55cddf6a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6a55f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd914104340 tx=0x7fd91402a340 comp rx=0 tx=0).stop 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 shutdown_connections 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf2a29f0 0x55cddf6b1950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2e4070 0x55cddf6a55f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf6a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 --2- 172.21.15.83:0/147769958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf69fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 >> 172.21.15.83:0/147769958 conn(0x55cddf2d8460 msgr2=0x55cddf68a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd90cff9640 0 client.25559 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 shutdown_connections 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 1 -- 172.21.15.83:0/147769958 wait complete. 2024-09-21T03:19:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.531+0000 7fd931992740 20 client.25559 trim_cache size 0 max 16384 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 Processor -- start 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 -- start start 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 0x55cddf69d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf68b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 0x55cddf69a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c13f850 con 0x55cddf2e4070 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c141450 con 0x55cddf2bb1b0 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd931992740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c13ff50 con 0x55cddf2c3390 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 0x55cddf69d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.554+0000 7fd9299bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 0x55cddf69a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 0x55cddf69d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:37610/0 (socket says 172.21.15.83:37610) 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9291ba640 1 -- 172.21.15.83:0/2462867437 learned_addr learned my addr 172.21.15.83:0/2462867437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf68b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 -- 172.21.15.83:0/2462867437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 msgr2=0x55cddf69d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 0x55cddf69d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 -- 172.21.15.83:0/2462867437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 msgr2=0x55cddf69a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 0x55cddf69a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 -- 172.21.15.83:0/2462867437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddf68c0d0 con 0x55cddf2c3390 2024-09-21T03:19:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9299bb640 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 0x55cddf69a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:57.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd9289b9640 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf68b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9140c9cb0 tx=0x7fd9140cd140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f09abf58be7800 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd90effd640 1 -- 172.21.15.83:0/2462867437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140cc1b0 con 0x55cddf2c3390 2024-09-21T03:19:57.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd90effd640 1 -- 172.21.15.83:0/2462867437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140b5070 con 0x55cddf2c3390 2024-09-21T03:19:57.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.555+0000 7fd90effd640 1 -- 172.21.15.83:0/2462867437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140d64b0 con 0x55cddf2c3390 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.556+0000 7fd931992740 1 -- 172.21.15.83:0/2462867437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 msgr2=0x55cddf68b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf68b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9140c9cb0 tx=0x7fd9140cd140 comp rx=0 tx=0).stop 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 -- 172.21.15.83:0/2462867437 shutdown_connections 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2e4070 0x55cddf69a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2c3390 0x55cddf68b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 --2- 172.21.15.83:0/2462867437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2bb1b0 0x55cddf69d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 -- 172.21.15.83:0/2462867437 >> 172.21.15.83:0/2462867437 conn(0x55cddf2d8460 msgr2=0x55cddf689260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 -- 172.21.15.83:0/2462867437 shutdown_connections 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.557+0000 7fd931992740 1 -- 172.21.15.83:0/2462867437 wait complete. 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.558+0000 7fd931992740 1 -- start start 2024-09-21T03:19:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.558+0000 7fd931992740 1 Processor -- start 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd931992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd931992740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2c3390 0x55cddf6b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd9291ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51978/0 (socket says 172.21.15.83:51978) 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd9291ba640 1 -- 172.21.15.83:0/3654990276 learned_addr learned my addr 172.21.15.83:0/3654990276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 0x55cddf6b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.559+0000 7fd9289b9640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2c3390 0x55cddf6b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.560+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c140d20 con 0x55cddf2bb1b0 2024-09-21T03:19:57.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.560+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c140650 con 0x55cddf2c3390 2024-09-21T03:19:57.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.560+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c13d830 con 0x55cddf2e4070 2024-09-21T03:19:57.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.560+0000 7fd9299bb640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 0x55cddf6b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.560+0000 7fd931992740 10 client.0 _ll_register_callbacks cb 0x7fd91402af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9299bb640 1 -- 172.21.15.83:0/3654990276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2c3390 msgr2=0x55cddf6b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9299bb640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2c3390 0x55cddf6b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9299bb640 1 -- 172.21.15.83:0/3654990276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 msgr2=0x55cddf6b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9299bb640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9299bb640 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddf6b7e40 con 0x55cddf2e4070 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.561+0000 7fd9291ba640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd9299bb640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 0x55cddf6b6670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55cddf69c360 tx=0x7fd924069570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d269fafb0e1986e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240809f0 con 0x55cddf2e4070 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9140b5070 con 0x55cddf2e4070 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd924080b90 con 0x55cddf2e4070 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd9240a9050 con 0x55cddf2e4070 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd931992740 10 client.25537 fetch_fsmap learned FSMap version 34 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.562+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cddf71cdb0 con 0x55cddf2e4070 2024-09-21T03:19:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140b5070 con 0x55cddf2e4070 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fd92408d3f0 con 0x55cddf2e4070 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 10 client.25537 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 10 client.25537 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 20 client.25537 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf277da0 0x55cddf71f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 4 client.25537 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cddf5ff8a0 con 0x55cddf277da0 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.563+0000 7fd9291ba640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf277da0 0x55cddf71f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd90cff9640 10 client.25537 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd9291ba640 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf277da0 0x55cddf71f280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad4943d814e7e360 server_cookie=413cdaea7f9e90cd in_seq=0 out_seq=0 2024-09-21T03:19:57.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd90cff9640 1 -- 172.21.15.83:0/3654990276 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cddf5ff8a0 con 0x55cddf277da0 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd90cff9640 10 client.25537 handle_command_reply: tid=0 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 client.25537 shutdown 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 msgr2=0x55cddf6b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 0x55cddf6b6670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55cddf69c360 tx=0x7fd924069570 comp rx=0 tx=0).stop 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 shutdown_connections 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55cddf277da0 0x55cddf71f280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cddf2e4070 0x55cddf6b6670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cddf2c3390 0x55cddf6b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 --2- 172.21.15.83:0/3654990276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddf2bb1b0 0x55cddf6b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd90cff9640 0 client.25537 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.564+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 >> 172.21.15.83:0/3654990276 conn(0x55cddf2d8460 msgr2=0x55cddf69a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 shutdown_connections 2024-09-21T03:19:57.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd931992740 1 -- 172.21.15.83:0/3654990276 wait complete. 2024-09-21T03:19:57.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd931992740 20 client.25537 trim_cache size 0 max 16384 2024-09-21T03:19:57.569 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 msgr2=0x7fd9100b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 0x7fd9100b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9240514a0 tx=0x7fd924068e50 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 msgr2=0x7fd91c12ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd914000c00 tx=0x7fd9140052f0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.565+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 shutdown_connections 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9100b65a0 0x7fd9100b8ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c09b2b0 0x7fd91c130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c09a960 0x7fd91c12ada0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 --2- 172.21.15.83:0/1909550129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd91c095020 0x7fd91c12a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 >> 172.21.15.83:0/1909550129 conn(0x7fd91c00cd70 msgr2=0x7fd91c0989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 shutdown_connections 2024-09-21T03:19:57.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:57.566+0000 7fd92a1bc640 1 -- 172.21.15.83:0/1909550129 wait complete. 2024-09-21T03:19:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:57 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:57.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:57 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:58 smithi083 ceph-mon[31219]: pgmap v424: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-09-21T03:19:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:19:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:19:58.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:19:58.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:19:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:58 smithi142 ceph-mon[34512]: pgmap v424: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-09-21T03:19:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:19:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:19:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:19:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:58 smithi193 ceph-mon[35224]: pgmap v424: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 651 op/s 2024-09-21T03:19:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:19:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:19:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:19:58.918 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:59 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:19:59 smithi083 ceph-mon[31219]: pgmap v425: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-09-21T03:19:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3192394430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 msgr2=0x7f9da810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3192394430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da810f520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009cc0 tx=0x7f9da002cad0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3192394430 shutdown_connections 2024-09-21T03:19:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3192394430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da810f520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3192394430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 0x7f9da81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3192394430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3192394430 >> 172.21.15.83:0/3192394430 conn(0x7f9da8079730 msgr2=0x7f9da8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3192394430 shutdown_connections 2024-09-21T03:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.791+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3192394430 wait complete. 2024-09-21T03:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.792+0000 7f9dae4c7640 1 Processor -- start 2024-09-21T03:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.792+0000 7f9dae4c7640 1 -- start start 2024-09-21T03:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 0x7f9da806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da806e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8111860 con 0x7f9da8102a60 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81116e0 con 0x7f9da81083a0 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9dae4c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81119e0 con 0x7f9da8108cf0 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56678/0 (socket says 172.21.15.83:56678) 2024-09-21T03:19:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9da7fff640 1 -- 172.21.15.83:0/3407978129 learned_addr learned my addr 172.21.15.83:0/3407978129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9daca64640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.793+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 0x7f9da806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 -- 172.21.15.83:0/3407978129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 msgr2=0x7f9da806e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 0x7f9da806e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 -- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 msgr2=0x7f9da8074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 -- 172.21.15.83:0/3407978129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da806ee00 con 0x7f9da8108cf0 2024-09-21T03:19:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9daca64640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da806e600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9da002d0c0 tx=0x7f9da0038b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9da57fa640 1 -- 172.21.15.83:0/3407978129 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0046070 con 0x7f9da8108cf0 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da8073a60 con 0x7f9da8108cf0 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9da57fa640 1 -- 172.21.15.83:0/3407978129 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da0038cc0 con 0x7f9da8108cf0 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9da57fa640 1 -- 172.21.15.83:0/3407978129 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da002ed80 con 0x7f9da8108cf0 2024-09-21T03:19:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.794+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da8073f90 con 0x7f9da8108cf0 2024-09-21T03:19:59.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.798+0000 7f9da57fa640 1 -- 172.21.15.83:0/3407978129 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9da0004250 con 0x7f9da8108cf0 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.798+0000 7f9db4a3b740 1 Processor -- start 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 -- start start 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb93a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb94ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646cb8b2ea0 con 0x5646cb56c1b0 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646cb8a22d0 con 0x5646cb595070 2024-09-21T03:19:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5646cb8a60e0 con 0x5646cb574390 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49370/0 (socket says 172.21.15.83:49370) 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9da77fe640 1 -- 172.21.15.83:0/3545640539 learned_addr learned my addr 172.21.15.83:0/3545640539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb93a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.799+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb94ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 -- 172.21.15.83:0/3545640539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 msgr2=0x5646cb94ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb94ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 -- 172.21.15.83:0/3545640539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 msgr2=0x5646cb93a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb93a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 -- 172.21.15.83:0/3545640539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cb600650 con 0x5646cb595070 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da57fa640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 0x7f9d900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c014570 tx=0x7f9d9c00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3b9089ae302e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9da57fa640 1 -- 172.21.15.83:0/3407978129 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9da0109730 con 0x7f9da8108cf0 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9d8e7fc640 1 -- 172.21.15.83:0/3545640539 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01a070 con 0x5646cb595070 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9d8e7fc640 1 -- 172.21.15.83:0/3545640539 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9c005020 con 0x5646cb595070 2024-09-21T03:19:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.800+0000 7f9d8e7fc640 1 -- 172.21.15.83:0/3545640539 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004a00 con 0x5646cb595070 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.802+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 0x7f9d900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.802+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 0x7f9d900b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5646cb93b2a0 tx=0x7f9d98005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.802+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3545640539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 msgr2=0x5646cb93a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.802+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c014570 tx=0x7f9d9c00b900 comp rx=0 tx=0).stop 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.802+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3545640539 shutdown_connections 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb94ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb93a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3545640539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb93a100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3545640539 >> 172.21.15.83:0/3545640539 conn(0x5646cb589460 msgr2=0x5646cb93c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3545640539 shutdown_connections 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.803+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3545640539 wait complete. 2024-09-21T03:19:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.804+0000 7f9db4a3b740 1 -- start start 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.804+0000 7f9db4a3b740 1 Processor -- start 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb950dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb9513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646cb703700 con 0x5646cb56c1b0 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646cb83d160 con 0x5646cb595070 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5646cb94d570 con 0x5646cb574390 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9db4a3b740 10 client.0 _ll_register_callbacks cb 0x5646cb8569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb950dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9daca64640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9daca64640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49374/0 (socket says 172.21.15.83:49374) 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9daca64640 1 -- 172.21.15.83:0/3943735584 learned_addr learned my addr 172.21.15.83:0/3943735584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.805+0000 7f9da77fe640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb9513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 -- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 msgr2=0x5646cb9513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb9513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 -- 172.21.15.83:0/3943735584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 msgr2=0x5646cb950dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb950dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 -- 172.21.15.83:0/3943735584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cb86ac90 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9daca64640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9da0032040 tx=0x7f9da00cf480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27611ce014053cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9d877fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da00cf1c0 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.806+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9da00ce910 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9c005020 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9da00d40e0 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 10 client.25577 fetch_fsmap learned FSMap version 35 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646cb95f320 con 0x5646cb595070 2024-09-21T03:19:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da00d9020 con 0x5646cb595070 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f9da002e040 con 0x5646cb595070 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 10 client.25577 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 10 client.25577 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 20 client.25577 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb5539f0 0x5646cb962760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 4 client.25577 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.807+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9d9c005020 con 0x5646cb5539f0 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.808+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb5539f0 0x5646cb962760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.808+0000 7f9d877fe640 10 client.25577 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.808+0000 7f9da7fff640 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb5539f0 0x5646cb962760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0b423c622635b44 server_cookie=326223904891f040 in_seq=0 out_seq=0 2024-09-21T03:19:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.809+0000 7f9d877fe640 1 -- 172.21.15.83:0/3943735584 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9d9c005020 con 0x5646cb5539f0 2024-09-21T03:19:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.809+0000 7f9d877fe640 10 client.25577 handle_command_reply: tid=0 2024-09-21T03:19:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.809+0000 7f9db4a3b740 1 client.25577 shutdown 2024-09-21T03:19:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 msgr2=0x5646cb9563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9da0032040 tx=0x7f9da00cf480 comp rx=0 tx=0).stop 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 shutdown_connections 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb595070 0x5646cb9563d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb5539f0 0x5646cb962760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9d877fe640 0 client.25577 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb9513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/3943735584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb56c1b0 0x5646cb950dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.810+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 >> 172.21.15.83:0/3943735584 conn(0x5646cb589460 msgr2=0x5646cb939570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.811+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 shutdown_connections 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.811+0000 7f9db4a3b740 1 -- 172.21.15.83:0/3943735584 wait complete. 2024-09-21T03:19:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.811+0000 7f9db4a3b740 20 client.25577 trim_cache size 0 max 16384 2024-09-21T03:19:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:59 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:19:59 smithi193 ceph-mon[35224]: pgmap v425: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.830+0000 7f9db4a3b740 1 Processor -- start 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.830+0000 7f9db4a3b740 1 -- start start 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 0x5646cb93c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb94b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 0x5646cb94bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b5ab0 con 0x5646cb574390 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b7680 con 0x5646cb56c1b0 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b61b0 con 0x5646cb5539f0 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb94b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9da77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb94b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49390/0 (socket says 172.21.15.83:49390) 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9da77fe640 1 -- 172.21.15.83:0/1192916333 learned_addr learned my addr 172.21.15.83:0/1192916333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 0x5646cb93c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.831+0000 7f9daca64640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 0x5646cb94bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 -- 172.21.15.83:0/1192916333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 msgr2=0x5646cb94b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb94b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 -- 172.21.15.83:0/1192916333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 msgr2=0x5646cb94bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 0x5646cb94bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 -- 172.21.15.83:0/1192916333 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cb934840 con 0x5646cb5539f0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9daca64640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 0x5646cb94bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 0x5646cb93c450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5646cb962e90 tx=0x7f9d98017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905b513874f7c373 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.832+0000 7f9d8dffb640 1 -- 172.21.15.83:0/1192916333 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800fb70 con 0x5646cb5539f0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.833+0000 7f9d8dffb640 1 -- 172.21.15.83:0/1192916333 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9800fd10 con 0x5646cb5539f0 2024-09-21T03:19:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.833+0000 7f9d8dffb640 1 -- 172.21.15.83:0/1192916333 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801f520 con 0x5646cb5539f0 2024-09-21T03:19:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1192916333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 msgr2=0x5646cb93c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 0x5646cb93c450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5646cb962e90 tx=0x7f9d98017000 comp rx=0 tx=0).stop 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1192916333 shutdown_connections 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb574390 0x5646cb94bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb94b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1192916333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb5539f0 0x5646cb93c450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1192916333 >> 172.21.15.83:0/1192916333 conn(0x5646cb589460 msgr2=0x5646cb94c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1192916333 shutdown_connections 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1192916333 wait complete. 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.834+0000 7f9db4a3b740 1 -- start start 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 Processor -- start 2024-09-21T03:19:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb953ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb954530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b6f80 con 0x5646cb5539f0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b68b0 con 0x5646cb56c1b0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b3a90 con 0x5646cb574390 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9db4a3b740 10 client.0 _ll_register_callbacks cb 0x7f9da0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56712/0 (socket says 172.21.15.83:56712) 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.835+0000 7f9da7fff640 1 -- 172.21.15.83:0/1748272274 learned_addr learned my addr 172.21.15.83:0/1748272274 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9daca64640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb954530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da77fe640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb953ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da7fff640 1 -- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 msgr2=0x5646cb954530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb954530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da7fff640 1 -- 172.21.15.83:0/1748272274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 msgr2=0x5646cb953ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:19:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb953ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da7fff640 1 -- 172.21.15.83:0/1748272274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cb95d5b0 con 0x5646cb5539f0 2024-09-21T03:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.836+0000 7f9da77fe640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb953ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.837+0000 7f9da7fff640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005230 tx=0x7f9d98005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18843a56c03af93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.837+0000 7f9d877fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:19:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.837+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9802e040 con 0x5646cb5539f0 2024-09-21T03:19:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.837+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9d980046c0 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9800fd10 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d98005600 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 10 client.16053 fetch_fsmap learned FSMap version 35 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646cb9cdaf0 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9800fd10 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f9d9803b470 con 0x5646cb5539f0 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 10 client.16053 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 10 client.16053 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.838+0000 7f9db4a3b740 20 client.16053 populate_metadata read hostname 'smithi083' 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.839+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb595070 0x5646cb9d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.839+0000 7f9db4a3b740 4 client.16053 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.839+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646cb8b4df0 con 0x5646cb595070 2024-09-21T03:19:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.839+0000 7f9da77fe640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb595070 0x5646cb9d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.839+0000 7f9d877fe640 10 client.16053 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.840+0000 7f9da77fe640 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb595070 0x5646cb9d01b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fabcdc4f1f6a244 server_cookie=82a4f5a82d689b0 in_seq=0 out_seq=0 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.840+0000 7f9d877fe640 1 -- 172.21.15.83:0/1748272274 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646cb8b4df0 con 0x5646cb595070 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.840+0000 7f9d877fe640 10 client.16053 handle_command_reply: tid=0 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 client.16053 shutdown 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 msgr2=0x5646cb955b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005230 tx=0x7f9d98005260 comp rx=0 tx=0).stop 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 shutdown_connections 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5646cb595070 0x5646cb9d01b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646cb574390 0x5646cb954530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cb56c1b0 0x5646cb953ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 --2- 172.21.15.83:0/1748272274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646cb5539f0 0x5646cb955b10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9d877fe640 0 client.16053 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:19:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.841+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 >> 172.21.15.83:0/1748272274 conn(0x5646cb589460 msgr2=0x5646cb94aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 shutdown_connections 2024-09-21T03:19:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9db4a3b740 1 -- 172.21.15.83:0/1748272274 wait complete. 2024-09-21T03:19:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9db4a3b740 20 client.16053 trim_cache size 0 max 16384 2024-09-21T03:19:59.846 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 msgr2=0x7f9d900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 0x7f9d900b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5646cb93b2a0 tx=0x7f9d98005f90 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.842+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 msgr2=0x7f9da806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da806e600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9da002d0c0 tx=0x7f9da0038b50 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 shutdown_connections 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9d900b6680 0x7f9d900b8b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da8108cf0 0x7f9da806e600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9da81083a0 0x7f9da806e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 --2- 172.21.15.83:0/3407978129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da8102a60 0x7f9da8074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 >> 172.21.15.83:0/3407978129 conn(0x7f9da8079730 msgr2=0x7f9da81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 shutdown_connections 2024-09-21T03:19:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:19:59.843+0000 7f9dae4c7640 1 -- 172.21.15.83:0/3407978129 wait complete. 2024-09-21T03:19:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:59 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:19:59.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:19:59 smithi142 ceph-mon[34512]: pgmap v425: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-09-21T03:20:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:00 smithi193 ceph-mon[35224]: overall HEALTH_OK 2024-09-21T03:20:00.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:00 smithi083 ceph-mon[31219]: overall HEALTH_OK 2024-09-21T03:20:00.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:00 smithi142 ceph-mon[34512]: overall HEALTH_OK 2024-09-21T03:20:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 160.0 s, 2871.0 tps, lat 34.870 ms stddev 37.728 2024-09-21T03:20:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:01 smithi193 ceph-mon[35224]: pgmap v426: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-09-21T03:20:01.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:01 smithi083 ceph-mon[31219]: pgmap v426: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-09-21T03:20:01.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:01 smithi142 ceph-mon[34512]: pgmap v426: 97 pgs: 97 active+clean; 2.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-09-21T03:20:02.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:20:02.669 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:03 smithi083 ceph-mon[31219]: pgmap v427: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-21T03:20:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:03 smithi142 ceph-mon[34512]: pgmap v427: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-21T03:20:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:03 smithi193 ceph-mon[35224]: pgmap v427: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.152+0000 7fa2717a2640 1 -- 172.21.15.83:0/520136748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10a640 msgr2=0x7fa26c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.152+0000 7fa2717a2640 1 --2- 172.21.15.83:0/520136748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10a640 0x7fa26c10aa50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009900 tx=0x7fa25c02a410 comp rx=0 tx=0).stop 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 -- 172.21.15.83:0/520136748 shutdown_connections 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 --2- 172.21.15.83:0/520136748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c116510 0x7fa26c118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 --2- 172.21.15.83:0/520136748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c10af90 0x7fa26c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 --2- 172.21.15.83:0/520136748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10a640 0x7fa26c10aa50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 -- 172.21.15.83:0/520136748 >> 172.21.15.83:0/520136748 conn(0x7fa26c06e9f0 msgr2=0x7fa26c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 -- 172.21.15.83:0/520136748 shutdown_connections 2024-09-21T03:20:04.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.153+0000 7fa2717a2640 1 -- 172.21.15.83:0/520136748 wait complete. 2024-09-21T03:20:04.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 Processor -- start 2024-09-21T03:20:04.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 -- start start 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c10a640 0x7fa26c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10af90 0x7fa26c1a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c10a640 0x7fa26c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa270fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa270fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49412/0 (socket says 172.21.15.83:49412) 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa270fa1640 1 -- 172.21.15.83:0/3109706700 learned_addr learned my addr 172.21.15.83:0/3109706700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c11c120 con 0x7fa26c10af90 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c11bfa0 con 0x7fa26c116510 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.154+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c11c2a0 con 0x7fa26c10a640 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 -- 172.21.15.83:0/3109706700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c10a640 msgr2=0x7fa26c1a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c10a640 0x7fa26c1a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 -- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10af90 msgr2=0x7fa26c1a3110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:20:04.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10af90 0x7fa26c1a3110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c1a8ae0 con 0x7fa26c116510 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.155+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa26400eb10 tx=0x7fa26400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.156+0000 7fa2697fa640 1 -- 172.21.15.83:0/3109706700 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264004b20 con 0x7fa26c116510 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.156+0000 7fa2697fa640 1 -- 172.21.15.83:0/3109706700 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa264004cc0 con 0x7fa26c116510 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.156+0000 7fa2697fa640 1 -- 172.21.15.83:0/3109706700 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264005610 con 0x7fa26c116510 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.156+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c1a8dd0 con 0x7fa26c116510 2024-09-21T03:20:04.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.156+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c10bd60 con 0x7fa26c116510 2024-09-21T03:20:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.159+0000 7fa2697fa640 1 -- 172.21.15.83:0/3109706700 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa2640057b0 con 0x7fa26c116510 2024-09-21T03:20:04.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.160+0000 7fa2697fa640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 0x7fa2540b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.160+0000 7fa278f78740 1 Processor -- start 2024-09-21T03:20:04.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.160+0000 7fa2697fa640 1 -- 172.21.15.83:0/3109706700 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2640eb980 con 0x7fa26c116510 2024-09-21T03:20:04.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.160+0000 7fa26bfff640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 0x7fa2540b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.160+0000 7fa278f78740 1 -- start start 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f44954460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f448caea0 con 0x560f445841b0 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f448ba2d0 con 0x560f4458c390 2024-09-21T03:20:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa278f78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f448be0e0 con 0x560f445ad070 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa26b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49414/0 (socket says 172.21.15.83:49414) 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa26b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56732/0 (socket says 172.21.15.83:56732) 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 -- 172.21.15.83:0/2434178234 learned_addr learned my addr 172.21.15.83:0/2434178234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa26bfff640 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f44954460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 -- 172.21.15.83:0/2434178234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 msgr2=0x560f44954460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f44954460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 -- 172.21.15.83:0/2434178234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 msgr2=0x560f449549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa270fa1640 1 -- 172.21.15.83:0/2434178234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f44618650 con 0x560f4458c390 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.161+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.162+0000 7fa26bfff640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 0x7fa2540b8d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa25c005ab0 tx=0x7fa25c000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.162+0000 7fa270fa1640 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa2640addc0 tx=0x7fa2640b15c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83ecb67c43f7c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.162+0000 7fa2527fc640 1 -- 172.21.15.83:0/2434178234 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640e9570 con 0x560f4458c390 2024-09-21T03:20:04.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.162+0000 7fa2527fc640 1 -- 172.21.15.83:0/2434178234 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa264005000 con 0x560f4458c390 2024-09-21T03:20:04.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.162+0000 7fa2527fc640 1 -- 172.21.15.83:0/2434178234 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640ad3a0 con 0x560f4458c390 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 -- 172.21.15.83:0/2434178234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 msgr2=0x560f44954ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa2640addc0 tx=0x7fa2640b15c0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 -- 172.21.15.83:0/2434178234 shutdown_connections 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44954ee0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f449549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 --2- 172.21.15.83:0/2434178234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f44954460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 -- 172.21.15.83:0/2434178234 >> 172.21.15.83:0/2434178234 conn(0x560f445a1460 msgr2=0x560f44952350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 -- 172.21.15.83:0/2434178234 shutdown_connections 2024-09-21T03:20:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.165+0000 7fa278f78740 1 -- 172.21.15.83:0/2434178234 wait complete. 2024-09-21T03:20:04.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.166+0000 7fa278f78740 1 -- start start 2024-09-21T03:20:04.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 Processor -- start 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f4495fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44960070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445ad070 0x560f4496bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f4495fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f4471b700 con 0x560f445ad070 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f4495fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53270/0 (socket says 172.21.15.83:53270) 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa26bfff640 1 -- 172.21.15.83:0/1229943353 learned_addr learned my addr 172.21.15.83:0/1229943353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:04.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f44855160 con 0x560f4458c390 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.167+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f44962c40 con 0x560f445841b0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44960070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa278f78740 10 client.0 _ll_register_callbacks cb 0x560f4486e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa270fa1640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445ad070 0x560f4496bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 -- 172.21.15.83:0/1229943353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 msgr2=0x560f4495fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f4495fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 -- 172.21.15.83:0/1229943353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445ad070 msgr2=0x560f4496bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445ad070 0x560f4496bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 -- 172.21.15.83:0/1229943353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f44882c90 con 0x560f4458c390 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44960070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa26c118290 tx=0x7fa2600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8449cf700280e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa24b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.168+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260005020 con 0x560f4458c390 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.169+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2600051c0 con 0x560f4458c390 2024-09-21T03:20:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.169+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f448ccdf0 con 0x560f4458c390 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.169+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa2600053e0 con 0x560f4458c390 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.169+0000 7fa278f78740 10 client.25592 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.169+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f4496ea00 con 0x560f4458c390 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560f448ccdf0 con 0x560f4458c390 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa260018070 con 0x560f4458c390 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 10 client.25592 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 10 client.25592 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 20 client.25592 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f4456b9f0 0x560f44972320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 4 client.25592 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:04.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.170+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f448c88a0 con 0x560f4456b9f0 2024-09-21T03:20:04.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.171+0000 7fa26bfff640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f4456b9f0 0x560f44972320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.172+0000 7fa26bfff640 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f4456b9f0 0x560f44972320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bf6d55dc48ae0a3 server_cookie=d5d62bde685ef5a6 in_seq=0 out_seq=0 2024-09-21T03:20:04.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.172+0000 7fa24b7fe640 10 client.25592 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:04.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.174+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1229943353 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f448c88a0 con 0x560f4456b9f0 2024-09-21T03:20:04.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.174+0000 7fa24b7fe640 10 client.25592 handle_command_reply: tid=0 2024-09-21T03:20:04.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.174+0000 7fa278f78740 1 client.25592 shutdown 2024-09-21T03:20:04.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 msgr2=0x560f44960070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44960070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa26c118290 tx=0x7fa2600118d0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 shutdown_connections 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f4456b9f0 0x560f44972320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445ad070 0x560f4496bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f44960070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 --2- 172.21.15.83:0/1229943353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f4495fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 >> 172.21.15.83:0/1229943353 conn(0x560f445a1460 msgr2=0x560f449575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.175+0000 7fa24b7fe640 0 client.25592 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:04.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.180+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 shutdown_connections 2024-09-21T03:20:04.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.180+0000 7fa278f78740 1 -- 172.21.15.83:0/1229943353 wait complete. 2024-09-21T03:20:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.181+0000 7fa278f78740 20 client.25592 trim_cache size 0 max 16384 2024-09-21T03:20:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.206+0000 7fa278f78740 1 Processor -- start 2024-09-21T03:20:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.206+0000 7fa278f78740 1 -- start start 2024-09-21T03:20:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f4495fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f4496be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f4495cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa25800e6e0 con 0x560f445841b0 2024-09-21T03:20:04.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2580102b0 con 0x560f4458c390 2024-09-21T03:20:04.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa278f78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa25800ede0 con 0x560f445ad070 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa26b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f4496be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa26b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f4496be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49434/0 (socket says 172.21.15.83:49434) 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa26b7fe640 1 -- 172.21.15.83:0/3800185277 learned_addr learned my addr 172.21.15.83:0/3800185277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f4495cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 -- 172.21.15.83:0/3800185277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 msgr2=0x560f4496be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f4496be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 -- 172.21.15.83:0/3800185277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 msgr2=0x560f4495fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f4495fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.207+0000 7fa270fa1640 1 -- 172.21.15.83:0/3800185277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f44972850 con 0x560f445ad070 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.208+0000 7fa270fa1640 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f4495cd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa26401a000 tx=0x7fa26400efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d55a4d44e3360f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.208+0000 7fa251ffb640 1 -- 172.21.15.83:0/3800185277 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640d6070 con 0x560f445ad070 2024-09-21T03:20:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.208+0000 7fa251ffb640 1 -- 172.21.15.83:0/3800185277 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2640b1ef0 con 0x560f445ad070 2024-09-21T03:20:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.208+0000 7fa251ffb640 1 -- 172.21.15.83:0/3800185277 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640ad140 con 0x560f445ad070 2024-09-21T03:20:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 -- 172.21.15.83:0/3800185277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 msgr2=0x560f4495cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f4495cd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa26401a000 tx=0x7fa26400efd0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 -- 172.21.15.83:0/3800185277 shutdown_connections 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445ad070 0x560f4495cd20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f4458c390 0x560f4496be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 --2- 172.21.15.83:0/3800185277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f445841b0 0x560f4495fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 -- 172.21.15.83:0/3800185277 >> 172.21.15.83:0/3800185277 conn(0x560f445a1460 msgr2=0x560f4495f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 -- 172.21.15.83:0/3800185277 shutdown_connections 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.209+0000 7fa278f78740 1 -- 172.21.15.83:0/3800185277 wait complete. 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 -- start start 2024-09-21T03:20:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 Processor -- start 2024-09-21T03:20:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 0x560f44963a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 0x560f44963fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa25800fbb0 con 0x560f4458c390 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa25800f4e0 con 0x560f445ad070 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.210+0000 7fa278f78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa25800c660 con 0x560f445841b0 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa278f78740 10 client.0 _ll_register_callbacks cb 0x7fa260009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa270fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 0x560f44963fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53294/0 (socket says 172.21.15.83:53294) 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 -- 172.21.15.83:0/1253194182 learned_addr learned my addr 172.21.15.83:0/1253194182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa270fa1640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 0x560f44963fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49440/0 (socket says 172.21.15.83:49440) 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 -- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 msgr2=0x560f44963fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 0x560f44963fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 0x560f44963a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 -- 172.21.15.83:0/1253194182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 msgr2=0x560f44963a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 0x560f44963a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 -- 172.21.15.83:0/1253194182 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f44964670 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 0x560f44963a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa26bfff640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009230 tx=0x7fa25c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c576cb3c5e59ce61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa24b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c035040 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.211+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2640b1ef0 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa25c0054d0 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa25c004780 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 10 client.25573 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f449dcbb0 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2640b1ef0 con 0x560f445841b0 2024-09-21T03:20:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa25c003ab0 con 0x560f445841b0 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 10 client.25573 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 10 client.25573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 20 client.25573 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f44540da0 0x560f449df380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 4 client.25573 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.212+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f448c5cd0 con 0x560f44540da0 2024-09-21T03:20:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.213+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f44540da0 0x560f449df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:04.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.213+0000 7fa24b7fe640 10 client.25573 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:04.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.214+0000 7fa26b7fe640 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f44540da0 0x560f449df380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26cb2ac6ad9e9bf8 server_cookie=793c07b4c34d9dec in_seq=0 out_seq=0 2024-09-21T03:20:04.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.214+0000 7fa24b7fe640 1 -- 172.21.15.83:0/1253194182 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f448c5cd0 con 0x560f44540da0 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.214+0000 7fa24b7fe640 10 client.25573 handle_command_reply: tid=0 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.214+0000 7fa278f78740 1 client.25573 shutdown 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.214+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 msgr2=0x560f449685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009230 tx=0x7fa25c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 shutdown_connections 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x560f44540da0 0x560f449df380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f445ad070 0x560f44963fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa24b7fe640 0 client.25573 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f4458c390 0x560f44963a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 --2- 172.21.15.83:0/1253194182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f445841b0 0x560f449685e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 >> 172.21.15.83:0/1253194182 conn(0x560f445a1460 msgr2=0x560f44955490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 shutdown_connections 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 1 -- 172.21.15.83:0/1253194182 wait complete. 2024-09-21T03:20:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa278f78740 20 client.25573 trim_cache size 0 max 16384 2024-09-21T03:20:04.220 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 msgr2=0x7fa2540b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 0x7fa2540b8d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa25c005ab0 tx=0x7fa25c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 msgr2=0x7fa26c1a83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.215+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa26400eb10 tx=0x7fa26400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 shutdown_connections 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2540b6880 0x7fa2540b8d90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa26c116510 0x7fa26c1a83b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa26c10af90 0x7fa26c1a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 --2- 172.21.15.83:0/3109706700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa26c10a640 0x7fa26c1a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 >> 172.21.15.83:0/3109706700 conn(0x7fa26c06e9f0 msgr2=0x7fa26c071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:04.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 shutdown_connections 2024-09-21T03:20:04.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:04.216+0000 7fa2717a2640 1 -- 172.21.15.83:0/3109706700 wait complete. 2024-09-21T03:20:05.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:20:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:05 smithi083 ceph-mon[31219]: pgmap v428: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-09-21T03:20:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:05 smithi142 ceph-mon[34512]: pgmap v428: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-09-21T03:20:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:05 smithi193 ceph-mon[35224]: pgmap v428: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2024-09-21T03:20:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 165.0 s, 1912.4 tps, lat 52.218 ms stddev 170.954 2024-09-21T03:20:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.400+0000 7feca29ed640 1 -- 172.21.15.83:0/4101139365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c101ce0 msgr2=0x7fec9c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.400+0000 7feca29ed640 1 --2- 172.21.15.83:0/4101139365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c101ce0 0x7fec9c1020f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec90009900 tx=0x7fec9002a410 comp rx=0 tx=0).stop 2024-09-21T03:20:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 -- 172.21.15.83:0/4101139365 shutdown_connections 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 --2- 172.21.15.83:0/4101139365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c102630 0x7fec9c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 --2- 172.21.15.83:0/4101139365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c101ce0 0x7fec9c1020f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 --2- 172.21.15.83:0/4101139365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c1053f0 0x7fec9c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 -- 172.21.15.83:0/4101139365 >> 172.21.15.83:0/4101139365 conn(0x7fec9c077c60 msgr2=0x7fec9c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 -- 172.21.15.83:0/4101139365 shutdown_connections 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 -- 172.21.15.83:0/4101139365 wait complete. 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 Processor -- start 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.401+0000 7feca29ed640 1 -- start start 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c102630 0x7fec9c1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c1053f0 0x7fec9c1a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c11a0a0 con 0x7fec9c101ce0 2024-09-21T03:20:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c119f20 con 0x7fec9c102630 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca29ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c11a220 con 0x7fec9c1053f0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56780/0 (socket says 172.21.15.83:56780) 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 -- 172.21.15.83:0/589713185 learned_addr learned my addr 172.21.15.83:0/589713185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7feca0f8a640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c1053f0 0x7fec9c1a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c102630 0x7fec9c1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 -- 172.21.15.83:0/589713185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c1053f0 msgr2=0x7fec9c1a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c1053f0 0x7fec9c1a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 -- 172.21.15.83:0/589713185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c102630 msgr2=0x7fec9c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c102630 0x7fec9c1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.402+0000 7fec9bfff640 1 -- 172.21.15.83:0/589713185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c1a6a80 con 0x7fec9c101ce0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7fec9bfff640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d950 tx=0x7fec8c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7fec997fa640 1 -- 172.21.15.83:0/589713185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c017030 con 0x7fec9c101ce0 2024-09-21T03:20:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c1a6d70 con 0x7fec9c101ce0 2024-09-21T03:20:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7fec997fa640 1 -- 172.21.15.83:0/589713185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec8c004020 con 0x7fec9c101ce0 2024-09-21T03:20:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7fec997fa640 1 -- 172.21.15.83:0/589713185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c005560 con 0x7fec9c101ce0 2024-09-21T03:20:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.403+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c1ad4d0 con 0x7fec9c101ce0 2024-09-21T03:20:06.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.406+0000 7feca8f61740 1 Processor -- start 2024-09-21T03:20:06.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.406+0000 7feca8f61740 1 -- start start 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde85d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5599dddfbea0 con 0x5599ddade070 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5599dddeb2d0 con 0x5599ddab51b0 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca8f61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599dddef0e0 con 0x5599ddabd390 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7fec997fa640 1 -- 172.21.15.83:0/589713185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fec8c0047a0 con 0x7fec9c101ce0 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56788/0 (socket says 172.21.15.83:56788) 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 -- 172.21.15.83:0/2882752877 learned_addr learned my addr 172.21.15.83:0/2882752877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde85d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7fec9bfff640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 -- 172.21.15.83:0/2882752877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 msgr2=0x5599dde85d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde85d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 -- 172.21.15.83:0/2882752877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 msgr2=0x5599dde85800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde85800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7feca0f8a640 1 -- 172.21.15.83:0/2882752877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ddb49650 con 0x5599ddade070 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.407+0000 7fec9bfff640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7feca0f8a640 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fec94011440 tx=0x7fec94011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e50059bb6029f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec827fc640 1 -- 172.21.15.83:0/2882752877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94003c30 con 0x5599ddade070 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec827fc640 1 -- 172.21.15.83:0/2882752877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec94003dd0 con 0x5599ddade070 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec827fc640 1 -- 172.21.15.83:0/2882752877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400f9e0 con 0x5599ddade070 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec997fa640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 0x7fec840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec9bfff640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 0x7fec840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.408+0000 7fec997fa640 1 -- 172.21.15.83:0/589713185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec8c0e6f20 con 0x7fec9c101ce0 2024-09-21T03:20:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 -- 172.21.15.83:0/2882752877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 msgr2=0x5599dde852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fec94011440 tx=0x7fec94011900 comp rx=0 tx=0).stop 2024-09-21T03:20:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7fec9bfff640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 0x7fec840b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0e47a0 tx=0x7fec8c0a94d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 -- 172.21.15.83:0/2882752877 shutdown_connections 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde85d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 --2- 172.21.15.83:0/2882752877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde852c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 -- 172.21.15.83:0/2882752877 >> 172.21.15.83:0/2882752877 conn(0x5599ddad2460 msgr2=0x5599dde831b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.409+0000 7feca8f61740 1 -- 172.21.15.83:0/2882752877 shutdown_connections 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.410+0000 7feca8f61740 1 -- 172.21.15.83:0/2882752877 wait complete. 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.410+0000 7feca8f61740 1 -- start start 2024-09-21T03:20:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.411+0000 7feca8f61740 1 Processor -- start 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.411+0000 7feca8f61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.411+0000 7feca8f61740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56800/0 (socket says 172.21.15.83:56800) 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7fec9b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca8f61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599dde9cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca8f61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5599ddc4c700 con 0x5599ddab51b0 2024-09-21T03:20:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca8f61740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5599ddd86160 con 0x5599ddabd390 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca8f61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599dde93aa0 con 0x5599ddade070 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca8f61740 10 client.0 _ll_register_callbacks cb 0x5599ddd9f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 -- 172.21.15.83:0/359548949 learned_addr learned my addr 172.21.15.83:0/359548949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599dde9cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 -- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 msgr2=0x5599dde9cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.412+0000 7feca0f8a640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599dde9cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7feca0f8a640 1 -- 172.21.15.83:0/359548949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 msgr2=0x5599dde90f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7feca0f8a640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde90f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7feca0f8a640 1 -- 172.21.15.83:0/359548949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599dddb3c90 con 0x5599ddab51b0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7feca0f8a640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fec9401b040 tx=0x7fec9400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bb1c581b99a09e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7fec7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400f760 con 0x5599ddab51b0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec9400f900 con 0x5599ddab51b0 2024-09-21T03:20:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec94003dd0 con 0x5599ddab51b0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.413+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fec9400fb00 con 0x5599ddab51b0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 10 client.16074 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599dde9f720 con 0x5599ddab51b0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec94003dd0 con 0x5599ddab51b0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fec94005040 con 0x5599ddab51b0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 10 client.16074 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 10 client.16074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 20 client.16074 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda9c9f0 0x5599ddea2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 4 client.16074 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.414+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599dddfddf0 con 0x5599dda9c9f0 2024-09-21T03:20:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.415+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda9c9f0 0x5599ddea2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.415+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda9c9f0 0x5599ddea2b30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0ccbf9e014093ed server_cookie=55a7522ae16b1fe3 in_seq=0 out_seq=0 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.415+0000 7fec7b7fe640 10 client.16074 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.416+0000 7fec7b7fe640 1 -- 172.21.15.83:0/359548949 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599dddfddf0 con 0x5599dda9c9f0 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.416+0000 7fec7b7fe640 10 client.16074 handle_command_reply: tid=0 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.416+0000 7feca8f61740 1 client.16074 shutdown 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 msgr2=0x5599dde909e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fec9401b040 tx=0x7fec9400f680 comp rx=0 tx=0).stop 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 shutdown_connections 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda9c9f0 0x5599ddea2b30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599dde9cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7fec7b7fe640 0 client.16074 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 --2- 172.21.15.83:0/359548949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde909e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 >> 172.21.15.83:0/359548949 conn(0x5599ddad2460 msgr2=0x5599dde884a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 shutdown_connections 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 1 -- 172.21.15.83:0/359548949 wait complete. 2024-09-21T03:20:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.417+0000 7feca8f61740 20 client.16074 trim_cache size 0 max 16384 2024-09-21T03:20:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 Processor -- start 2024-09-21T03:20:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 -- start start 2024-09-21T03:20:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde9cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b5c00 con 0x5599ddade070 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b77d0 con 0x5599ddab51b0 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.436+0000 7feca8f61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b6300 con 0x5599ddabd390 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde9cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde9cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56804/0 (socket says 172.21.15.83:56804) 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7feca0f8a640 1 -- 172.21.15.83:0/2321008411 learned_addr learned my addr 172.21.15.83:0/2321008411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:06.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7feca0f8a640 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 -- 172.21.15.83:0/2321008411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 msgr2=0x5599dde86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde86cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 -- 172.21.15.83:0/2321008411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 msgr2=0x5599dde9cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde9cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.437+0000 7fec9b7fe640 1 -- 172.21.15.83:0/2321008411 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599dde7d3b0 con 0x5599ddabd390 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.438+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde92610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec9002a930 tx=0x7fec90005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72fa26153df9dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.438+0000 7fec81ffb640 1 -- 172.21.15.83:0/2321008411 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9004a020 con 0x5599ddabd390 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.438+0000 7fec81ffb640 1 -- 172.21.15.83:0/2321008411 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90004750 con 0x5599ddabd390 2024-09-21T03:20:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.438+0000 7fec81ffb640 1 -- 172.21.15.83:0/2321008411 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90035040 con 0x5599ddabd390 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.439+0000 7feca8f61740 1 -- 172.21.15.83:0/2321008411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 msgr2=0x5599dde92610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.439+0000 7feca8f61740 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde92610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec9002a930 tx=0x7fec90005a20 comp rx=0 tx=0).stop 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 -- 172.21.15.83:0/2321008411 shutdown_connections 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddade070 0x5599dde9cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddabd390 0x5599dde92610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 --2- 172.21.15.83:0/2321008411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddab51b0 0x5599dde86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 -- 172.21.15.83:0/2321008411 >> 172.21.15.83:0/2321008411 conn(0x5599ddad2460 msgr2=0x5599dde91f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 -- 172.21.15.83:0/2321008411 shutdown_connections 2024-09-21T03:20:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.440+0000 7feca8f61740 1 -- 172.21.15.83:0/2321008411 wait complete. 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 -- start start 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 Processor -- start 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde97580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599ddf06370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b70d0 con 0x5599ddab51b0 2024-09-21T03:20:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b6a00 con 0x5599ddabd390 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1b3be0 con 0x5599ddade070 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca8f61740 10 client.0 _ll_register_callbacks cb 0x7fec90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7fec9b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde97580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7fec9b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49482/0 (socket says 172.21.15.83:49482) 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca0f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde97580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56812/0 (socket says 172.21.15.83:56812) 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7fec9b7fe640 1 -- 172.21.15.83:0/3650349538 learned_addr learned my addr 172.21.15.83:0/3650349538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.441+0000 7feca0f8a640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599ddf06370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 -- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 msgr2=0x5599ddf06370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599ddf06370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 -- 172.21.15.83:0/3650349538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 msgr2=0x5599dde97580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde97580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 -- 172.21.15.83:0/3650349538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ddf069b0 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec9b7fe640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec90005ab0 tx=0x7fec90004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dfe8927abc3e074 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90035030 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fec90005040 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec90004750 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fec9005e050 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7feca8f61740 10 client.25613 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.442+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599ddf0dce0 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90004750 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fec900314a0 con 0x5599ddabd390 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 10 client.25613 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 10 client.25613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 20 client.25613 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda71da0 0x5599ddf10470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:06.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 4 client.25613 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:06.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599dddf98a0 con 0x5599dda71da0 2024-09-21T03:20:06.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.443+0000 7feca0f8a640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda71da0 0x5599ddf10470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:06.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.444+0000 7fec7b7fe640 10 client.25613 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca0f8a640 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda71da0 0x5599ddf10470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=362a174f32192657 server_cookie=64ee4aa59d15f41d in_seq=0 out_seq=0 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7fec7b7fe640 1 -- 172.21.15.83:0/3650349538 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599dddf98a0 con 0x5599dda71da0 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7fec7b7fe640 10 client.25613 handle_command_reply: tid=0 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 client.25613 shutdown 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 msgr2=0x5599dde97b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec90005ab0 tx=0x7fec90004b30 comp rx=0 tx=0).stop 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 shutdown_connections 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5599dda71da0 0x5599ddf10470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599ddade070 0x5599ddf06370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599ddabd390 0x5599dde97b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 --2- 172.21.15.83:0/3650349538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599ddab51b0 0x5599dde97580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7fec7b7fe640 0 client.25613 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.445+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 >> 172.21.15.83:0/3650349538 conn(0x5599ddad2460 msgr2=0x5599dde869b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 shutdown_connections 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca8f61740 1 -- 172.21.15.83:0/3650349538 wait complete. 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca8f61740 20 client.25613 trim_cache size 0 max 16384 2024-09-21T03:20:06.451 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 msgr2=0x7fec840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 0x7fec840b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0e47a0 tx=0x7fec8c0a94d0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 msgr2=0x7fec9c1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d950 tx=0x7fec8c00dd50 comp rx=0 tx=0).stop 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 shutdown_connections 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fec840b6680 0x7fec840b8b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec9c1053f0 0x7fec9c1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c102630 0x7fec9c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 --2- 172.21.15.83:0/589713185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9c101ce0 0x7fec9c1a0b10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 >> 172.21.15.83:0/589713185 conn(0x7fec9c077c60 msgr2=0x7fec9c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 shutdown_connections 2024-09-21T03:20:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:06.446+0000 7feca29ed640 1 -- 172.21.15.83:0/589713185 wait complete. 2024-09-21T03:20:07.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:07 smithi083 ceph-mon[31219]: pgmap v429: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T03:20:07.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:20:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:07 smithi142 ceph-mon[34512]: pgmap v429: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T03:20:07.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:07 smithi193 ceph-mon[35224]: pgmap v429: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T03:20:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 -- 172.21.15.83:0/1321636921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f41083a0 msgr2=0x7fb0f41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 --2- 172.21.15.83:0/1321636921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f41083a0 0x7fb0f41087b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009900 tx=0x7fb0e802a410 comp rx=0 tx=0).stop 2024-09-21T03:20:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 -- 172.21.15.83:0/1321636921 shutdown_connections 2024-09-21T03:20:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 --2- 172.21.15.83:0/1321636921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f4108cf0 0x7fb0f410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 --2- 172.21.15.83:0/1321636921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f41083a0 0x7fb0f41087b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 --2- 172.21.15.83:0/1321636921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4102a60 0x7fb0f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 -- 172.21.15.83:0/1321636921 >> 172.21.15.83:0/1321636921 conn(0x7fb0f4079730 msgr2=0x7fb0f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 -- 172.21.15.83:0/1321636921 shutdown_connections 2024-09-21T03:20:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.503+0000 7fb0fa624640 1 -- 172.21.15.83:0/1321636921 wait complete. 2024-09-21T03:20:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.504+0000 7fb0fa624640 1 Processor -- start 2024-09-21T03:20:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.504+0000 7fb0fa624640 1 -- start start 2024-09-21T03:20:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41083a0 0x7fb0f41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108cf0 0x7fb0f419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112960 con 0x7fb0f4102a60 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41127e0 con 0x7fb0f41083a0 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0fa624640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112ae0 con 0x7fb0f4108cf0 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56818/0 (socket says 172.21.15.83:56818) 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0f3fff640 1 -- 172.21.15.83:0/3686389283 learned_addr learned my addr 172.21.15.83:0/3686389283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108cf0 0x7fb0f419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.505+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41083a0 0x7fb0f41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 -- 172.21.15.83:0/3686389283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108cf0 msgr2=0x7fb0f419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108cf0 0x7fb0f419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 -- 172.21.15.83:0/3686389283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41083a0 msgr2=0x7fb0f41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41083a0 0x7fb0f41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 -- 172.21.15.83:0/3686389283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f419b010 con 0x7fb0f4102a60 2024-09-21T03:20:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f3fff640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9f0 tx=0x7fb0e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.506+0000 7fb0f17fa640 1 -- 172.21.15.83:0/3686389283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400cd80 con 0x7fb0f4102a60 2024-09-21T03:20:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.507+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f419b2a0 con 0x7fb0f4102a60 2024-09-21T03:20:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.507+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41a2cb0 con 0x7fb0f4102a60 2024-09-21T03:20:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.507+0000 7fb0f17fa640 1 -- 172.21.15.83:0/3686389283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e40054d0 con 0x7fb0f4102a60 2024-09-21T03:20:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.507+0000 7fb0f17fa640 1 -- 172.21.15.83:0/3686389283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4010690 con 0x7fb0f4102a60 2024-09-21T03:20:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.510+0000 7fb0f17fa640 1 -- 172.21.15.83:0/3686389283 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb0e4010830 con 0x7fb0f4102a60 2024-09-21T03:20:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.510+0000 7fb100b98740 1 Processor -- start 2024-09-21T03:20:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.510+0000 7fb100b98740 1 -- start start 2024-09-21T03:20:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.510+0000 7fb100b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.510+0000 7fb100b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb100b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a5c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb100b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621259d1ea0 con 0x56212568b1b0 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49516/0 (socket says 172.21.15.83:49516) 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb100b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5621259c12d0 con 0x5621256b4070 2024-09-21T03:20:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621259c50e0 con 0x562125693390 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/941727874 learned_addr learned my addr 172.21.15.83:0/941727874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a5c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/941727874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 msgr2=0x562125a5c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a5c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/941727874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 msgr2=0x562125a5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/941727874 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212571f650 con 0x5621256b4070 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a5c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009f50 tx=0x7fb0ec0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff7a6f698092871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f17fa640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 0x7fb0dc0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.511+0000 7fb0f3fff640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 0x7fb0dc0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.512+0000 7fb0da7fc640 1 -- 172.21.15.83:0/941727874 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec004d20 con 0x5621256b4070 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.512+0000 7fb0da7fc640 1 -- 172.21.15.83:0/941727874 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ec004050 con 0x5621256b4070 2024-09-21T03:20:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.512+0000 7fb0da7fc640 1 -- 172.21.15.83:0/941727874 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec005280 con 0x5621256b4070 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.512+0000 7fb0f17fa640 1 -- 172.21.15.83:0/3686389283 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb0e40e7870 con 0x7fb0f4102a60 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 msgr2=0x562125a5b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb0f3fff640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 0x7fb0dc0b8d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x562125a5a320 tx=0x7fb0e40e9040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009f50 tx=0x7fb0ec0118d0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 shutdown_connections 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a5c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 --2- 172.21.15.83:0/941727874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a5b6a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 >> 172.21.15.83:0/941727874 conn(0x5621256a8460 msgr2=0x562125a59590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 shutdown_connections 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- 172.21.15.83:0/941727874 wait complete. 2024-09-21T03:20:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.513+0000 7fb100b98740 1 -- start start 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.514+0000 7fb100b98740 1 Processor -- start 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562125693390 0x562125a67310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53384/0 (socket says 172.21.15.83:53384) 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/2405571942 learned_addr learned my addr 172.21.15.83:0/2405571942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562125693390 0x562125a67310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a73200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562125822700 con 0x562125693390 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56212595c160 con 0x5621256b4070 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.515+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562125a69e80 con 0x56212568b1b0 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a73200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 msgr2=0x562125a73200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a73200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/2405571942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562125693390 msgr2=0x562125a67310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562125693390 0x562125a67310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562125989c90 con 0x56212568b1b0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb100b98740 10 client.0 _ll_register_callbacks cb 0x5621259759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009250 tx=0x7fb0ec004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14125d9ab335b60c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb0d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.516+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ec004050 con 0x56212568b1b0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb0d37fe640 1 -- 172.21.15.83:0/2405571942 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec02b020 con 0x56212568b1b0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb0d37fe640 1 -- 172.21.15.83:0/2405571942 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb0ec005040 con 0x56212568b1b0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb0d37fe640 1 -- 172.21.15.83:0/2405571942 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec0052c0 con 0x56212568b1b0 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 10 client.25603 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=35}) -- 0x562125a75a30 con 0x56212568b1b0 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb0d37fe640 1 -- 172.21.15.83:0/2405571942 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb0ec005460 con 0x56212568b1b0 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 10 client.25603 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 20 client.25603 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5621256729f0 0x562125a78df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 4 client.25603 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621259d3df0 con 0x5621256729f0 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.517+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5621256729f0 0x562125a78df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.518+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5621256729f0 0x562125a78df0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3741ec8cb54865e server_cookie=8b543fed63135d9d in_seq=0 out_seq=0 2024-09-21T03:20:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.518+0000 7fb0d37fe640 10 client.25603 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb0d37fe640 1 -- 172.21.15.83:0/2405571942 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5621259d3df0 con 0x5621256729f0 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb0d37fe640 10 client.25603 handle_command_reply: tid=0 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 client.25603 shutdown 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 msgr2=0x562125a66dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009250 tx=0x7fb0ec004830 comp rx=0 tx=0).stop 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 shutdown_connections 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5621256729f0 0x562125a78df0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621256b4070 0x562125a73200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562125693390 0x562125a67310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 --2- 172.21.15.83:0/2405571942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56212568b1b0 0x562125a66dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 >> 172.21.15.83:0/2405571942 conn(0x5621256a8460 msgr2=0x562125a5e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb0d37fe640 0 client.25603 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 shutdown_connections 2024-09-21T03:20:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 1 -- 172.21.15.83:0/2405571942 wait complete. 2024-09-21T03:20:08.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.519+0000 7fb100b98740 20 client.25603 trim_cache size 0 max 16384 2024-09-21T03:20:08.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 Processor -- start 2024-09-21T03:20:08.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 -- start start 2024-09-21T03:20:08.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 0x562125a73100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621256b4070 0x562125a5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e000e6a0 con 0x56212568b1b0 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0010270 con 0x562125693390 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb100b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e000eda0 con 0x5621256b4070 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb0f37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 0x562125a73100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.537+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a59600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56848/0 (socket says 172.21.15.83:56848) 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f8bc1640 1 -- 172.21.15.83:0/1545688461 learned_addr learned my addr 172.21.15.83:0/1545688461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621256b4070 0x562125a5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 -- 172.21.15.83:0/1545688461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621256b4070 msgr2=0x562125a5e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621256b4070 0x562125a5e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 -- 172.21.15.83:0/1545688461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 msgr2=0x562125a59600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a59600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 -- 172.21.15.83:0/1545688461 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562125a68090 con 0x562125693390 2024-09-21T03:20:08.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.538+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 0x562125a73100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8006010 tx=0x7fb0e8031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10ba50b82d54a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.539+0000 7fb0d9ffb640 1 -- 172.21.15.83:0/1545688461 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8035040 con 0x562125693390 2024-09-21T03:20:08.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.539+0000 7fb0d9ffb640 1 -- 172.21.15.83:0/1545688461 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80054f0 con 0x562125693390 2024-09-21T03:20:08.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.539+0000 7fb0d9ffb640 1 -- 172.21.15.83:0/1545688461 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004c80 con 0x562125693390 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 -- 172.21.15.83:0/1545688461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 msgr2=0x562125a73100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 0x562125a73100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8006010 tx=0x7fb0e8031400 comp rx=0 tx=0).stop 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 -- 172.21.15.83:0/1545688461 shutdown_connections 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621256b4070 0x562125a5e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562125693390 0x562125a73100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 --2- 172.21.15.83:0/1545688461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56212568b1b0 0x562125a59600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 -- 172.21.15.83:0/1545688461 >> 172.21.15.83:0/1545688461 conn(0x5621256a8460 msgr2=0x562125a58ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 -- 172.21.15.83:0/1545688461 shutdown_connections 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.540+0000 7fb100b98740 1 -- 172.21.15.83:0/1545688461 wait complete. 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 -- start start 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 Processor -- start 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 0x562125a6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621256b4070 0x562125a6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e000fb70 con 0x5621256b4070 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e000f4a0 con 0x56212568b1b0 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e000c680 con 0x562125693390 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.541+0000 7fb100b98740 10 client.0 _ll_register_callbacks cb 0x7fb0ec009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 0x562125a6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53408/0 (socket says 172.21.15.83:53408) 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 -- 172.21.15.83:0/18415834 learned_addr learned my addr 172.21.15.83:0/18415834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f8bc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 0x562125a6fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49542/0 (socket says 172.21.15.83:49542) 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621256b4070 0x562125a6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 -- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 msgr2=0x562125a6fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 0x562125a6fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 -- 172.21.15.83:0/18415834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621256b4070 msgr2=0x562125a6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621256b4070 0x562125a6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 -- 172.21.15.83:0/18415834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562125a6bcf0 con 0x562125693390 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0f37fe640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0e802a930 tx=0x7fb0e80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4852ffad6483ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8053020 con 0x562125693390 2024-09-21T03:20:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb0e8004450 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.542+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e80054f0 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.543+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0e8035040 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 10 client.25609 fetch_fsmap learned FSMap version 35 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562125ae37b0 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80054f0 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb0e8004c30 con 0x562125693390 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 10 client.25609 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 20 client.25609 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562125647da0 0x562125ae5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 4 client.25609 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.544+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621259cf8a0 con 0x562125647da0 2024-09-21T03:20:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.545+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562125647da0 0x562125ae5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.545+0000 7fb0d37fe640 10 client.25609 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.545+0000 7fb0f8bc1640 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562125647da0 0x562125ae5e80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82076e4d63999a2d server_cookie=73ee3dd3ec831a71 in_seq=0 out_seq=0 2024-09-21T03:20:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb0d37fe640 1 -- 172.21.15.83:0/18415834 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621259cf8a0 con 0x562125647da0 2024-09-21T03:20:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb0d37fe640 10 client.25609 handle_command_reply: tid=0 2024-09-21T03:20:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 client.25609 shutdown 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 msgr2=0x562125a6af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0e802a930 tx=0x7fb0e80047b0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 shutdown_connections 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562125647da0 0x562125ae5e80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621256b4070 0x562125a6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562125693390 0x562125a6af10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb0d37fe640 0 client.25609 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 --2- 172.21.15.83:0/18415834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56212568b1b0 0x562125a6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 >> 172.21.15.83:0/18415834 conn(0x5621256a8460 msgr2=0x562125a68260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 shutdown_connections 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 1 -- 172.21.15.83:0/18415834 wait complete. 2024-09-21T03:20:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.546+0000 7fb100b98740 20 client.25609 trim_cache size 0 max 16384 2024-09-21T03:20:08.554 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:08.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 msgr2=0x7fb0dc0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 0x7fb0dc0b8d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x562125a5a320 tx=0x7fb0e40e9040 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 msgr2=0x7fb0f4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9f0 tx=0x7fb0e400edf0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 shutdown_connections 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb0dc0b6850 0x7fb0dc0b8d60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0f4108cf0 0x7fb0f419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0f41083a0 0x7fb0f41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 --2- 172.21.15.83:0/3686389283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0f4102a60 0x7fb0f4198360 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.547+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 >> 172.21.15.83:0/3686389283 conn(0x7fb0f4079730 msgr2=0x7fb0f41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.549+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 shutdown_connections 2024-09-21T03:20:08.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:08.549+0000 7fb0fa624640 1 -- 172.21.15.83:0/3686389283 wait complete. 2024-09-21T03:20:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:09 smithi083 ceph-mon[31219]: pgmap v430: 97 pgs: 97 active+clean; 2.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 1.63k op/s 2024-09-21T03:20:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:09 smithi142 ceph-mon[34512]: pgmap v430: 97 pgs: 97 active+clean; 2.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 1.63k op/s 2024-09-21T03:20:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:09 smithi193 ceph-mon[35224]: pgmap v430: 97 pgs: 97 active+clean; 2.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 40 MiB/s wr, 1.63k op/s 2024-09-21T03:20:10.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:20:10.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:20:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 170.0 s, 2652.2 tps, lat 37.730 ms stddev 46.946 2024-09-21T03:20:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:11.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.602+0000 7fa30155a640 1 -- 172.21.15.83:0/2925485178 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f402f310 con 0x7fa2fc111f40 2024-09-21T03:20:11.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 -- 172.21.15.83:0/2925485178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc111f40 msgr2=0x7fa2fc10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 --2- 172.21.15.83:0/2925485178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc111f40 0x7fa2fc10a3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009ca0 tx=0x7fa2f402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 -- 172.21.15.83:0/2925485178 shutdown_connections 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 --2- 172.21.15.83:0/2925485178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc11cc30 0x7fa2fc11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 --2- 172.21.15.83:0/2925485178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 --2- 172.21.15.83:0/2925485178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc111f40 0x7fa2fc10a3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 -- 172.21.15.83:0/2925485178 >> 172.21.15.83:0/2925485178 conn(0x7fa2fc070ec0 msgr2=0x7fa2fc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 -- 172.21.15.83:0/2925485178 shutdown_connections 2024-09-21T03:20:11.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.603+0000 7fa3047c0640 1 -- 172.21.15.83:0/2925485178 wait complete. 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.604+0000 7fa3047c0640 1 Processor -- start 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.604+0000 7fa3047c0640 1 -- start start 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 0x7fa2fc1a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc11cc30 0x7fa2fc1aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc122900 con 0x7fa2fc10a9e0 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc122780 con 0x7fa2fc111f40 2024-09-21T03:20:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa3047c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc122a80 con 0x7fa2fc11cc30 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa301d5b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 0x7fa2fc1a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa30255c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa30255c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47690/0 (socket says 172.21.15.83:47690) 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.605+0000 7fa30255c640 1 -- 172.21.15.83:0/4189134053 learned_addr learned my addr 172.21.15.83:0/4189134053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 -- 172.21.15.83:0/4189134053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc11cc30 msgr2=0x7fa2fc1aeb50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc11cc30 0x7fa2fc1aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 -- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 msgr2=0x7fa2fc1a9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 -- 172.21.15.83:0/4189134053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc1af280 con 0x7fa2fc111f40 2024-09-21T03:20:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa30255c640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.606+0000 7fa301d5b640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 0x7fa2fc1a98b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc06c6e0 tx=0x7fa2f800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.607+0000 7fa2f37fe640 1 -- 172.21.15.83:0/4189134053 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800cc40 con 0x7fa2fc111f40 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.607+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc1af570 con 0x7fa2fc111f40 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.607+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc1b5cd0 con 0x7fa2fc111f40 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.607+0000 7fa2f37fe640 1 -- 172.21.15.83:0/4189134053 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f800cde0 con 0x7fa2fc111f40 2024-09-21T03:20:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.607+0000 7fa2f37fe640 1 -- 172.21.15.83:0/4189134053 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8010430 con 0x7fa2fc111f40 2024-09-21T03:20:11.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.610+0000 7fa2f37fe640 1 -- 172.21.15.83:0/4189134053 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa2f80105d0 con 0x7fa2fc111f40 2024-09-21T03:20:11.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.610+0000 7fa30ad34740 1 Processor -- start 2024-09-21T03:20:11.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30ad34740 1 -- start start 2024-09-21T03:20:11.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa2f37fe640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 0x7fa2e80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 0x5641b8fe3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa302d5d640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 0x7fa2e80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30255c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 0x5641b8fe3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30255c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 0x5641b8fe3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55458/0 (socket says 172.21.15.83:55458) 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.612+0000 7fa30255c640 1 -- 172.21.15.83:0/3922194398 learned_addr learned my addr 172.21.15.83:0/3922194398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa30ad34740 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8fe3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641b8f59e60 con 0x5641b8c1b670 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5641b8f49290 con 0x5641b8c13490 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641b8f4d0a0 con 0x5641b8c3c350 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa301d5b640 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa302d5d640 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8fe3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa2f37fe640 1 -- 172.21.15.83:0/4189134053 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f80e79c0 con 0x7fa2fc111f40 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.613+0000 7fa302d5d640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 0x7fa2e80b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005d80 tx=0x7fa2ec005c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 -- 172.21.15.83:0/3922194398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 msgr2=0x5641b8fe3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 0x5641b8fe3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 -- 172.21.15.83:0/3922194398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 msgr2=0x5641b8fe3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8fe3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 -- 172.21.15.83:0/3922194398 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b8ca7930 con 0x5641b8c13490 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa301d5b640 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe3970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80e5190 tx=0x7fa2f800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27cb69e29830c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa2f0ff9640 1 -- 172.21.15.83:0/3922194398 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80b2370 con 0x5641b8c13490 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa2f0ff9640 1 -- 172.21.15.83:0/3922194398 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f80b72b0 con 0x5641b8c13490 2024-09-21T03:20:11.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.614+0000 7fa2f0ff9640 1 -- 172.21.15.83:0/3922194398 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80035f0 con 0x5641b8c13490 2024-09-21T03:20:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.615+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 msgr2=0x5641b8fe3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.615+0000 7fa30ad34740 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe3970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80e5190 tx=0x7fa2f800cf90 comp rx=0 tx=0).stop 2024-09-21T03:20:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.615+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 shutdown_connections 2024-09-21T03:20:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.615+0000 7fa30ad34740 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8fe3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.616+0000 7fa30ad34740 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe3970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.616+0000 7fa30ad34740 1 --2- 172.21.15.83:0/3922194398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c3c350 0x5641b8fe3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.616+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 >> 172.21.15.83:0/3922194398 conn(0x5641b8c30740 msgr2=0x5641b8fe1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.617+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 shutdown_connections 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.617+0000 7fa30ad34740 1 -- 172.21.15.83:0/3922194398 wait complete. 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 -- start start 2024-09-21T03:20:11.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 Processor -- start 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c1b670 0x5641b8fef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8ffaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641b8daa890 con 0x5641b8c3c350 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5641b8ee3d40 con 0x5641b8c1b670 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641b8ff1b50 con 0x5641b8c13490 2024-09-21T03:20:11.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.618+0000 7fa30ad34740 10 client.0 _ll_register_callbacks cb 0x5641b8f47c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.619+0000 7fa30255c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.619+0000 7fa30255c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55472/0 (socket says 172.21.15.83:55472) 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.619+0000 7fa30255c640 1 -- 172.21.15.83:0/2697864741 learned_addr learned my addr 172.21.15.83:0/2697864741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.619+0000 7fa302d5d640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8ffaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.619+0000 7fa301d5b640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c1b670 0x5641b8fef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 -- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c1b670 msgr2=0x5641b8fef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c1b670 0x5641b8fef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 -- 172.21.15.83:0/2697864741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 msgr2=0x5641b8ffaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8ffaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 -- 172.21.15.83:0/2697864741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b8ffb4e0 con 0x5641b8c13490 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa30255c640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009a30 tx=0x7fa2f4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293dc892897f9f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.620+0000 7fa2e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f404b060 con 0x5641b8c13490 2024-09-21T03:20:11.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f80b72b0 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f4003b80 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa2f4003d80 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 10 client.25618 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641b8ffde40 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f80b72b0 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa2f402f7d0 con 0x5641b8c13490 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 10 client.25618 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 20 client.25618 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bfacd0 0x5641b9001490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 4 client.25618 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.621+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b8f5bdb0 con 0x5641b8bfacd0 2024-09-21T03:20:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.622+0000 7fa301d5b640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bfacd0 0x5641b9001490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.623+0000 7fa2e5ffb640 10 client.25618 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.623+0000 7fa301d5b640 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bfacd0 0x5641b9001490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fce7ab91aa0d01 server_cookie=126d125ca3ecf8ca in_seq=0 out_seq=0 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.623+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/2697864741 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641b8f5bdb0 con 0x5641b8bfacd0 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.623+0000 7fa2e5ffb640 10 client.25618 handle_command_reply: tid=0 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.623+0000 7fa30ad34740 1 client.25618 shutdown 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 msgr2=0x5641b8fef0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009a30 tx=0x7fa2f4031a20 comp rx=0 tx=0).stop 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 shutdown_connections 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bfacd0 0x5641b9001490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8ffaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c1b670 0x5641b8fef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 --2- 172.21.15.83:0/2697864741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8fef0b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa2e5ffb640 0 client.25618 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 >> 172.21.15.83:0/2697864741 conn(0x5641b8c30740 msgr2=0x5641b8fe65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 shutdown_connections 2024-09-21T03:20:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 1 -- 172.21.15.83:0/2697864741 wait complete. 2024-09-21T03:20:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.624+0000 7fa30ad34740 20 client.25618 trim_cache size 0 max 16384 2024-09-21T03:20:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.648+0000 7fa30ad34740 1 Processor -- start 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.648+0000 7fa30ad34740 1 -- start start 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8fe84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000e620 con 0x5641b8c3c350 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e00101c0 con 0x5641b8c13490 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30ad34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000ecf0 con 0x5641b8c1b670 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa301d5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa302d5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8fe84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa301d5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55482/0 (socket says 172.21.15.83:55482) 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:48512/0 (socket says 172.21.15.83:48512) 2024-09-21T03:20:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa301d5b640 1 -- 172.21.15.83:0/1754670870 learned_addr learned my addr 172.21.15.83:0/1754670870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa302d5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8fe84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47722/0 (socket says 172.21.15.83:47722) 2024-09-21T03:20:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 -- 172.21.15.83:0/1754670870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 msgr2=0x5641b8fe6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 -- 172.21.15.83:0/1754670870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 msgr2=0x5641b8fe84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8fe84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa30255c640 1 -- 172.21.15.83:0/1754670870 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b8fe5720 con 0x5641b8c13490 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.649+0000 7fa301d5b640 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.650+0000 7fa30255c640 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40097b0 tx=0x7fa2f402ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b047220f5e5012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.650+0000 7fa2e7fff640 1 -- 172.21.15.83:0/1754670870 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f404b040 con 0x5641b8c13490 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.650+0000 7fa2e7fff640 1 -- 172.21.15.83:0/1754670870 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f4036040 con 0x5641b8c13490 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.650+0000 7fa2e7fff640 1 -- 172.21.15.83:0/1754670870 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4030980 con 0x5641b8c13490 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- 172.21.15.83:0/1754670870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 msgr2=0x5641b8fe6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40097b0 tx=0x7fa2f402ae60 comp rx=0 tx=0).stop 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- 172.21.15.83:0/1754670870 shutdown_connections 2024-09-21T03:20:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c3c350 0x5641b8fe84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c1b670 0x5641b8fe6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 --2- 172.21.15.83:0/1754670870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c13490 0x5641b8fe6530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- 172.21.15.83:0/1754670870 >> 172.21.15.83:0/1754670870 conn(0x5641b8c30740 msgr2=0x5641b8fed730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- 172.21.15.83:0/1754670870 shutdown_connections 2024-09-21T03:20:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- 172.21.15.83:0/1754670870 wait complete. 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.651+0000 7fa30ad34740 1 -- start start 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 Processor -- start 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8ff3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c3c350 0x5641b906ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000fac0 con 0x5641b8c1b670 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000f3f0 con 0x5641b8c3c350 2024-09-21T03:20:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa301d5b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000c600 con 0x5641b8c13490 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa301d5b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47732/0 (socket says 172.21.15.83:47732) 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa301d5b640 1 -- 172.21.15.83:0/251401409 learned_addr learned my addr 172.21.15.83:0/251401409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30255c640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8ff3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa302d5d640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c3c350 0x5641b906ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.652+0000 7fa30ad34740 10 client.0 _ll_register_callbacks cb 0x7fa2f4032f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 -- 172.21.15.83:0/251401409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 msgr2=0x5641b8ff3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8ff3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 -- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c3c350 msgr2=0x5641b906ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c3c350 0x5641b906ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 -- 172.21.15.83:0/251401409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b8ff3db0 con 0x5641b8c1b670 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa30255c640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8ff3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa301d5b640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009250 tx=0x7fa2f80e5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeba54545b065e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa2e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80a9730 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa2f80aebc0 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f4036040 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa2f80b72b0 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa30ad34740 10 client.16116 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.653+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641b906c370 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f4036040 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa2f80035f0 con 0x5641b8c1b670 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 10 client.16116 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 20 client.16116 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bd0080 0x5641b906e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 4 client.16116 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:20:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641b8f57860 con 0x5641b8bd0080 2024-09-21T03:20:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30255c640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bd0080 0x5641b906e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa2e5ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.654+0000 7fa30255c640 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bd0080 0x5641b906e710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2049e47aa5475f54 server_cookie=96e5fe224f8983cd in_seq=0 out_seq=0 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa2e5ffb640 1 -- 172.21.15.83:0/251401409 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641b8f57860 con 0x5641b8bd0080 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa2e5ffb640 10 client.16116 handle_command_reply: tid=0 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 client.16116 shutdown 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 msgr2=0x5641b8ff35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009250 tx=0x7fa2f80e5640 comp rx=0 tx=0).stop 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 shutdown_connections 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5641b8bd0080 0x5641b906e710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641b8c3c350 0x5641b906ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641b8c1b670 0x5641b8ff35a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 --2- 172.21.15.83:0/251401409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641b8c13490 0x5641b8ff3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 >> 172.21.15.83:0/251401409 conn(0x5641b8c30740 msgr2=0x5641b8fe6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa2e5ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 shutdown_connections 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.657+0000 7fa30ad34740 1 -- 172.21.15.83:0/251401409 wait complete. 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.658+0000 7fa30ad34740 20 client.16116 trim_cache size 0 max 16384 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "90c69917-6ace-4c38-8d33-f0da1deff7a8", 2024-09-21T03:20:11.662 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 msgr2=0x7fa2e80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 0x7fa2e80b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005d80 tx=0x7fa2ec005c90 comp rx=0 tx=0).stop 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 msgr2=0x7fa2fc1a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 0x7fa2fc1a98b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc06c6e0 tx=0x7fa2f800ee50 comp rx=0 tx=0).stop 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 shutdown_connections 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa2e80b66b0 0x7fa2e80b8bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2fc11cc30 0x7fa2fc1aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2fc111f40 0x7fa2fc1a98b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 --2- 172.21.15.83:0/4189134053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2fc10a9e0 0x7fa2fc1a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 >> 172.21.15.83:0/4189134053 conn(0x7fa2fc070ec0 msgr2=0x7fa2fc071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 shutdown_connections 2024-09-21T03:20:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:11.659+0000 7fa3047c0640 1 -- 172.21.15.83:0/4189134053 wait complete. 2024-09-21T03:20:12.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: pgmap v431: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:12 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: pgmap v431: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2024-09-21T03:20:12.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:12 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: pgmap v431: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:12 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:12.692 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 -- 172.21.15.83:0/2734760279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1010108d20 msgr2=0x7f101010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 --2- 172.21.15.83:0/2734760279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1010108d20 0x7f101010f550 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1008009ca0 tx=0x7f100802a8b0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 -- 172.21.15.83:0/2734760279 shutdown_connections 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 --2- 172.21.15.83:0/2734760279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1010108d20 0x7f101010f550 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 --2- 172.21.15.83:0/2734760279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10101083d0 0x7f10101087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 --2- 172.21.15.83:0/2734760279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010102ae0 0x7f1010102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 -- 172.21.15.83:0/2734760279 >> 172.21.15.83:0/2734760279 conn(0x7f10100797b0 msgr2=0x7f1010079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 -- 172.21.15.83:0/2734760279 shutdown_connections 2024-09-21T03:20:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.239+0000 7f10162e4640 1 -- 172.21.15.83:0/2734760279 wait complete. 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 Processor -- start 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 -- start start 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102ae0 0x7f101019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010108d20 0x7f101019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1010113ae0 con 0x7f10101083d0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1010113960 con 0x7f1010102ae0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f10162e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1010113c60 con 0x7f1010108d20 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f1007fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f1007fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47752/0 (socket says 172.21.15.83:47752) 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f100ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102ae0 0x7f101019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f1007fff640 1 -- 172.21.15.83:0/3067284014 learned_addr learned my addr 172.21.15.83:0/3067284014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.240+0000 7f1014881640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010108d20 0x7f101019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 -- 172.21.15.83:0/3067284014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010108d20 msgr2=0x7f101019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010108d20 0x7f101019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 -- 172.21.15.83:0/3067284014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102ae0 msgr2=0x7f101019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102ae0 0x7f101019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 -- 172.21.15.83:0/3067284014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10101a0320 con 0x7f10101083d0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f1007fff640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c840 tx=0x7f0ff800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f100dffb640 1 -- 172.21.15.83:0/3067284014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8007bb0 con 0x7f10101083d0 2024-09-21T03:20:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f100dffb640 1 -- 172.21.15.83:0/3067284014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff8007d50 con 0x7f10101083d0 2024-09-21T03:20:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101a0610 con 0x7f10101083d0 2024-09-21T03:20:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f100dffb640 1 -- 172.21.15.83:0/3067284014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8018730 con 0x7f10101083d0 2024-09-21T03:20:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.241+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101a6e30 con 0x7f10101083d0 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.244+0000 7f101c858740 1 Processor -- start 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.244+0000 7f101c858740 1 -- start start 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.244+0000 7f101c858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 0x561713154650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.244+0000 7f101c858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.244+0000 7f101c858740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 0x5617131550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f101c858740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5617130cae50 con 0x561712d8c390 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f101c858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5617130ba280 con 0x561712dad070 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f101c858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617130be090 con 0x561712d841b0 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f100ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 0x561713154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f100ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55524/0 (socket says 172.21.15.83:55524) 2024-09-21T03:20:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f100ffff640 1 -- 172.21.15.83:0/2824453925 learned_addr learned my addr 172.21.15.83:0/2824453925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f100dffb640 1 -- 172.21.15.83:0/3067284014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0ff80188d0 con 0x7f10101083d0 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1007fff640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 0x5617131550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 -- 172.21.15.83:0/2824453925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 msgr2=0x561713154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 -- 172.21.15.83:0/2824453925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 msgr2=0x5617131550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 0x5617131550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 -- 172.21.15.83:0/2824453925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561712e18650 con 0x561712dad070 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f100ffff640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1007fff640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 0x5617131550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:13.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.245+0000 7f1014881640 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 0x561713154650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1008005e30 tx=0x7f1008030450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e73d6b1f45b831e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f10067fc640 1 -- 172.21.15.83:0/2824453925 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008030a00 con 0x561712dad070 2024-09-21T03:20:13.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f10067fc640 1 -- 172.21.15.83:0/2824453925 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008036030 con 0x561712dad070 2024-09-21T03:20:13.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f10067fc640 1 -- 172.21.15.83:0/2824453925 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080044e0 con 0x561712dad070 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f100dffb640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 0x7f0ffc0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f100ffff640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 0x7f0ffc0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.246+0000 7f100dffb640 1 -- 172.21.15.83:0/3067284014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0ff80e75d0 con 0x7f10101083d0 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- 172.21.15.83:0/2824453925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 msgr2=0x561713154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f100ffff640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 0x7f0ffc0b8c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5617131532d0 tx=0x7f100000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 0x561713154650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1008005e30 tx=0x7f1008030450 comp rx=0 tx=0).stop 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- 172.21.15.83:0/2824453925 shutdown_connections 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d8c390 0x5617131550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d841b0 0x561713154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 --2- 172.21.15.83:0/2824453925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712dad070 0x561713154650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- 172.21.15.83:0/2824453925 >> 172.21.15.83:0/2824453925 conn(0x561712da1460 msgr2=0x561713152540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- 172.21.15.83:0/2824453925 shutdown_connections 2024-09-21T03:20:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- 172.21.15.83:0/2824453925 wait complete. 2024-09-21T03:20:13.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.248+0000 7f101c858740 1 -- start start 2024-09-21T03:20:13.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.249+0000 7f101c858740 1 Processor -- start 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d841b0 0x56171315fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712dad070 0x56171316c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561712f1b700 con 0x561712dad070 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561713055110 con 0x561712d841b0 2024-09-21T03:20:13.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561713162dd0 con 0x561712d8c390 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f101c858740 10 client.0 _ll_register_callbacks cb 0x56171306e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f1007fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f1007fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55540/0 (socket says 172.21.15.83:55540) 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f1007fff640 1 -- 172.21.15.83:0/1482990750 learned_addr learned my addr 172.21.15.83:0/1482990750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f100ffff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d841b0 0x56171315fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.250+0000 7f1014881640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712dad070 0x56171316c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 -- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d841b0 msgr2=0x56171315fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d841b0 0x56171315fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 -- 172.21.15.83:0/1482990750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712dad070 msgr2=0x56171316c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712dad070 0x56171316c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 -- 172.21.15.83:0/1482990750 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561713082c40 con 0x561712d8c390 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f1007fff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8007590 tx=0x7f0ff80b6df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd151d0895f920cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f0fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80acc10 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1008036030 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0ff8094be0 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ff80acfb0 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f101c858740 10 client.25636 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.251+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561713172ee0 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008036030 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f0ff80e70d0 con 0x561712d8c390 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 10 client.25636 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 20 client.25636 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d6b9f0 0x5617131763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 4 client.25636 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617130ccda0 con 0x561712d6b9f0 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f100ffff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d6b9f0 0x5617131763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.252+0000 7f0fef7fe640 10 client.25636 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f100ffff640 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d6b9f0 0x5617131763b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2fdf33ef0e809f server_cookie=4ee4fd66fbde4c4c in_seq=0 out_seq=0 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1482990750 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617130ccda0 con 0x561712d6b9f0 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f0fef7fe640 10 client.25636 handle_command_reply: tid=0 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f101c858740 1 client.25636 shutdown 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 msgr2=0x561713160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.253+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8007590 tx=0x7f0ff80b6df0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 shutdown_connections 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d6b9f0 0x5617131763b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712dad070 0x56171316c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712d8c390 0x561713160220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 --2- 172.21.15.83:0/1482990750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d841b0 0x56171315fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 >> 172.21.15.83:0/1482990750 conn(0x561712da1460 msgr2=0x561713157830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f0fef7fe640 0 client.25636 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 shutdown_connections 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 1 -- 172.21.15.83:0/1482990750 wait complete. 2024-09-21T03:20:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.254+0000 7f101c858740 20 client.25636 trim_cache size 0 max 16384 2024-09-21T03:20:13.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.286+0000 7f101c858740 1 Processor -- start 2024-09-21T03:20:13.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.287+0000 7f101c858740 1 -- start start 2024-09-21T03:20:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.287+0000 7f101c858740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f101c858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x56171316bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f101c858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171315ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f101c858740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ad220 con 0x561712d841b0 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f101c858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10101aedc0 con 0x561712d8c390 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f101c858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ad8f0 con 0x561712dad070 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47764/0 (socket says 172.21.15.83:47764) 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 -- 172.21.15.83:0/2050403067 learned_addr learned my addr 172.21.15.83:0/2050403067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f1007fff640 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x56171316bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f1014881640 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171315ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 -- 172.21.15.83:0/2050403067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 msgr2=0x56171315ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171315ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 -- 172.21.15.83:0/2050403067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 msgr2=0x56171316bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x56171316bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.288+0000 7f100ffff640 1 -- 172.21.15.83:0/2050403067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561713160f20 con 0x561712d841b0 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.289+0000 7f100ffff640 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f100000bfd0 tx=0x7f1000019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f327a71d98dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.289+0000 7f1005ffb640 1 -- 172.21.15.83:0/2050403067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000033a30 con 0x561712d841b0 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.289+0000 7f1005ffb640 1 -- 172.21.15.83:0/2050403067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100001ee20 con 0x561712d841b0 2024-09-21T03:20:13.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.289+0000 7f1005ffb640 1 -- 172.21.15.83:0/2050403067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000005530 con 0x561712d841b0 2024-09-21T03:20:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.290+0000 7f101c858740 1 -- 172.21.15.83:0/2050403067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 msgr2=0x5617131601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.290+0000 7f101c858740 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f100000bfd0 tx=0x7f1000019000 comp rx=0 tx=0).stop 2024-09-21T03:20:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 -- 172.21.15.83:0/2050403067 shutdown_connections 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171315ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x56171316bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 --2- 172.21.15.83:0/2050403067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x5617131601a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 -- 172.21.15.83:0/2050403067 >> 172.21.15.83:0/2050403067 conn(0x561712da1460 msgr2=0x56171315fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 -- 172.21.15.83:0/2050403067 shutdown_connections 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 -- 172.21.15.83:0/2050403067 wait complete. 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 -- start start 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.291+0000 7f101c858740 1 Processor -- start 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x561713168310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x5617131688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171316e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ae6c0 con 0x561712d841b0 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10101adff0 con 0x561712d8c390 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10101ab200 con 0x561712dad070 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.292+0000 7f101c858740 10 client.0 _ll_register_callbacks cb 0x7f0ff800fc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1014881640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171316e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1014881640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171316e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55552/0 (socket says 172.21.15.83:55552) 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1014881640 1 -- 172.21.15.83:0/1104163253 learned_addr learned my addr 172.21.15.83:0/1104163253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x5617131688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f100ffff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x561713168310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 -- 172.21.15.83:0/1104163253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 msgr2=0x56171316e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171316e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 -- 172.21.15.83:0/1104163253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 msgr2=0x561713168310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x561713168310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.293+0000 7f1007fff640 1 -- 172.21.15.83:0/1104163253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171316e7d0 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f1007fff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x5617131688d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800fcf0 tx=0x7f0ff80acc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fc665b79572299 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f0fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80b1f80 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0ff80a7db0 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100001ee20 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.294+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ff80b6ec0 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 10 client.25661 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561713164cb0 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff80c8af0 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f0ff80ac170 con 0x561712d8c390 2024-09-21T03:20:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 10 client.25661 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 20 client.25661 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d40da0 0x5617131df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 4 client.25661 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f100001ee20 con 0x561712d40da0 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.295+0000 7f100ffff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d40da0 0x5617131df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.296+0000 7f0fef7fe640 10 client.25661 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.296+0000 7f100ffff640 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d40da0 0x5617131df160 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc1ac0fa9090363 server_cookie=7c094df81d987acf in_seq=0 out_seq=0 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.296+0000 7f0fef7fe640 1 -- 172.21.15.83:0/1104163253 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x7f100001ee20 con 0x561712d40da0 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.296+0000 7f0fef7fe640 10 client.25661 handle_command_reply: tid=0 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.296+0000 7f101c858740 1 client.25661 shutdown 2024-09-21T03:20:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 msgr2=0x5617131688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x5617131688d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800fcf0 tx=0x7f0ff80acc10 comp rx=0 tx=0).stop 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 shutdown_connections 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561712d40da0 0x5617131df160 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561712dad070 0x56171316e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561712d8c390 0x5617131688d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 --2- 172.21.15.83:0/1104163253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561712d841b0 0x561713168310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 >> 172.21.15.83:0/1104163253 conn(0x561712da1460 msgr2=0x561713155630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f0fef7fe640 0 client.25661 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 shutdown_connections 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 1 -- 172.21.15.83:0/1104163253 wait complete. 2024-09-21T03:20:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.297+0000 7f101c858740 20 client.25661 trim_cache size 0 max 16384 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "90c69917-6ace-4c38-8d33-f0da1deff7a8", 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:20:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:20:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:20:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:20:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 msgr2=0x7f0ffc0b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 0x7f0ffc0b8c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5617131532d0 tx=0x7f100000b390 comp rx=0 tx=0).stop 2024-09-21T03:20:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 msgr2=0x7f101019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c840 tx=0x7f0ff800cd00 comp rx=0 tx=0).stop 2024-09-21T03:20:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 shutdown_connections 2024-09-21T03:20:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0ffc0b6760 0x7f0ffc0b8c70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010108d20 0x7f101019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10101083d0 0x7f101019aa10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 --2- 172.21.15.83:0/3067284014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010102ae0 0x7f101019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.298+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 >> 172.21.15.83:0/3067284014 conn(0x7f10100797b0 msgr2=0x7f1010106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.299+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 shutdown_connections 2024-09-21T03:20:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:13.299+0000 7f10162e4640 1 -- 172.21.15.83:0/3067284014 wait complete. 2024-09-21T03:20:13.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:13 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:13.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:13 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:13 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:20:13.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:13 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:13 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:13 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:13.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:13 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:13 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:20:13.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:13 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:13 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:13 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:13 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:13 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:20:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:13 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:13 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:20:14.142 INFO:tasks.cephfs.filesystem:scrub status for tag:90c69917-6ace-4c38-8d33-f0da1deff7a8 - {'path': '/', 'tag': '90c69917-6ace-4c38-8d33-f0da1deff7a8', 'options': 'recursive,force'} 2024-09-21T03:20:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:14 smithi083 ceph-mon[31219]: pgmap v432: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-09-21T03:20:14.681 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:14 smithi142 ceph-mon[34512]: pgmap v432: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-09-21T03:20:14.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:14 smithi193 ceph-mon[35224]: pgmap v432: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 37 MiB/s wr, 1.51k op/s 2024-09-21T03:20:15.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:15 smithi083 ceph-mon[31219]: pgmap v433: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-09-21T03:20:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:15 smithi142 ceph-mon[34512]: pgmap v433: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-09-21T03:20:15.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:15 smithi193 ceph-mon[35224]: pgmap v433: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-09-21T03:20:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 175.0 s, 3122.6 tps, lat 32.080 ms stddev 39.353 2024-09-21T03:20:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:17 smithi083 ceph-mon[31219]: pgmap v434: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-09-21T03:20:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:17 smithi142 ceph-mon[34512]: pgmap v434: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-09-21T03:20:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:17 smithi193 ceph-mon[35224]: pgmap v434: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-09-21T03:20:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:18 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:20:18.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:18 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:20:18.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:18 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:20:18.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:18 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:20:18.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:18 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:20:18.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:18 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:20:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:19 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:20:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:19 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:20:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:19 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:20:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:19 smithi142 ceph-mon[34512]: pgmap v435: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 951 op/s 2024-09-21T03:20:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:19 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:20:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:19 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:20:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:19 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:20:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:19 smithi193 ceph-mon[35224]: pgmap v435: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 951 op/s 2024-09-21T03:20:19.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:19 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:20:19.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:19 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:20:19.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:19 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:20:19.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:19 smithi083 ceph-mon[31219]: pgmap v435: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 951 op/s 2024-09-21T03:20:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 180.0 s, 2878.8 tps, lat 34.542 ms stddev 35.992 2024-09-21T03:20:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:21 smithi083 ceph-mon[31219]: pgmap v436: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-09-21T03:20:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:21 smithi142 ceph-mon[34512]: pgmap v436: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-09-21T03:20:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:21 smithi193 ceph-mon[35224]: pgmap v436: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-09-21T03:20:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:23 smithi083 ceph-mon[31219]: pgmap v437: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 804 op/s 2024-09-21T03:20:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:23 smithi142 ceph-mon[34512]: pgmap v437: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 804 op/s 2024-09-21T03:20:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:23 smithi193 ceph-mon[35224]: pgmap v437: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 804 op/s 2024-09-21T03:20:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:25 smithi083 ceph-mon[31219]: pgmap v438: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 735 op/s 2024-09-21T03:20:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:25 smithi142 ceph-mon[34512]: pgmap v438: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 735 op/s 2024-09-21T03:20:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:25 smithi193 ceph-mon[35224]: pgmap v438: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 735 op/s 2024-09-21T03:20:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 185.0 s, 2749.4 tps, lat 36.596 ms stddev 40.747 2024-09-21T03:20:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:27 smithi083 ceph-mon[31219]: pgmap v439: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 720 op/s 2024-09-21T03:20:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:27 smithi142 ceph-mon[34512]: pgmap v439: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 720 op/s 2024-09-21T03:20:27.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:27 smithi193 ceph-mon[35224]: pgmap v439: 97 pgs: 97 active+clean; 2.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 MiB/s wr, 720 op/s 2024-09-21T03:20:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:29 smithi083 ceph-mon[31219]: pgmap v440: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-21T03:20:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:29 smithi142 ceph-mon[34512]: pgmap v440: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-21T03:20:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:29 smithi193 ceph-mon[35224]: pgmap v440: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-21T03:20:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 190.0 s, 3271.8 tps, lat 30.598 ms stddev 38.536 2024-09-21T03:20:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:31 smithi083 ceph-mon[31219]: pgmap v441: 97 pgs: 97 active+clean; 2.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 810 op/s 2024-09-21T03:20:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:31 smithi142 ceph-mon[34512]: pgmap v441: 97 pgs: 97 active+clean; 2.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 810 op/s 2024-09-21T03:20:31.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:31 smithi193 ceph-mon[35224]: pgmap v441: 97 pgs: 97 active+clean; 2.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 810 op/s 2024-09-21T03:20:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:33 smithi142 ceph-mon[34512]: pgmap v442: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 975 op/s 2024-09-21T03:20:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:33 smithi193 ceph-mon[35224]: pgmap v442: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 975 op/s 2024-09-21T03:20:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:33 smithi083 ceph-mon[31219]: pgmap v442: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 975 op/s 2024-09-21T03:20:35.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:35 smithi142 ceph-mon[34512]: pgmap v443: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 843 op/s 2024-09-21T03:20:35.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:35 smithi193 ceph-mon[35224]: pgmap v443: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 843 op/s 2024-09-21T03:20:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:35 smithi083 ceph-mon[31219]: pgmap v443: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 34 MiB/s wr, 843 op/s 2024-09-21T03:20:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 195.0 s, 3647.4 tps, lat 27.233 ms stddev 32.549 2024-09-21T03:20:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:37 smithi142 ceph-mon[34512]: pgmap v444: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 876 op/s 2024-09-21T03:20:37.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:37 smithi193 ceph-mon[35224]: pgmap v444: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 876 op/s 2024-09-21T03:20:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:37 smithi083 ceph-mon[31219]: pgmap v444: 97 pgs: 97 active+clean; 2.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 876 op/s 2024-09-21T03:20:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:39 smithi142 ceph-mon[34512]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 51 MiB/s wr, 990 op/s 2024-09-21T03:20:39.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:39 smithi193 ceph-mon[35224]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 51 MiB/s wr, 990 op/s 2024-09-21T03:20:39.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:39 smithi083 ceph-mon[31219]: pgmap v445: 97 pgs: 97 active+clean; 2.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 51 MiB/s wr, 990 op/s 2024-09-21T03:20:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 200.0 s, 3682.0 tps, lat 27.285 ms stddev 31.559 2024-09-21T03:20:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:41 smithi142 ceph-mon[34512]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 754 op/s 2024-09-21T03:20:41.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:41 smithi193 ceph-mon[35224]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 754 op/s 2024-09-21T03:20:41.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:41 smithi083 ceph-mon[31219]: pgmap v446: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 754 op/s 2024-09-21T03:20:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:43 smithi142 ceph-mon[34512]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 880 op/s 2024-09-21T03:20:43.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:43 smithi193 ceph-mon[35224]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 880 op/s 2024-09-21T03:20:43.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:43 smithi083 ceph-mon[31219]: pgmap v447: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 880 op/s 2024-09-21T03:20:44.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:20:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:45.225 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:20:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:20:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:20:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:20:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:20:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:20:45.229 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:20:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 -- 172.21.15.83:0/2186043133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 msgr2=0x7fd72010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 --2- 172.21.15.83:0/2186043133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 0x7fd72010f530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd718009ca0 tx=0x7fd71802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 -- 172.21.15.83:0/2186043133 shutdown_connections 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 --2- 172.21.15.83:0/2186043133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 0x7fd72010f530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 --2- 172.21.15.83:0/2186043133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd7201087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 --2- 172.21.15.83:0/2186043133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd720102a70 0x7fd720102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 -- 172.21.15.83:0/2186043133 >> 172.21.15.83:0/2186043133 conn(0x7fd720079730 msgr2=0x7fd720079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 -- 172.21.15.83:0/2186043133 shutdown_connections 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.141+0000 7fd726985640 1 -- 172.21.15.83:0/2186043133 wait complete. 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.142+0000 7fd726985640 1 Processor -- start 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.142+0000 7fd726985640 1 -- start start 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd720102a70 0x7fd72010c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 0x7fd72010d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd720112970 con 0x7fd7201083b0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201127f0 con 0x7fd720108d00 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd726985640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd720112af0 con 0x7fd720102a70 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.143+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34010/0 (socket says 172.21.15.83:34010) 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 -- 172.21.15.83:0/2859362085 learned_addr learned my addr 172.21.15.83:0/2859362085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 -- 172.21.15.83:0/2859362085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd720102a70 msgr2=0x7fd72010c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd720102a70 0x7fd72010c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 -- 172.21.15.83:0/2859362085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 msgr2=0x7fd72010d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.144+0000 7fd7177fe640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 0x7fd72010d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.145+0000 7fd7177fe640 1 -- 172.21.15.83:0/2859362085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7201ac8b0 con 0x7fd7201083b0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.145+0000 7fd7177fe640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd71000e9c0 tx=0x7fd71000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.145+0000 7fd71dffb640 1 -- 172.21.15.83:0/2859362085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71000cd80 con 0x7fd7201083b0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.146+0000 7fd71dffb640 1 -- 172.21.15.83:0/2859362085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7100054d0 con 0x7fd7201083b0 2024-09-21T03:20:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.146+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7201acba0 con 0x7fd7201083b0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.146+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201ad080 con 0x7fd7201083b0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.146+0000 7fd71dffb640 1 -- 172.21.15.83:0/2859362085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710010430 con 0x7fd7201083b0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 Processor -- start 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 -- start start 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 0x55a345fd9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fd9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a345f4fe50 con 0x55a345c091b0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.150+0000 7fd72cef9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a345f3f280 con 0x55a345c11390 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd72cef9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a345f43090 con 0x55a345c32070 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71dffb640 1 -- 172.21.15.83:0/2859362085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd7100106b0 con 0x7fd7201083b0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd724f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 0x55a345fd9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34020/0 (socket says 172.21.15.83:34020) 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd724f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 0x55a345fd9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:33250/0 (socket says 172.21.15.83:33250) 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 -- 172.21.15.83:0/1848806454 learned_addr learned my addr 172.21.15.83:0/1848806454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd7177fe640 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fd9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 -- 172.21.15.83:0/1848806454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 msgr2=0x55a345fd9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 0x55a345fd9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 -- 172.21.15.83:0/1848806454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 msgr2=0x55a345fd9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fd9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd71ffff640 1 -- 172.21.15.83:0/1848806454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a345c9d650 con 0x55a345c091b0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.151+0000 7fd7177fe640 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fd9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.152+0000 7fd71ffff640 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd708009be0 tx=0x7fd70800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d730c592295274c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.152+0000 7fd7167fc640 1 -- 172.21.15.83:0/1848806454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708005020 con 0x55a345c091b0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.152+0000 7fd7167fc640 1 -- 172.21.15.83:0/1848806454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7080047a0 con 0x55a345c091b0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.152+0000 7fd7167fc640 1 -- 172.21.15.83:0/1848806454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708010690 con 0x55a345c091b0 2024-09-21T03:20:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.152+0000 7fd71dffb640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 0x7fd70c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.153+0000 7fd71dffb640 1 -- 172.21.15.83:0/2859362085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd7100e7ba0 con 0x7fd7201083b0 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.153+0000 7fd724f22640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 0x7fd70c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.153+0000 7fd72cef9740 1 -- 172.21.15.83:0/1848806454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 msgr2=0x55a345fd97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.153+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd708009be0 tx=0x7fd70800e720 comp rx=0 tx=0).stop 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 -- 172.21.15.83:0/1848806454 shutdown_connections 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fd9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c091b0 0x55a345fd97b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1848806454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c32070 0x55a345fd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 -- 172.21.15.83:0/1848806454 >> 172.21.15.83:0/1848806454 conn(0x55a345c26460 msgr2=0x55a345fd7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 -- 172.21.15.83:0/1848806454 shutdown_connections 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd72cef9740 1 -- 172.21.15.83:0/1848806454 wait complete. 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.156+0000 7fd724f22640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 0x7fd70c0b8d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd718009620 tx=0x7fd71802f310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.157+0000 7fd72cef9740 1 -- start start 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.157+0000 7fd72cef9740 1 Processor -- start 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.157+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.157+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.157+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c32070 0x55a345fe52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd72cef9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a345da0700 con 0x55a345c32070 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd72cef9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a345eda110 con 0x55a345c11390 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd72cef9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a345ff8a60 con 0x55a345c091b0 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38490/0 (socket says 172.21.15.83:38490) 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 -- 172.21.15.83:0/1755127538 learned_addr learned my addr 172.21.15.83:0/1755127538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd72cef9740 10 client.0 _ll_register_callbacks cb 0x55a345ef3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 -- 172.21.15.83:0/1755127538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 msgr2=0x55a345fe4830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:20:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 -- 172.21.15.83:0/1755127538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c32070 msgr2=0x55a345fe52b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c32070 0x55a345fe52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.158+0000 7fd7177fe640 1 -- 172.21.15.83:0/1755127538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a345f07c40 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd7177fe640 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7100a9fe0 tx=0x7fd7100a9920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8050c3a66e7334 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd6ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100b2020 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd7100b21c0 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7080047a0 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd7100ad180 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd72cef9740 10 client.25667 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.159+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a346015650 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7080047a0 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd7100c3bc0 con 0x55a345c11390 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 10 client.25667 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 20 client.25667 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345bf09f0 0x55a346018c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 4 client.25667 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a345f51da0 con 0x55a345bf09f0 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.160+0000 7fd71ffff640 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345bf09f0 0x55a346018c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.161+0000 7fd6ff7fe640 10 client.25667 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.161+0000 7fd71ffff640 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345bf09f0 0x55a346018c50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57619721b28d760f server_cookie=606db509035b9ac2 in_seq=0 out_seq=0 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/1755127538 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a345f51da0 con 0x55a345bf09f0 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd6ff7fe640 10 client.25667 handle_command_reply: tid=0 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 client.25667 shutdown 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 msgr2=0x55a345fe4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7100a9fe0 tx=0x7fd7100a9920 comp rx=0 tx=0).stop 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 shutdown_connections 2024-09-21T03:20:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345bf09f0 0x55a346018c50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c32070 0x55a345fe52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd6ff7fe640 0 client.25667 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345fe4d70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.163+0000 7fd72cef9740 1 --2- 172.21.15.83:0/1755127538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.164+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 >> 172.21.15.83:0/1755127538 conn(0x55a345c26460 msgr2=0x55a345fdc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.164+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 shutdown_connections 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.164+0000 7fd72cef9740 1 -- 172.21.15.83:0/1755127538 wait complete. 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.164+0000 7fd72cef9740 20 client.25667 trim_cache size 0 max 16384 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.191+0000 7fd72cef9740 1 Processor -- start 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.191+0000 7fd72cef9740 1 -- start start 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.191+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345bf09f0 0x55a345fdac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.191+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.191+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd72cef9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006f160 con 0x55a345c11390 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd72cef9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006cef0 con 0x55a345bf09f0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34034/0 (socket says 172.21.15.83:34034) 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 -- 172.21.15.83:0/3290389233 learned_addr learned my addr 172.21.15.83:0/3290389233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd7177fe640 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006ea90 con 0x55a345c091b0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd71ffff640 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345bf09f0 0x55a345fdac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 -- 172.21.15.83:0/3290389233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 msgr2=0x55a345fe6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 -- 172.21.15.83:0/3290389233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345bf09f0 msgr2=0x55a345fdac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345bf09f0 0x55a345fdac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.192+0000 7fd724f22640 1 -- 172.21.15.83:0/3290389233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a345fd13f0 con 0x55a345c11390 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.193+0000 7fd724f22640 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd718009ca0 tx=0x7fd71803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4a50df9ee184a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.194+0000 7fd715ffb640 1 -- 172.21.15.83:0/3290389233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71804e070 con 0x55a345c11390 2024-09-21T03:20:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.194+0000 7fd715ffb640 1 -- 172.21.15.83:0/3290389233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd718036040 con 0x55a345c11390 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.194+0000 7fd715ffb640 1 -- 172.21.15.83:0/3290389233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718030b00 con 0x55a345c11390 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.195+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 msgr2=0x55a345fd6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.195+0000 7fd72cef9740 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd718009ca0 tx=0x7fd71803e040 comp rx=0 tx=0).stop 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.195+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 shutdown_connections 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.195+0000 7fd72cef9740 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345c11390 0x55a345fd6710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.195+0000 7fd72cef9740 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345fe6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.196+0000 7fd72cef9740 1 --2- 172.21.15.83:0/3290389233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345bf09f0 0x55a345fdac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.196+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 >> 172.21.15.83:0/3290389233 conn(0x55a345c26460 msgr2=0x55a345fe5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.196+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 shutdown_connections 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.196+0000 7fd72cef9740 1 -- 172.21.15.83:0/3290389233 wait complete. 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.196+0000 7fd72cef9740 1 -- start start 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.197+0000 7fd72cef9740 1 Processor -- start 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345ffa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345ffa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd71ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34044/0 (socket says 172.21.15.83:34044) 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006dcc0 con 0x55a345bf09f0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd71ffff640 1 -- 172.21.15.83:0/459461485 learned_addr learned my addr 172.21.15.83:0/459461485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006d5c0 con 0x55a345c11390 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd72006e390 con 0x55a345c091b0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd72cef9740 10 client.0 _ll_register_callbacks cb 0x7fd7100a9eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd724f22640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345ffa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.198+0000 7fd7177fe640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345ffa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 -- 172.21.15.83:0/459461485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 msgr2=0x55a345ffa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345ffa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 -- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 msgr2=0x55a345ffa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345ffa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a346082610 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd71ffff640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd708009250 tx=0x7fd708005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d1bf7a8fa072ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.199+0000 7fd6ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.200+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70801d040 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.200+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd708004040 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.200+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd718036040 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.200+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd708004240 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 10 client.16146 fetch_fsmap learned FSMap version 36 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a346083870 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd718036040 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd70802b590 con 0x55a345bf09f0 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 10 client.16146 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T03:20:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 20 client.16146 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345c32070 0x55a346085c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 4 client.16146 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.201+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a345f4d850 con 0x55a345c32070 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.202+0000 7fd7177fe640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345c32070 0x55a346085c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.202+0000 7fd6ff7fe640 10 client.16146 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.203+0000 7fd7177fe640 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345c32070 0x55a346085c60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ff96192fb43f5e server_cookie=49bff7138cedb862 in_seq=0 out_seq=0 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.203+0000 7fd6ff7fe640 1 -- 172.21.15.83:0/459461485 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55a345f4d850 con 0x55a345c32070 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.203+0000 7fd6ff7fe640 10 client.16146 handle_command_reply: tid=0 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.204+0000 7fd72cef9740 1 client.16146 shutdown 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 msgr2=0x55a345ff9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd708009250 tx=0x7fd708005d80 comp rx=0 tx=0).stop 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 shutdown_connections 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a345c32070 0x55a346085c60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a345c11390 0x55a345ffa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd6ff7fe640 0 client.16146 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a345c091b0 0x55a345ffa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 --2- 172.21.15.83:0/459461485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a345bf09f0 0x55a345ff9c50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 >> 172.21.15.83:0/459461485 conn(0x55a345c26460 msgr2=0x55a345fdaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 shutdown_connections 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.205+0000 7fd72cef9740 1 -- 172.21.15.83:0/459461485 wait complete. 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.206+0000 7fd72cef9740 20 client.16146 trim_cache size 0 max 16384 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 msgr2=0x7fd70c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 0x7fd70c0b8d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd718009620 tx=0x7fd71802f310 comp rx=0 tx=0).stop 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 msgr2=0x7fd72010cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd71000e9c0 tx=0x7fd71000ee80 comp rx=0 tx=0).stop 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 shutdown_connections 2024-09-21T03:20:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd70c0b6850 0x7fd70c0b8d60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd720108d00 0x7fd72010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7201083b0 0x7fd72010cb20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 --2- 172.21.15.83:0/2859362085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd720102a70 0x7fd72010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 >> 172.21.15.83:0/2859362085 conn(0x7fd720079730 msgr2=0x7fd720114160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 shutdown_connections 2024-09-21T03:20:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:45.207+0000 7fd726985640 1 -- 172.21.15.83:0/2859362085 wait complete. 2024-09-21T03:20:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:45 smithi083 ceph-mon[31219]: pgmap v448: 97 pgs: 97 active+clean; 2.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 743 op/s 2024-09-21T03:20:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:45 smithi142 ceph-mon[34512]: pgmap v448: 97 pgs: 97 active+clean; 2.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 743 op/s 2024-09-21T03:20:45.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:45 smithi193 ceph-mon[35224]: pgmap v448: 97 pgs: 97 active+clean; 2.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 743 op/s 2024-09-21T03:20:45.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 205.0 s, 3281.9 tps, lat 26.202 ms stddev 32.304 2024-09-21T03:20:46.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:20:46.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:20:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:47 smithi083 ceph-mon[31219]: pgmap v449: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 911 op/s 2024-09-21T03:20:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:47 smithi142 ceph-mon[34512]: pgmap v449: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 911 op/s 2024-09-21T03:20:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:47 smithi193 ceph-mon[35224]: pgmap v449: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 MiB/s wr, 911 op/s 2024-09-21T03:20:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 -- 172.21.15.83:0/1812867007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468101f70 msgr2=0x7f7468102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1812867007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468101f70 0x7f7468102380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7458009860 tx=0x7f745802a340 comp rx=0 tx=0).stop 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 -- 172.21.15.83:0/1812867007 shutdown_connections 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1812867007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468109e00 0x7f7468110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1812867007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74681094b0 0x7f74681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1812867007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468101f70 0x7f7468102380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 -- 172.21.15.83:0/1812867007 >> 172.21.15.83:0/1812867007 conn(0x7f7468077c60 msgr2=0x7f74680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 -- 172.21.15.83:0/1812867007 shutdown_connections 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.099+0000 7f746e1b3640 1 -- 172.21.15.83:0/1812867007 wait complete. 2024-09-21T03:20:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.100+0000 7f746e1b3640 1 Processor -- start 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.100+0000 7f746e1b3640 1 -- start start 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 0x7f7468104920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74681094b0 0x7f7468102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113ac0 con 0x7f74681094b0 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113940 con 0x7f7468101f70 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f746e1b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7468113c40 con 0x7f7468109e00 2024-09-21T03:20:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f74677fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 0x7f7468104920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f74677fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 0x7f7468104920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38532/0 (socket says 172.21.15.83:38532) 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:33282/0 (socket says 172.21.15.83:33282) 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f74677fe640 1 -- 172.21.15.83:0/1404807766 learned_addr learned my addr 172.21.15.83:0/1404807766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 -- 172.21.15.83:0/1404807766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 msgr2=0x7f7468104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 0x7f7468104920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 -- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74681094b0 msgr2=0x7f7468102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74681094b0 0x7f7468102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 -- 172.21.15.83:0/1404807766 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7468103c70 con 0x7f7468109e00 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.101+0000 7f7467fff640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f746000e940 tx=0x7f746000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.102+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460004d20 con 0x7f7468109e00 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.102+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7460005020 con 0x7f7468109e00 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.102+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681aced0 con 0x7f7468109e00 2024-09-21T03:20:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.102+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74600116b0 con 0x7f7468109e00 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.102+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681ad380 con 0x7f7468109e00 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.103+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7468102380 con 0x7f7468109e00 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.106+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f746000cb20 con 0x7f7468109e00 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.107+0000 7f7464ff9640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 0x7f74500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.107+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f74600e76d0 con 0x7f7468109e00 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.107+0000 7f74677fe640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 0x7f74500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.108+0000 7f74677fe640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 0x7f74500b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74580057c0 tx=0x7f7458030340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.112+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7460016030 con 0x7f7468109e00 2024-09-21T03:20:48.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.469+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f746802e640 con 0x7f7468109e00 2024-09-21T03:20:48.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.472+0000 7f7464ff9640 1 -- 172.21.15.83:0/1404807766 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11984 (secure 0 0 0) 0x7f74600adca0 con 0x7f7468109e00 2024-09-21T03:20:48.474 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:20:48.474 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"btime":"2024-09-21T03:20:11:307413+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:20:11.257166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:20:48.474 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 msgr2=0x7f74500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 0x7f74500b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74580057c0 tx=0x7f7458030340 comp rx=0 tx=0).stop 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 msgr2=0x7f7468103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f746000e940 tx=0x7f746000ee00 comp rx=0 tx=0).stop 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 shutdown_connections 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f74500b6700 0x7f74500b8c10 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74580057c0 tx=0x7f7458030340 comp rx=0 tx=0).stop 2024-09-21T03:20:48.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7468109e00 0x7f7468103450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74681094b0 0x7f7468102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.475+0000 7f746e1b3640 1 --2- 172.21.15.83:0/1404807766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7468101f70 0x7f7468104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.476+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 >> 172.21.15.83:0/1404807766 conn(0x7f7468077c60 msgr2=0x7f74681075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.477+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 shutdown_connections 2024-09-21T03:20:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:48.477+0000 7f746e1b3640 1 -- 172.21.15.83:0/1404807766 wait complete. 2024-09-21T03:20:48.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:48 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1404807766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:20:48.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:48 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1404807766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:20:48.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:48 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1404807766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:20:49.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:20:49.684 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:49.812 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:49 smithi083 ceph-mon[31219]: pgmap v450: 97 pgs: 97 active+clean; 3.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 60 MiB/s wr, 1.74k op/s 2024-09-21T03:20:49.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:49 smithi193 ceph-mon[35224]: pgmap v450: 97 pgs: 97 active+clean; 3.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 60 MiB/s wr, 1.74k op/s 2024-09-21T03:20:49.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:49 smithi142 ceph-mon[34512]: pgmap v450: 97 pgs: 97 active+clean; 3.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 60 MiB/s wr, 1.74k op/s 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 -- 172.21.15.83:0/431794130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a50d0 msgr2=0x7f2c640a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 --2- 172.21.15.83:0/431794130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a50d0 0x7f2c640a5bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c6002a420 comp rx=0 tx=0).stop 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 -- 172.21.15.83:0/431794130 shutdown_connections 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 --2- 172.21.15.83:0/431794130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c640a6540 0x7f2c640a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 --2- 172.21.15.83:0/431794130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a6130 0x7f2c640a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 --2- 172.21.15.83:0/431794130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a50d0 0x7f2c640a5bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 -- 172.21.15.83:0/431794130 >> 172.21.15.83:0/431794130 conn(0x7f2c6401d090 msgr2=0x7f2c640a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 -- 172.21.15.83:0/431794130 shutdown_connections 2024-09-21T03:20:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.508+0000 7f2c717ae640 1 -- 172.21.15.83:0/431794130 wait complete. 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.509+0000 7f2c717ae640 1 Processor -- start 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 -- start start 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 0x7f2c6413db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c640a6130 0x7f2c6413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 0x7f2c641433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640b7120 con 0x7f2c640a50d0 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640b6fa0 con 0x7f2c640a6540 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c717ae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640b72a0 con 0x7f2c640a6130 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.510+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 0x7f2c6413db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 0x7f2c6413db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54368/0 (socket says 172.21.15.83:54368) 2024-09-21T03:20:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c6bfff640 1 -- 172.21.15.83:0/401455179 learned_addr learned my addr 172.21.15.83:0/401455179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 0x7f2c641433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 -- 172.21.15.83:0/401455179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c640a6130 msgr2=0x7f2c6413e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c640a6130 0x7f2c6413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 -- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 msgr2=0x7f2c6413db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 0x7f2c6413db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.511+0000 7f2c70fad640 1 -- 172.21.15.83:0/401455179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c64143b00 con 0x7f2c640a6540 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.512+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 0x7f2c641433d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000b60 tx=0x7f2c6c06a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.512+0000 7f2c697fa640 1 -- 172.21.15.83:0/401455179 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c072a80 con 0x7f2c640a6540 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.512+0000 7f2c697fa640 1 -- 172.21.15.83:0/401455179 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c6c072c20 con 0x7f2c640a6540 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.512+0000 7f2c697fa640 1 -- 172.21.15.83:0/401455179 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0776f0 con 0x7f2c640a6540 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.513+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c64143df0 con 0x7f2c640a6540 2024-09-21T03:20:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.513+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6414a550 con 0x7f2c640a6540 2024-09-21T03:20:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.516+0000 7f2c78f84740 1 Processor -- start 2024-09-21T03:20:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 -- start start 2024-09-21T03:20:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d712fc1b0 0x564d716cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 0x564d716ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d71642ea0 con 0x564d71325070 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564d716322d0 con 0x564d712fc1b0 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c78f84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564d716360e0 con 0x564d71304390 2024-09-21T03:20:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.517+0000 7f2c697fa640 1 -- 172.21.15.83:0/401455179 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2c6c07f030 con 0x7f2c640a6540 2024-09-21T03:20:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.518+0000 7f2c697fa640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 0x7f2c5c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 0x7f2c5c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 0x564d716ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54382/0 (socket says 172.21.15.83:54382) 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/2809978928 learned_addr learned my addr 172.21.15.83:0/2809978928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d712fc1b0 0x564d716cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/2809978928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d712fc1b0 msgr2=0x564d716cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d712fc1b0 0x564d716cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/2809978928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 msgr2=0x564d716cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/2809978928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d71390650 con 0x564d71304390 2024-09-21T03:20:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.519+0000 7f2c697fa640 1 -- 172.21.15.83:0/401455179 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2c6c14b670 con 0x7f2c640a6540 2024-09-21T03:20:50.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.520+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 0x564d716ccee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c58012900 tx=0x7f2c580090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f322e061a0ea8c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.520+0000 7f2c70fad640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 0x7f2c5c0b8d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c083000 tx=0x7f2c6c0de8d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.520+0000 7f2c567fc640 1 -- 172.21.15.83:0/2809978928 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004ca0 con 0x564d71304390 2024-09-21T03:20:50.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.520+0000 7f2c567fc640 1 -- 172.21.15.83:0/2809978928 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c58012e80 con 0x564d71304390 2024-09-21T03:20:50.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.520+0000 7f2c567fc640 1 -- 172.21.15.83:0/2809978928 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580066a0 con 0x564d71304390 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 -- 172.21.15.83:0/2809978928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 msgr2=0x564d716ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 0x564d716ccee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c58012900 tx=0x7f2c580090e0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 -- 172.21.15.83:0/2809978928 shutdown_connections 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71304390 0x564d716ccee0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d712fc1b0 0x564d716cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 --2- 172.21.15.83:0/2809978928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.521+0000 7f2c78f84740 1 -- 172.21.15.83:0/2809978928 >> 172.21.15.83:0/2809978928 conn(0x564d71319460 msgr2=0x564d716ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.522+0000 7f2c78f84740 1 -- 172.21.15.83:0/2809978928 shutdown_connections 2024-09-21T03:20:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.522+0000 7f2c78f84740 1 -- 172.21.15.83:0/2809978928 wait complete. 2024-09-21T03:20:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.523+0000 7f2c78f84740 1 -- start start 2024-09-21T03:20:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.523+0000 7f2c78f84740 1 Processor -- start 2024-09-21T03:20:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.523+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.523+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d71493700 con 0x564d71325070 2024-09-21T03:20:50.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564d715cd160 con 0x564d71304390 2024-09-21T03:20:50.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564d716e33d0 con 0x564d712fc1b0 2024-09-21T03:20:50.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44040/0 (socket says 172.21.15.83:44040) 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 -- 172.21.15.83:0/1429064910 learned_addr learned my addr 172.21.15.83:0/1429064910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 10 client.0 _ll_register_callbacks cb 0x564d715e69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 -- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 msgr2=0x564d716d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 -- 172.21.15.83:0/1429064910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 msgr2=0x564d716d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 -- 172.21.15.83:0/1429064910 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d715fac90 con 0x564d71304390 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d71644df0 con 0x564d71304390 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c60000c00 tx=0x7f2c600316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a909ec0af0c0a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.524+0000 7f2c4b7fe640 10 client.25691 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.525+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1429064910 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60035040 con 0x564d71304390 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.525+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1429064910 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2c60004250 con 0x564d71304390 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.525+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1429064910 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c60004490 con 0x564d71304390 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.525+0000 7f2c78f84740 10 client.25691 fetch_fsmap learned FSMap version 37 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.525+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=37}) -- 0x564d716d8610 con 0x564d71304390 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1429064910 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f2c600035f0 con 0x564d71304390 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 10 client.25691 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 10 client.25691 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 20 client.25691 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712e39f0 0x564d716dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 4 client.25691 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d716408a0 con 0x564d712e39f0 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.526+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712e39f0 0x564d716dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.527+0000 7f2c4b7fe640 10 client.25691 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:20:50.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.527+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712e39f0 0x564d716dbed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c8ad77c74672d2db server_cookie=100739c2d4a5cd76 in_seq=0 out_seq=0 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1429064910 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d716408a0 con 0x564d712e39f0 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c4b7fe640 10 client.25691 handle_command_reply: tid=0 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 client.25691 shutdown 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 msgr2=0x564d716d4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c60000c00 tx=0x7f2c600316d0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 shutdown_connections 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712e39f0 0x564d716dbed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d4020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1429064910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 >> 172.21.15.83:0/1429064910 conn(0x564d71319460 msgr2=0x564d716cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c4b7fe640 0 client.25691 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:20:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.528+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 shutdown_connections 2024-09-21T03:20:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.529+0000 7f2c78f84740 1 -- 172.21.15.83:0/1429064910 wait complete. 2024-09-21T03:20:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.529+0000 7f2c78f84740 20 client.25691 trim_cache size 0 max 16384 2024-09-21T03:20:50.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 Processor -- start 2024-09-21T03:20:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 -- start start 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d712fc1b0 0x564d716e16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64150a30 con 0x564d712fc1b0 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64152600 con 0x564d71304390 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.550+0000 7f2c78f84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64151100 con 0x564d71325070 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46102/0 (socket says 172.21.15.83:46102) 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/385884884 learned_addr learned my addr 172.21.15.83:0/385884884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d712fc1b0 0x564d716e16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/385884884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 msgr2=0x564d716d0cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d0cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/385884884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d712fc1b0 msgr2=0x564d716e16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d712fc1b0 0x564d716e16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/385884884 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d716e33f0 con 0x564d71325070 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.551+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800a230 tx=0x7f2c58006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04ea2b79401a12b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.553+0000 7f2c55ffb640 1 -- 172.21.15.83:0/385884884 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004ca0 con 0x564d71325070 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.553+0000 7f2c55ffb640 1 -- 172.21.15.83:0/385884884 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c58004e40 con 0x564d71325070 2024-09-21T03:20:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.553+0000 7f2c55ffb640 1 -- 172.21.15.83:0/385884884 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58010b40 con 0x564d71325070 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 -- 172.21.15.83:0/385884884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 msgr2=0x564d716d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800a230 tx=0x7f2c58006040 comp rx=0 tx=0).stop 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 -- 172.21.15.83:0/385884884 shutdown_connections 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d71325070 0x564d716d1210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 --2- 172.21.15.83:0/385884884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d712fc1b0 0x564d716e16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 -- 172.21.15.83:0/385884884 >> 172.21.15.83:0/385884884 conn(0x564d71319460 msgr2=0x564d716d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 -- 172.21.15.83:0/385884884 shutdown_connections 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.554+0000 7f2c78f84740 1 -- 172.21.15.83:0/385884884 wait complete. 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.555+0000 7f2c78f84740 1 -- start start 2024-09-21T03:20:50.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.555+0000 7f2c78f84740 1 Processor -- start 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64151f00 con 0x564d71325070 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64151800 con 0x564d71304390 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6414ea10 con 0x564d712fc1b0 2024-09-21T03:20:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46112/0 (socket says 172.21.15.83:46112) 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44044/0 (socket says 172.21.15.83:44044) 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/1800014471 learned_addr learned my addr 172.21.15.83:0/1800014471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c78f84740 10 client.0 _ll_register_callbacks cb 0x7f2c60009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/1800014471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 msgr2=0x564d716f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/1800014471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 msgr2=0x564d716f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.556+0000 7f2c6b7fe640 1 -- 172.21.15.83:0/1800014471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d717537d0 con 0x564d712fc1b0 2024-09-21T03:20:50.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.558+0000 7f2c6b7fe640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800aa50 tx=0x7f2c5800a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4654e960c7f92622 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.558+0000 7f2c4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:50.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004ca0 con 0x564d712fc1b0 2024-09-21T03:20:50.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2c580108c0 con 0x564d712fc1b0 2024-09-21T03:20:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d7163dcd0 con 0x564d712fc1b0 2024-09-21T03:20:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c58010ac0 con 0x564d712fc1b0 2024-09-21T03:20:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c78f84740 10 client.25672 fetch_fsmap learned FSMap version 37 2024-09-21T03:20:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.559+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d71754a30 con 0x564d712fc1b0 2024-09-21T03:20:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.560+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564d7163dcd0 con 0x564d712fc1b0 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f2c58015730 con 0x564d712fc1b0 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 10 client.25672 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 10 client.25672 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 20 client.25672 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712b8da0 0x564d71756dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 4 client.25672 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d715b3aa0 con 0x564d712b8da0 2024-09-21T03:20:50.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.561+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712b8da0 0x564d71756dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.562+0000 7f2c4b7fe640 10 client.25672 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:20:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.562+0000 7f2c6bfff640 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712b8da0 0x564d71756dc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4e283adb2dfe7a0 server_cookie=92eba6310e2b7f69 in_seq=0 out_seq=0 2024-09-21T03:20:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.562+0000 7f2c4b7fe640 1 -- 172.21.15.83:0/1800014471 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d715b3aa0 con 0x564d712b8da0 2024-09-21T03:20:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.562+0000 7f2c4b7fe640 10 client.25672 handle_command_reply: tid=0 2024-09-21T03:20:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.562+0000 7f2c78f84740 1 client.25672 shutdown 2024-09-21T03:20:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.563+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 msgr2=0x564d716f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.563+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800aa50 tx=0x7f2c5800a2f0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 shutdown_connections 2024-09-21T03:20:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x564d712b8da0 0x564d71756dc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d71325070 0x564d716f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c4b7fe640 0 client.25672 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:20:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d71304390 0x564d716f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 --2- 172.21.15.83:0/1800014471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d712fc1b0 0x564d716f15c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 >> 172.21.15.83:0/1800014471 conn(0x564d71319460 msgr2=0x564d716eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 shutdown_connections 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.565+0000 7f2c78f84740 1 -- 172.21.15.83:0/1800014471 wait complete. 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.566+0000 7f2c78f84740 20 client.25672 trim_cache size 0 max 16384 2024-09-21T03:20:50.568 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 msgr2=0x7f2c5c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 0x7f2c5c0b8d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c083000 tx=0x7f2c6c0de8d0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 msgr2=0x7f2c641433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 0x7f2c641433d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000b60 tx=0x7f2c6c06a4f0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 shutdown_connections 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2c5c0b6850 0x7f2c5c0b8d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c640a6540 0x7f2c641433d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c640a6130 0x7f2c6413e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 --2- 172.21.15.83:0/401455179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c640a50d0 0x7f2c6413db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 >> 172.21.15.83:0/401455179 conn(0x7f2c6401d090 msgr2=0x7f2c640a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 shutdown_connections 2024-09-21T03:20:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:50.567+0000 7f2c717ae640 1 -- 172.21.15.83:0/401455179 wait complete. 2024-09-21T03:20:50.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:50 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:50.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:50 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:50.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:50 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 210.0 s, 3423.3 tps, lat 33.224 ms stddev 69.864 2024-09-21T03:20:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:51 smithi193 ceph-mon[35224]: pgmap v451: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 51 MiB/s wr, 1.63k op/s 2024-09-21T03:20:51.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:51 smithi083 ceph-mon[31219]: pgmap v451: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 51 MiB/s wr, 1.63k op/s 2024-09-21T03:20:51.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:51 smithi142 ceph-mon[34512]: pgmap v451: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 51 MiB/s wr, 1.63k op/s 2024-09-21T03:20:52.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:20:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 -- 172.21.15.83:0/2327854387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10a590 msgr2=0x7f04bc10a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/2327854387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10a590 0x7f04bc10a9a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009900 tx=0x7f04b802a410 comp rx=0 tx=0).stop 2024-09-21T03:20:53.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 -- 172.21.15.83:0/2327854387 shutdown_connections 2024-09-21T03:20:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/2327854387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 0x7f04bc118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/2327854387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10aee0 0x7f04bc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/2327854387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10a590 0x7f04bc10a9a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.093+0000 7f04c3ff5640 1 -- 172.21.15.83:0/2327854387 >> 172.21.15.83:0/2327854387 conn(0x7f04bc0712f0 msgr2=0x7f04bc073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.094+0000 7f04c3ff5640 1 -- 172.21.15.83:0/2327854387 shutdown_connections 2024-09-21T03:20:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.094+0000 7f04c3ff5640 1 -- 172.21.15.83:0/2327854387 wait complete. 2024-09-21T03:20:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.095+0000 7f04c3ff5640 1 Processor -- start 2024-09-21T03:20:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.095+0000 7f04c3ff5640 1 -- start start 2024-09-21T03:20:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10aee0 0x7f04bc1a3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c1d91640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 0x7f04bc1a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c1d91640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44068/0 (socket says 172.21.15.83:44068) 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc11c2d0 con 0x7f04bc10aee0 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc11c150 con 0x7f04bc10a590 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c3ff5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc11c450 con 0x7f04bc116640 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c1590640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10aee0 0x7f04bc1a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c1d91640 1 -- 172.21.15.83:0/3438219953 learned_addr learned my addr 172.21.15.83:0/3438219953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.096+0000 7f04c2592640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 0x7f04bc1a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 -- 172.21.15.83:0/3438219953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 msgr2=0x7f04bc1a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 0x7f04bc1a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 -- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10aee0 msgr2=0x7f04bc1a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10aee0 0x7f04bc1a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 -- 172.21.15.83:0/3438219953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc1a8c30 con 0x7f04bc10a590 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c1d91640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f04b8004770 tx=0x7f04b80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04b2ffd640 1 -- 172.21.15.83:0/3438219953 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b803f030 con 0x7f04bc10a590 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1a8ec0 con 0x7f04bc10a590 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04b2ffd640 1 -- 172.21.15.83:0/3438219953 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b80044c0 con 0x7f04bc10a590 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04b2ffd640 1 -- 172.21.15.83:0/3438219953 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8035070 con 0x7f04bc10a590 2024-09-21T03:20:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.097+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc10bd60 con 0x7f04bc10a590 2024-09-21T03:20:53.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.103+0000 7f04b2ffd640 1 -- 172.21.15.83:0/3438219953 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f04b8042030 con 0x7f04bc10a590 2024-09-21T03:20:53.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.104+0000 7f04ca569740 1 Processor -- start 2024-09-21T03:20:53.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.104+0000 7f04ca569740 1 -- start start 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.104+0000 7f04ca569740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04ca569740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f11b0 0x55745f4c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04ca569740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04ca569740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55745f437ea0 con 0x55745f0f9390 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04ca569740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55745f4272d0 con 0x55745f0f11b0 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04ca569740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55745f42b0e0 con 0x55745f11a070 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54434/0 (socket says 172.21.15.83:54434) 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 -- 172.21.15.83:0/2146367697 learned_addr learned my addr 172.21.15.83:0/2146367697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 -- 172.21.15.83:0/2146367697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 msgr2=0x55745f4c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 -- 172.21.15.83:0/2146367697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f11b0 msgr2=0x55745f4c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f11b0 0x55745f4c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.105+0000 7f04c1590640 1 -- 172.21.15.83:0/2146367697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745f185650 con 0x55745f0f9390 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.106+0000 7f04c1590640 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009c20 tx=0x7f04ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6540ee2f80eb60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.107+0000 7f04a7fff640 1 -- 172.21.15.83:0/2146367697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004230 con 0x55745f0f9390 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.107+0000 7f04a7fff640 1 -- 172.21.15.83:0/2146367697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac0043d0 con 0x55745f0f9390 2024-09-21T03:20:53.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.107+0000 7f04a7fff640 1 -- 172.21.15.83:0/2146367697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004d10 con 0x55745f0f9390 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.108+0000 7f04ca569740 1 -- 172.21.15.83:0/2146367697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 msgr2=0x55745f4c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.108+0000 7f04ca569740 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009c20 tx=0x7f04ac011810 comp rx=0 tx=0).stop 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.110+0000 7f04ca569740 1 -- 172.21.15.83:0/2146367697 shutdown_connections 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.110+0000 7f04ca569740 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4c1d40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.110+0000 7f04ca569740 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f11b0 0x55745f4c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.110+0000 7f04ca569740 1 --2- 172.21.15.83:0/2146367697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.111+0000 7f04ca569740 1 -- 172.21.15.83:0/2146367697 >> 172.21.15.83:0/2146367697 conn(0x55745f10e460 msgr2=0x55745f4bf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.111+0000 7f04ca569740 1 -- 172.21.15.83:0/2146367697 shutdown_connections 2024-09-21T03:20:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.111+0000 7f04ca569740 1 -- 172.21.15.83:0/2146367697 wait complete. 2024-09-21T03:20:53.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.112+0000 7f04ca569740 1 -- start start 2024-09-21T03:20:53.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.112+0000 7f04ca569740 1 Processor -- start 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.113+0000 7f04ca569740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.113+0000 7f04b2ffd640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 0x7f04a80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.113+0000 7f04ca569740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 0x55745f4d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.113+0000 7f04ca569740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04b2ffd640 1 -- 172.21.15.83:0/3438219953 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04b8107c50 con 0x7f04bc10a590 2024-09-21T03:20:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04c2592640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04c1590640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 0x55745f4d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04c1590640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 0x55745f4d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46136/0 (socket says 172.21.15.83:46136) 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04c1590640 1 -- 172.21.15.83:0/3569594452 learned_addr learned my addr 172.21.15.83:0/3569594452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55745f288700 con 0x55745f0f11b0 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55745f3c2160 con 0x55745f11a070 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55745f4d81d0 con 0x55745f0f9390 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.114+0000 7f04c1d91640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 0x7f04a80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.115+0000 7f04ca569740 10 client.0 _ll_register_callbacks cb 0x55745f3db9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.116+0000 7f04c1d91640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 0x7f04a80b8b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55745f4bff40 tx=0x7f04b803d000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.117+0000 7f04c2592640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.118+0000 7f04c2592640 1 -- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 msgr2=0x55745f4d5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.118+0000 7f04c2592640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 0x55745f4d5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.118+0000 7f04c2592640 1 -- 172.21.15.83:0/3569594452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 msgr2=0x55745f4d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.118+0000 7f04c2592640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.118+0000 7f04c2592640 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745f3efc90 con 0x55745f11a070 2024-09-21T03:20:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.120+0000 7f04c2592640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4e1560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55745f4b94f0 tx=0x7f04b4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90089693078b1ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.121+0000 7f04a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.121+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4010e00 con 0x55745f11a070 2024-09-21T03:20:53.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.121+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04b4004b20 con 0x55745f11a070 2024-09-21T03:20:53.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.121+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac0043d0 con 0x55745f11a070 2024-09-21T03:20:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.124+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f04b4024050 con 0x55745f11a070 2024-09-21T03:20:53.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.127+0000 7f04ca569740 10 client.25703 fetch_fsmap learned FSMap version 37 2024-09-21T03:20:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.127+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55745f4e4020 con 0x55745f11a070 2024-09-21T03:20:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.127+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4003e20 con 0x55745f11a070 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f04b40044c0 con 0x55745f11a070 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 10 client.25703 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 10 client.25703 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 20 client.25703 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0d89f0 0x55745f4e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 4 client.25703 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.128+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04ac0043d0 con 0x55745f0d89f0 2024-09-21T03:20:53.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.129+0000 7f04c1590640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0d89f0 0x55745f4e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.131+0000 7f04a57fa640 10 client.25703 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:20:53.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.131+0000 7f04c1590640 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0d89f0 0x55745f4e74f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f071c49db89d5ef server_cookie=7966db27b9e29656 in_seq=0 out_seq=0 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04a57fa640 1 -- 172.21.15.83:0/3569594452 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f04ac0043d0 con 0x55745f0d89f0 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04a57fa640 10 client.25703 handle_command_reply: tid=0 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 client.25703 shutdown 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 msgr2=0x55745f4e1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4e1560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55745f4b94f0 tx=0x7f04b4005bd0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 shutdown_connections 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4e1560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0d89f0 0x55745f4e74f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f9390 0x55745f4d5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 --2- 172.21.15.83:0/3569594452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04a57fa640 0 client.25703 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.132+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 >> 172.21.15.83:0/3569594452 conn(0x55745f10e460 msgr2=0x55745f4c22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.133+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 shutdown_connections 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.133+0000 7f04ca569740 1 -- 172.21.15.83:0/3569594452 wait complete. 2024-09-21T03:20:53.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.133+0000 7f04ca569740 20 client.25703 trim_cache size 0 max 16384 2024-09-21T03:20:53.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 Processor -- start 2024-09-21T03:20:53.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 -- start start 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f9390 0x55745f4e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b5c10 con 0x55745f0f11b0 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b77e0 con 0x55745f0f9390 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04ca569740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b62e0 con 0x55745f11a070 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04c2592640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04c2592640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54454/0 (socket says 172.21.15.83:54454) 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.155+0000 7f04c2592640 1 -- 172.21.15.83:0/3072165546 learned_addr learned my addr 172.21.15.83:0/3072165546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:53.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c1590640 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f9390 0x55745f4e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 -- 172.21.15.83:0/3072165546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 msgr2=0x55745f4ecc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4ecc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 -- 172.21.15.83:0/3072165546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f9390 msgr2=0x55745f4e1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f9390 0x55745f4e1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 -- 172.21.15.83:0/3072165546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745f4ba5d0 con 0x55745f0f11b0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04c2592640 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009560 tx=0x7f04b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b402661642f1cbd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04a77fe640 1 -- 172.21.15.83:0/3072165546 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4012500 con 0x55745f0f11b0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04a77fe640 1 -- 172.21.15.83:0/3072165546 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4009a70 con 0x55745f0f11b0 2024-09-21T03:20:53.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.156+0000 7f04a77fe640 1 -- 172.21.15.83:0/3072165546 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4029900 con 0x55745f0f11b0 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 -- 172.21.15.83:0/3072165546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 msgr2=0x55745f4bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009560 tx=0x7f04b4009ca0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 -- 172.21.15.83:0/3072165546 shutdown_connections 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f11a070 0x55745f4ecc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f0f9390 0x55745f4e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 --2- 172.21.15.83:0/3072165546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f11b0 0x55745f4bf1e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.157+0000 7f04ca569740 1 -- 172.21.15.83:0/3072165546 >> 172.21.15.83:0/3072165546 conn(0x55745f10e460 msgr2=0x55745f4bead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.158+0000 7f04ca569740 1 -- 172.21.15.83:0/3072165546 shutdown_connections 2024-09-21T03:20:53.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.158+0000 7f04ca569740 1 -- 172.21.15.83:0/3072165546 wait complete. 2024-09-21T03:20:53.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.159+0000 7f04ca569740 1 -- start start 2024-09-21T03:20:53.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.159+0000 7f04ca569740 1 Processor -- start 2024-09-21T03:20:53.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 0x55745f4dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b70e0 con 0x55745f0f9390 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b69e0 con 0x55745f11a070 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1b3c10 con 0x55745f0f11b0 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04ca569740 10 client.0 _ll_register_callbacks cb 0x7f04b4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c2592640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 0x55745f4dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c2592640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 0x55745f4dd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46146/0 (socket says 172.21.15.83:46146) 2024-09-21T03:20:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c2592640 1 -- 172.21.15.83:0/3577185815 learned_addr learned my addr 172.21.15.83:0/3577185815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c2592640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 -- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 msgr2=0x55745f4dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 0x55745f4dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 -- 172.21.15.83:0/3577185815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 msgr2=0x55745f4d8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4d8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c1590640 1 -- 172.21.15.83:0/3577185815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745f4d89c0 con 0x55745f0f9390 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.160+0000 7f04c2592640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.161+0000 7f04c1590640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4dda10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55745f4e7bf0 tx=0x7f04ac004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f610eb25cbe9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.161+0000 7f04a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.161+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac031020 con 0x55745f0f9390 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.161+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f04ac0037f0 con 0x55745f0f9390 2024-09-21T03:20:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.161+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b4009a70 con 0x55745f0f9390 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.162+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f04ac01e040 con 0x55745f0f9390 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 10 client.16179 fetch_fsmap learned FSMap version 37 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55745f5524c0 con 0x55745f0f9390 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4009a70 con 0x55745f0f9390 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f04ac024040 con 0x55745f0f9390 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 10 client.16179 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 10 client.16179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 20 client.16179 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0adda0 0x55745f554b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 4 client.16179 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.163+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55745f439df0 con 0x55745f0adda0 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.164+0000 7f04c2592640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0adda0 0x55745f554b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.164+0000 7f04a57fa640 10 client.16179 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.165+0000 7f04c2592640 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0adda0 0x55745f554b10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9ceb2083add3fdd server_cookie=3a8d2e62ab8cd0fa in_seq=0 out_seq=0 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.165+0000 7f04a57fa640 1 -- 172.21.15.83:0/3577185815 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55745f439df0 con 0x55745f0adda0 2024-09-21T03:20:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.165+0000 7f04a57fa640 10 client.16179 handle_command_reply: tid=0 2024-09-21T03:20:53.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 client.16179 shutdown 2024-09-21T03:20:53.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 msgr2=0x55745f4dda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4dda10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55745f4e7bf0 tx=0x7f04ac004340 comp rx=0 tx=0).stop 2024-09-21T03:20:53.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 shutdown_connections 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55745f11a070 0x55745f4d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55745f0f9390 0x55745f4dda10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55745f0adda0 0x55745f554b10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 --2- 172.21.15.83:0/3577185815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55745f0f11b0 0x55745f4dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 >> 172.21.15.83:0/3577185815 conn(0x55745f10e460 msgr2=0x55745f4c22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.166+0000 7f04a57fa640 0 client.16179 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 shutdown_connections 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04ca569740 1 -- 172.21.15.83:0/3577185815 wait complete. 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04ca569740 20 client.16179 trim_cache size 0 max 16384 2024-09-21T03:20:53.171 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 msgr2=0x7f04a80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 0x7f04a80b8b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55745f4bff40 tx=0x7f04b803d000 comp rx=0 tx=0).stop 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 msgr2=0x7f04bc1a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.167+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f04b8004770 tx=0x7f04b80040c0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 shutdown_connections 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f04a80b6610 0x7f04a80b8b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04bc116640 0x7f04bc1a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04bc10aee0 0x7f04bc1a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 --2- 172.21.15.83:0/3438219953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04bc10a590 0x7f04bc1a2cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 >> 172.21.15.83:0/3438219953 conn(0x7f04bc0712f0 msgr2=0x7f04bc10efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 shutdown_connections 2024-09-21T03:20:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:53.168+0000 7f04c3ff5640 1 -- 172.21.15.83:0/3438219953 wait complete. 2024-09-21T03:20:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:54 smithi193 ceph-mon[35224]: pgmap v452: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 54 MiB/s wr, 1.78k op/s 2024-09-21T03:20:54.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:54 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:54 smithi083 ceph-mon[31219]: pgmap v452: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 54 MiB/s wr, 1.78k op/s 2024-09-21T03:20:54.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:54 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:54 smithi142 ceph-mon[34512]: pgmap v452: 97 pgs: 97 active+clean; 3.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 54 MiB/s wr, 1.78k op/s 2024-09-21T03:20:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:54 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:20:54.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:20:55.074 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.909+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2510945024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 msgr2=0x7fa584115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.909+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2510945024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa584115fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa574009860 tx=0x7fa57402a340 comp rx=0 tx=0).stop 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.913+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2510945024 shutdown_connections 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.913+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2510945024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa584116640 0x7fa584118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.913+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2510945024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa584115fd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.913+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2510945024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa58410a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.913+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2510945024 >> 172.21.15.83:0/2510945024 conn(0x7fa584070ec0 msgr2=0x7fa5840712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.914+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2510945024 shutdown_connections 2024-09-21T03:20:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.914+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2510945024 wait complete. 2024-09-21T03:20:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.915+0000 7fa58b7b2640 1 Processor -- start 2024-09-21T03:20:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.917+0000 7fa58b7b2640 1 -- start start 2024-09-21T03:20:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.918+0000 7fa58b7b2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.918+0000 7fa58b7b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa5841a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.919+0000 7fa58954e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.919+0000 7fa58954e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44126/0 (socket says 172.21.15.83:44126) 2024-09-21T03:20:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.919+0000 7fa58954e640 1 -- 172.21.15.83:0/2925570227 learned_addr learned my addr 172.21.15.83:0/2925570227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.919+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa584116640 0x7fa5841a8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.919+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa58411c2d0 con 0x7fa58410aee0 2024-09-21T03:20:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.920+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa58411c150 con 0x7fa58410a590 2024-09-21T03:20:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.920+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa58411c450 con 0x7fa584116640 2024-09-21T03:20:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.920+0000 7fa588d4d640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa5841a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.921+0000 7fa58954e640 1 -- 172.21.15.83:0/2925570227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa584116640 msgr2=0x7fa5841a8560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:20:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.921+0000 7fa58954e640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa584116640 0x7fa5841a8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.921+0000 7fa58954e640 1 -- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 msgr2=0x7fa5841a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.921+0000 7fa58954e640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa5841a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.921+0000 7fa58954e640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5841a8c90 con 0x7fa58410a590 2024-09-21T03:20:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.922+0000 7fa58954e640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c890 tx=0x7fa58000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.923+0000 7fa57a7fc640 1 -- 172.21.15.83:0/2925570227 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580004220 con 0x7fa58410a590 2024-09-21T03:20:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.923+0000 7fa57a7fc640 1 -- 172.21.15.83:0/2925570227 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5800043c0 con 0x7fa58410a590 2024-09-21T03:20:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.923+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5841a8f80 con 0x7fa58410a590 2024-09-21T03:20:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.923+0000 7fa57a7fc640 1 -- 172.21.15.83:0/2925570227 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580004a00 con 0x7fa58410a590 2024-09-21T03:20:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.923+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58410bd60 con 0x7fa58410a590 2024-09-21T03:20:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.927+0000 7fa591d26740 1 Processor -- start 2024-09-21T03:20:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 -- start start 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6cb070 0x56142ea7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6a21b0 0x56142ea7b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56142e9e8ea0 con 0x56142e6cb070 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56142e9d82d0 con 0x56142e6aa390 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.928+0000 7fa591d26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56142e9dc0e0 con 0x56142e6a21b0 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44142/0 (socket says 172.21.15.83:44142) 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 -- 172.21.15.83:0/2693455742 learned_addr learned my addr 172.21.15.83:0/2693455742 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa57a7fc640 1 -- 172.21.15.83:0/2925570227 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa580020020 con 0x7fa58410a590 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa589d4f640 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6cb070 0x56142ea7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 -- 172.21.15.83:0/2693455742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6a21b0 msgr2=0x56142ea7b3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6a21b0 0x56142ea7b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 -- 172.21.15.83:0/2693455742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6cb070 msgr2=0x56142ea7aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6cb070 0x56142ea7aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.929+0000 7fa588d4d640 1 -- 172.21.15.83:0/2693455742 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142e736650 con 0x56142e6aa390 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.930+0000 7fa588d4d640 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa574009830 tx=0x7fa574004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe243283fc9c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.931+0000 7fa56f7fe640 1 -- 172.21.15.83:0/2693455742 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574037040 con 0x56142e6aa390 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.931+0000 7fa56f7fe640 1 -- 172.21.15.83:0/2693455742 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa574042070 con 0x56142e6aa390 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.931+0000 7fa57a7fc640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 0x7fa5700b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.931+0000 7fa56f7fe640 1 -- 172.21.15.83:0/2693455742 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574004d40 con 0x56142e6aa390 2024-09-21T03:20:55.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.931+0000 7fa589d4f640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 0x7fa5700b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa57a7fc640 1 -- 172.21.15.83:0/2925570227 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa5800e6ae0 con 0x7fa58410a590 2024-09-21T03:20:55.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 -- 172.21.15.83:0/2693455742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 msgr2=0x56142ea7b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa589d4f640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 0x7fa5700b8d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005fd0 tx=0x7fa57c0095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa574009830 tx=0x7fa574004040 comp rx=0 tx=0).stop 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 -- 172.21.15.83:0/2693455742 shutdown_connections 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea7b930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6a21b0 0x56142ea7b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 --2- 172.21.15.83:0/2693455742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6cb070 0x56142ea7aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.932+0000 7fa591d26740 1 -- 172.21.15.83:0/2693455742 >> 172.21.15.83:0/2693455742 conn(0x56142e6bf460 msgr2=0x56142ea78da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.933+0000 7fa591d26740 1 -- 172.21.15.83:0/2693455742 shutdown_connections 2024-09-21T03:20:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.933+0000 7fa591d26740 1 -- 172.21.15.83:0/2693455742 wait complete. 2024-09-21T03:20:55.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.933+0000 7fa591d26740 1 -- start start 2024-09-21T03:20:55.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 215.0 s, 3449.2 tps, lat 28.675 ms stddev 31.784 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 Processor -- start 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56142e839700 con 0x56142e6a21b0 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56142e973160 con 0x56142e6aa390 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56142ea896d0 con 0x56142e6cb070 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.934+0000 7fa591d26740 10 client.0 _ll_register_callbacks cb 0x56142e98c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54478/0 (socket says 172.21.15.83:54478) 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 -- 172.21.15.83:0/3617883792 learned_addr learned my addr 172.21.15.83:0/3617883792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa589d4f640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa588d4d640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 -- 172.21.15.83:0/3617883792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 msgr2=0x56142ea87090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea87090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 -- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 msgr2=0x56142ea86b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea86b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.935+0000 7fa58954e640 1 -- 172.21.15.83:0/3617883792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142e9a0c90 con 0x56142e6a21b0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.936+0000 7fa58954e640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa5800a9bc0 tx=0x7fa5800b62a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f0de81cd8055fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.936+0000 7fa56cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:55.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.936+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800ac0c0 con 0x56142e6a21b0 2024-09-21T03:20:55.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.936+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa574042070 con 0x56142e6a21b0 2024-09-21T03:20:55.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.936+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa5800abbb0 con 0x56142e6a21b0 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa580097020 con 0x56142e6a21b0 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 10 client.16191 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56142ea95430 con 0x56142e6a21b0 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa574042070 con 0x56142e6a21b0 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fa5800a82a0 con 0x56142e6a21b0 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 10 client.16191 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 20 client.16191 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e6899f0 0x56142ea98e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 4 client.16191 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.937+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142e9eadf0 con 0x56142e6899f0 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.938+0000 7fa588d4d640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e6899f0 0x56142ea98e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.938+0000 7fa56cff9640 10 client.16191 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.938+0000 7fa588d4d640 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e6899f0 0x56142ea98e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7443c3e748d7d6ec server_cookie=d037dfcd47b03ff9 in_seq=0 out_seq=0 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa56cff9640 1 -- 172.21.15.83:0/3617883792 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56142e9eadf0 con 0x56142e6899f0 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa56cff9640 10 client.16191 handle_command_reply: tid=0 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 client.16191 shutdown 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 msgr2=0x56142ea865d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa5800a9bc0 tx=0x7fa5800b62a0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 shutdown_connections 2024-09-21T03:20:55.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e6899f0 0x56142ea98e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea87090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6aa390 0x56142ea86b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 --2- 172.21.15.83:0/3617883792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142ea865d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 >> 172.21.15.83:0/3617883792 conn(0x56142e6bf460 msgr2=0x56142ea7e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa56cff9640 0 client.16191 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 shutdown_connections 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 1 -- 172.21.15.83:0/3617883792 wait complete. 2024-09-21T03:20:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.940+0000 7fa591d26740 20 client.16191 trim_cache size 0 max 16384 2024-09-21T03:20:55.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.966+0000 7fa591d26740 1 Processor -- start 2024-09-21T03:20:55.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.966+0000 7fa591d26740 1 -- start start 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.966+0000 7fa591d26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.967+0000 7fa591d26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6aa390 0x56142ea868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.967+0000 7fa591d26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.967+0000 7fa591d26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa56800e700 con 0x56142e6aa390 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.967+0000 7fa591d26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5680102a0 con 0x56142e6a21b0 2024-09-21T03:20:55.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.967+0000 7fa591d26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa56800edd0 con 0x56142e6cb070 2024-09-21T03:20:55.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.968+0000 7fa589d4f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.968+0000 7fa58954e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.968+0000 7fa58954e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44162/0 (socket says 172.21.15.83:44162) 2024-09-21T03:20:55.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.968+0000 7fa58954e640 1 -- 172.21.15.83:0/110665441 learned_addr learned my addr 172.21.15.83:0/110665441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.970+0000 7fa58954e640 1 -- 172.21.15.83:0/110665441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 msgr2=0x56142ea837c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.970+0000 7fa58954e640 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea837c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.970+0000 7fa58954e640 1 -- 172.21.15.83:0/110665441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6aa390 msgr2=0x56142ea868c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:20:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.970+0000 7fa58954e640 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6aa390 0x56142ea868c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.970+0000 7fa58954e640 1 -- 172.21.15.83:0/110665441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142ea79b30 con 0x56142e6a21b0 2024-09-21T03:20:55.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.972+0000 7fa58954e640 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa580007590 tx=0x7fa5800d00d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f07657a41137394 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.974+0000 7fa56effd640 1 -- 172.21.15.83:0/110665441 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800ac0c0 con 0x56142e6a21b0 2024-09-21T03:20:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.974+0000 7fa56effd640 1 -- 172.21.15.83:0/110665441 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5800ac460 con 0x56142e6a21b0 2024-09-21T03:20:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.974+0000 7fa56effd640 1 -- 172.21.15.83:0/110665441 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800e46e0 con 0x56142e6a21b0 2024-09-21T03:20:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.976+0000 7fa591d26740 1 -- 172.21.15.83:0/110665441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 msgr2=0x56142ea86380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.976+0000 7fa591d26740 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa580007590 tx=0x7fa5800d00d0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.977+0000 7fa591d26740 1 -- 172.21.15.83:0/110665441 shutdown_connections 2024-09-21T03:20:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.977+0000 7fa591d26740 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6cb070 0x56142ea837c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.977+0000 7fa591d26740 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6aa390 0x56142ea868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.977+0000 7fa591d26740 1 --2- 172.21.15.83:0/110665441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6a21b0 0x56142ea86380 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.978+0000 7fa591d26740 1 -- 172.21.15.83:0/110665441 >> 172.21.15.83:0/110665441 conn(0x56142e6bf460 msgr2=0x56142ea84e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.982+0000 7fa591d26740 1 -- 172.21.15.83:0/110665441 shutdown_connections 2024-09-21T03:20:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.982+0000 7fa591d26740 1 -- 172.21.15.83:0/110665441 wait complete. 2024-09-21T03:20:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.983+0000 7fa591d26740 1 -- start start 2024-09-21T03:20:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.983+0000 7fa591d26740 1 Processor -- start 2024-09-21T03:20:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.984+0000 7fa591d26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa591d26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 0x56142eafbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa58954e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa58954e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54480/0 (socket says 172.21.15.83:54480) 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa588d4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 0x56142eafbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa588d4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 0x56142eafbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46214/0 (socket says 172.21.15.83:46214) 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa588d4d640 1 -- 172.21.15.83:0/3584211428 learned_addr learned my addr 172.21.15.83:0/3584211428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.985+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 0x56142ea8e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa56800fbd0 con 0x56142e6a21b0 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa56800f4d0 con 0x56142e6cb070 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa56800c6e0 con 0x56142e6aa390 2024-09-21T03:20:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa589d4f640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 0x56142ea8e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa591d26740 10 client.0 _ll_register_callbacks cb 0x7fa5800046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa58954e640 1 -- 172.21.15.83:0/3584211428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 msgr2=0x56142eafbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa58954e640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 0x56142eafbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa58954e640 1 -- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 msgr2=0x56142ea8e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa58954e640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 0x56142ea8e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa58954e640 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142ea8ee10 con 0x56142e6a21b0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.986+0000 7fa589d4f640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 0x56142ea8e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.987+0000 7fa58954e640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa580005fd0 tx=0x7fa5800ab920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fab8cd270aaa74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.987+0000 7fa56cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.987+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800bb060 con 0x56142e6a21b0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.987+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa5800c2b00 con 0x56142e6a21b0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.987+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5800ac460 con 0x56142e6a21b0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.988+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa5800e46e0 con 0x56142e6a21b0 2024-09-21T03:20:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.988+0000 7fa591d26740 10 client.16194 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.988+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56142eb03170 con 0x56142e6a21b0 2024-09-21T03:20:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5800b6370 con 0x56142e6a21b0 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fa5800ce480 con 0x56142e6a21b0 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 10 client.16194 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 20 client.16194 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e65eda0 0x56142eb05690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 4 client.16194 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.989+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa5800ac460 con 0x56142e65eda0 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.990+0000 7fa588d4d640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e65eda0 0x56142eb05690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.990+0000 7fa56cff9640 10 client.16194 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.990+0000 7fa588d4d640 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e65eda0 0x56142eb05690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92b75f8cbee927cc server_cookie=168c585ad5c87416 in_seq=0 out_seq=0 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa56cff9640 1 -- 172.21.15.83:0/3584211428 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa5800ac460 con 0x56142e65eda0 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa56cff9640 10 client.16194 handle_command_reply: tid=0 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 client.16194 shutdown 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 msgr2=0x56142eafb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa580005fd0 tx=0x7fa5800ab920 comp rx=0 tx=0).stop 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 shutdown_connections 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56142e65eda0 0x56142eb05690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56142e6cb070 0x56142ea8e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56142e6aa390 0x56142eafbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 --2- 172.21.15.83:0/3584211428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142e6a21b0 0x56142eafb9b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.991+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 >> 172.21.15.83:0/3584211428 conn(0x56142e6bf460 msgr2=0x56142ea86380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.992+0000 7fa56cff9640 0 client.16194 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.992+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 shutdown_connections 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.992+0000 7fa591d26740 1 -- 172.21.15.83:0/3584211428 wait complete. 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.992+0000 7fa591d26740 20 client.16194 trim_cache size 0 max 16384 2024-09-21T03:20:55.997 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.994+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 msgr2=0x7fa5700b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.994+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 0x7fa5700b8d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005fd0 tx=0x7fa57c0095c0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.994+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 msgr2=0x7fa5841a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.994+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c890 tx=0x7fa58000cd50 comp rx=0 tx=0).stop 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 shutdown_connections 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa5700b6880 0x7fa5700b8d90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa584116640 0x7fa5841a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa58410aee0 0x7fa5841a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 --2- 172.21.15.83:0/2925570227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58410a590 0x7fa5841a2d20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 >> 172.21.15.83:0/2925570227 conn(0x7fa584070ec0 msgr2=0x7fa584073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 shutdown_connections 2024-09-21T03:20:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:55.995+0000 7fa58b7b2640 1 -- 172.21.15.83:0/2925570227 wait complete. 2024-09-21T03:20:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:56 smithi193 ceph-mon[35224]: pgmap v453: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 44 MiB/s wr, 1.63k op/s 2024-09-21T03:20:56.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:56 smithi083 ceph-mon[31219]: pgmap v453: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 44 MiB/s wr, 1.63k op/s 2024-09-21T03:20:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:56 smithi142 ceph-mon[34512]: pgmap v453: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 44 MiB/s wr, 1.63k op/s 2024-09-21T03:20:56.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:20:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.766+0000 7f2bcf57a640 1 -- 172.21.15.83:0/4073988359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8101fc0 msgr2=0x7f2bc81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.766+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/4073988359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8101fc0 0x7f2bc81023d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009900 tx=0x7f2bc402a410 comp rx=0 tx=0).stop 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.767+0000 7f2bcf57a640 1 -- 172.21.15.83:0/4073988359 shutdown_connections 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.767+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/4073988359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8109e50 0x7f2bc8110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.767+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/4073988359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109500 0x7f2bc8109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.767+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/4073988359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8101fc0 0x7f2bc81023d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.767+0000 7f2bcf57a640 1 -- 172.21.15.83:0/4073988359 >> 172.21.15.83:0/4073988359 conn(0x7f2bc8079f90 msgr2=0x7f2bc807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.768+0000 7f2bcf57a640 1 -- 172.21.15.83:0/4073988359 shutdown_connections 2024-09-21T03:20:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.768+0000 7f2bcf57a640 1 -- 172.21.15.83:0/4073988359 wait complete. 2024-09-21T03:20:57.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 Processor -- start 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 -- start start 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8101fc0 0x7f2bc8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109e50 0x7f2bc819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8111910 con 0x7f2bc8101fc0 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8111790 con 0x7f2bc8109500 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcf57a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8111a90 con 0x7f2bc8109e50 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bccb15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bccb15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44174/0 (socket says 172.21.15.83:44174) 2024-09-21T03:20:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bccb15640 1 -- 172.21.15.83:0/2908108897 learned_addr learned my addr 172.21.15.83:0/2908108897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcd316640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8101fc0 0x7f2bc8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.769+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109e50 0x7f2bc819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 -- 172.21.15.83:0/2908108897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109e50 msgr2=0x7f2bc819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109e50 0x7f2bc819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 -- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8101fc0 msgr2=0x7f2bc8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8101fc0 0x7f2bc8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 -- 172.21.15.83:0/2908108897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc819e290 con 0x7f2bc8109500 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.770+0000 7f2bccb15640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e9c0 tx=0x7f2bb800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.771+0000 7f2bbe7fc640 1 -- 172.21.15.83:0/2908108897 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800ccc0 con 0x7f2bc8109500 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.771+0000 7f2bbe7fc640 1 -- 172.21.15.83:0/2908108897 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80054d0 con 0x7f2bc8109500 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.771+0000 7f2bbe7fc640 1 -- 172.21.15.83:0/2908108897 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8010430 con 0x7f2bc8109500 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.771+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc819e580 con 0x7f2bc8109500 2024-09-21T03:20:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.771+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc81a4ce0 con 0x7f2bc8109500 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bd5aee740 1 Processor -- start 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bd5aee740 1 -- start start 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 0x5587b6cc52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bbe7fc640 1 -- 172.21.15.83:0/2908108897 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2bb8005020 con 0x7f2bc8109500 2024-09-21T03:20:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.776+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46248/0 (socket says 172.21.15.83:46248) 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bcd316640 1 -- 172.21.15.83:0/2933510277 learned_addr learned my addr 172.21.15.83:0/2933510277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6cc5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587b6c3bea0 con 0x5587b68fd390 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5587b6c2b2d0 con 0x5587b691e070 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587b6c2f0e0 con 0x5587b68f51b0 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 0x5587b6cc52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.777+0000 7f2bccb15640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6cc5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 -- 172.21.15.83:0/2933510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 msgr2=0x5587b6cc5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 -- 172.21.15.83:0/2933510277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 msgr2=0x5587b6cc5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6cc5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 -- 172.21.15.83:0/2933510277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b6989650 con 0x5587b691e070 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bbe7fc640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 0x7f2bb40b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcd316640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcd316640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 0x7f2bb40b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bbe7fc640 1 -- 172.21.15.83:0/2908108897 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2bb80e79d0 con 0x7f2bc8109500 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.778+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 0x5587b6cc52c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0003c90 tx=0x7f2bc0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a8629f3b259ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.780+0000 7f2bb37fe640 1 -- 172.21.15.83:0/2933510277 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0022030 con 0x5587b691e070 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.780+0000 7f2bb37fe640 1 -- 172.21.15.83:0/2933510277 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc0004100 con 0x5587b691e070 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.780+0000 7f2bb37fe640 1 -- 172.21.15.83:0/2933510277 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0005310 con 0x5587b691e070 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 msgr2=0x5587b6cc52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bcd316640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 0x7f2bb40b8ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5587b6cc3f40 tx=0x7f2bc4005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 0x5587b6cc52c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0003c90 tx=0x7f2bc0009500 comp rx=0 tx=0).stop 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 shutdown_connections 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6cc5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b68f51b0 0x5587b6cc5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/2933510277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b691e070 0x5587b6cc52c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.781+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 >> 172.21.15.83:0/2933510277 conn(0x5587b6912460 msgr2=0x5587b6cc31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.782+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 shutdown_connections 2024-09-21T03:20:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.782+0000 7f2bd5aee740 1 -- 172.21.15.83:0/2933510277 wait complete. 2024-09-21T03:20:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.783+0000 7f2bd5aee740 1 -- start start 2024-09-21T03:20:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.783+0000 7f2bd5aee740 1 Processor -- start 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.783+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.783+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.783+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6cd2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587b6a8c700 con 0x5587b68f51b0 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5587b6bc6160 con 0x5587b68fd390 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587b6cd1910 con 0x5587b691e070 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bd5aee740 10 client.0 _ll_register_callbacks cb 0x5587b6bdf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44194/0 (socket says 172.21.15.83:44194) 2024-09-21T03:20:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 -- 172.21.15.83:0/1005294288 learned_addr learned my addr 172.21.15.83:0/1005294288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6cd2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bcd316640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 -- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 msgr2=0x5587b6cd2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6cd2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 -- 172.21.15.83:0/1005294288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 msgr2=0x5587b6cd1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.784+0000 7f2bccb15640 1 -- 172.21.15.83:0/1005294288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b6bf3c90 con 0x5587b68fd390 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.785+0000 7f2bccb15640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c6a0 tx=0x7f2bb80a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf295134c6b3127 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.785+0000 7f2bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.785+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80ad650 con 0x5587b68fd390 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.786+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc0004100 con 0x5587b68fd390 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.786+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2bb80accf0 con 0x5587b68fd390 2024-09-21T03:20:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.786+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2bb800ce60 con 0x5587b68fd390 2024-09-21T03:20:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.786+0000 7f2bd5aee740 10 client.25736 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.786+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5587b6cdf5c0 con 0x5587b68fd390 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc0004100 con 0x5587b68fd390 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7f2bb80ac550 con 0x5587b68fd390 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 10 client.25736 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 10 client.25736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 20 client.25736 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68dc9f0 0x5587b6ce2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 4 client.25736 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:57.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.787+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587b6c3ddf0 con 0x5587b68dc9f0 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.788+0000 7f2bcd316640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68dc9f0 0x5587b6ce2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.788+0000 7f2bb0ff9640 10 client.25736 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.788+0000 7f2bcd316640 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68dc9f0 0x5587b6ce2e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc463568d9a4c54 server_cookie=384dcce41656a2f5 in_seq=0 out_seq=0 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.789+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/1005294288 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587b6c3ddf0 con 0x5587b68dc9f0 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.789+0000 7f2bb0ff9640 10 client.25736 handle_command_reply: tid=0 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.789+0000 7f2bd5aee740 1 client.25736 shutdown 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 msgr2=0x5587b6cd2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c6a0 tx=0x7f2bb80a9f90 comp rx=0 tx=0).stop 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 shutdown_connections 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68dc9f0 0x5587b6ce2e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6cd2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd2110 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/1005294288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 >> 172.21.15.83:0/1005294288 conn(0x5587b6912460 msgr2=0x5587b6cc8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.790+0000 7f2bb0ff9640 0 client.25736 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.791+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 shutdown_connections 2024-09-21T03:20:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.791+0000 7f2bd5aee740 1 -- 172.21.15.83:0/1005294288 wait complete. 2024-09-21T03:20:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.791+0000 7f2bd5aee740 20 client.25736 trim_cache size 0 max 16384 2024-09-21T03:20:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.811+0000 7f2bd5aee740 1 Processor -- start 2024-09-21T03:20:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.811+0000 7f2bd5aee740 1 -- start start 2024-09-21T03:20:57.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.811+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6ccda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6ccdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8075290 con 0x5587b68fd390 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81a66f0 con 0x5587b68f51b0 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81a5220 con 0x5587b691e070 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44196/0 (socket says 172.21.15.83:44196) 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 -- 172.21.15.83:0/4024074868 learned_addr learned my addr 172.21.15.83:0/4024074868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6ccdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bccb15640 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6ccda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 -- 172.21.15.83:0/4024074868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 msgr2=0x5587b6ccdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6ccdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 -- 172.21.15.83:0/4024074868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 msgr2=0x5587b6ccda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6ccda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.812+0000 7f2bcd316640 1 -- 172.21.15.83:0/4024074868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b6cc7a10 con 0x5587b68f51b0 2024-09-21T03:20:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.813+0000 7f2bcd316640 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005d90 tx=0x7f2bc4060040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29837fd9ca6fef40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.814+0000 7f2bb2ffd640 1 -- 172.21.15.83:0/4024074868 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4035070 con 0x5587b68f51b0 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.814+0000 7f2bb2ffd640 1 -- 172.21.15.83:0/4024074868 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc40055b0 con 0x5587b68f51b0 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.814+0000 7f2bb2ffd640 1 -- 172.21.15.83:0/4024074868 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40046c0 con 0x5587b68f51b0 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4024074868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 msgr2=0x5587b6cc6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005d90 tx=0x7f2bc4060040 comp rx=0 tx=0).stop 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4024074868 shutdown_connections 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6ccdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68fd390 0x5587b6ccda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4024074868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68f51b0 0x5587b6cc6cb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4024074868 >> 172.21.15.83:0/4024074868 conn(0x5587b6912460 msgr2=0x5587b6ccf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4024074868 shutdown_connections 2024-09-21T03:20:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.815+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4024074868 wait complete. 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.816+0000 7f2bd5aee740 1 -- start start 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.816+0000 7f2bd5aee740 1 Processor -- start 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.816+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6d5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81a6020 con 0x5587b68f51b0 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81a5920 con 0x5587b68fd390 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8073270 con 0x5587b691e070 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bcd316640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54530/0 (socket says 172.21.15.83:54530) 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bcd316640 1 -- 172.21.15.83:0/4043365008 learned_addr learned my addr 172.21.15.83:0/4043365008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bd5aee740 10 client.0 _ll_register_callbacks cb 0x7f2bb80eb1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bcdb17640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6d5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 -- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 msgr2=0x5587b6d5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6d5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 -- 172.21.15.83:0/4043365008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 msgr2=0x5587b6cd57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.817+0000 7f2bccb15640 1 -- 172.21.15.83:0/4043365008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b6d5dff0 con 0x5587b68fd390 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bccb15640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd5ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80ad810 tx=0x7f2bb8004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7ad99b6edf2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80c8060 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f2bb80aedb0 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc40055b0 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2bb80cf050 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 10 client.25742 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5587b6d5f190 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc40055b0 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7f2bb80b34f0 con 0x5587b68fd390 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 10 client.25742 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 10 client.25742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 20 client.25742 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:57.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68b1da0 0x5587b6d61960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 4 client.25742 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:57.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.818+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587b6c398a0 con 0x5587b68b1da0 2024-09-21T03:20:57.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.819+0000 7f2bcd316640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68b1da0 0x5587b6d61960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.821+0000 7f2bcd316640 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68b1da0 0x5587b6d61960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7d1f0c3d0e75286 server_cookie=1daaf0bf0c9bf60 in_seq=0 out_seq=0 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.821+0000 7f2bb0ff9640 10 client.25742 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.821+0000 7f2bb0ff9640 1 -- 172.21.15.83:0/4043365008 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587b6c398a0 con 0x5587b68b1da0 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.821+0000 7f2bb0ff9640 10 client.25742 handle_command_reply: tid=0 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.821+0000 7f2bd5aee740 1 client.25742 shutdown 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 msgr2=0x5587b6cd5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd5ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80ad810 tx=0x7f2bb8004990 comp rx=0 tx=0).stop 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 shutdown_connections 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x5587b68b1da0 0x5587b6d61960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bb0ff9640 0 client.25742 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b691e070 0x5587b6d5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587b68fd390 0x5587b6cd5ce0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 --2- 172.21.15.83:0/4043365008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587b68f51b0 0x5587b6cd57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 >> 172.21.15.83:0/4043365008 conn(0x5587b6912460 msgr2=0x5587b6ce3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 shutdown_connections 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 1 -- 172.21.15.83:0/4043365008 wait complete. 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.822+0000 7f2bd5aee740 20 client.25742 trim_cache size 0 max 16384 2024-09-21T03:20:57.827 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 msgr2=0x7f2bb40b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 0x7f2bb40b8ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5587b6cc3f40 tx=0x7f2bc4005850 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 msgr2=0x7f2bc81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e9c0 tx=0x7f2bb800ee80 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 shutdown_connections 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2bb40b67d0 0x7f2bb40b8ce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bc8109e50 0x7f2bc819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bc8109500 0x7f2bc81988c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 --2- 172.21.15.83:0/2908108897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc8101fc0 0x7f2bc8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 >> 172.21.15.83:0/2908108897 conn(0x7f2bc8079f90 msgr2=0x7f2bc8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 shutdown_connections 2024-09-21T03:20:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:57.824+0000 7f2bcf57a640 1 -- 172.21.15.83:0/2908108897 wait complete. 2024-09-21T03:20:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:58 smithi193 ceph-mon[35224]: pgmap v454: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 43 MiB/s wr, 1.65k op/s 2024-09-21T03:20:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:58.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:20:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:20:58.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:58 smithi083 ceph-mon[31219]: pgmap v454: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 43 MiB/s wr, 1.65k op/s 2024-09-21T03:20:58.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:58.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:58.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:58.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:20:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:20:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:58 smithi142 ceph-mon[34512]: pgmap v454: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 43 MiB/s wr, 1.65k op/s 2024-09-21T03:20:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:20:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:20:58.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:20:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 -- 172.21.15.83:0/490818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88110250 msgr2=0x7f0a88116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 --2- 172.21.15.83:0/490818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88110250 0x7f0a88116a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009860 tx=0x7f0a7c02a340 comp rx=0 tx=0).stop 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 -- 172.21.15.83:0/490818899 shutdown_connections 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 --2- 172.21.15.83:0/490818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88110250 0x7f0a88116a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 --2- 172.21.15.83:0/490818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8810f900 0x7f0a8810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 --2- 172.21.15.83:0/490818899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a88109fc0 0x7f0a8810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 -- 172.21.15.83:0/490818899 >> 172.21.15.83:0/490818899 conn(0x7f0a880795c0 msgr2=0x7f0a881081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 -- 172.21.15.83:0/490818899 shutdown_connections 2024-09-21T03:20:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.792+0000 7f0a87fff640 1 -- 172.21.15.83:0/490818899 wait complete. 2024-09-21T03:20:59.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.793+0000 7f0a87fff640 1 Processor -- start 2024-09-21T03:20:59.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.793+0000 7f0a87fff640 1 -- start start 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88109fc0 0x7f0a881a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 0x7f0a881a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88119f10 con 0x7f0a8810f900 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88119d90 con 0x7f0a88110250 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a87fff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8811a090 con 0x7f0a88109fc0 2024-09-21T03:20:59.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41220/0 (socket says 172.21.15.83:41220) 2024-09-21T03:20:59.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a877fe640 1 -- 172.21.15.83:0/3784410048 learned_addr learned my addr 172.21.15.83:0/3784410048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.794+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88109fc0 0x7f0a881a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 -- 172.21.15.83:0/3784410048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88109fc0 msgr2=0x7f0a881a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88109fc0 0x7f0a881a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 -- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 msgr2=0x7f0a881a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 0x7f0a881a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 0x7f0a881a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 -- 172.21.15.83:0/3784410048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a881a6870 con 0x7f0a88110250 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 0x7f0a881a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a877fe640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0303f0 tx=0x7f0a7c032d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a8ca4e640 1 -- 172.21.15.83:0/3784410048 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c042070 con 0x7f0a88110250 2024-09-21T03:20:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a8ca4e640 1 -- 172.21.15.83:0/3784410048 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a7c030d50 con 0x7f0a88110250 2024-09-21T03:20:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a8ca4e640 1 -- 172.21.15.83:0/3784410048 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c037040 con 0x7f0a88110250 2024-09-21T03:20:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a881a6b00 con 0x7f0a88110250 2024-09-21T03:20:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.795+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a881ad2c0 con 0x7f0a88110250 2024-09-21T03:20:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.798+0000 7f0a93fc4740 1 Processor -- start 2024-09-21T03:20:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.799+0000 7f0a8ca4e640 1 -- 172.21.15.83:0/3784410048 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a7c045070 con 0x7f0a88110250 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.799+0000 7f0a93fc4740 1 -- start start 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 0x561fab991460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5c9390 0x561fab991ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fab907ea0 con 0x561fab5c9390 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561fab8f72d0 con 0x561fab5ea070 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fab8fb0e0 con 0x561fab5c11b0 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a867fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 0x561fab991460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.800+0000 7f0a86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39096/0 (socket says 172.21.15.83:39096) 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3939520856 learned_addr learned my addr 172.21.15.83:0/3939520856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a8ca4e640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 0x7f0a700b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.801+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3939520856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 msgr2=0x561fab991460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 0x561fab991460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a8ca4e640 1 -- 172.21.15.83:0/3784410048 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0a7c106ce0 con 0x7f0a88110250 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3939520856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5c9390 msgr2=0x561fab991ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a877fe640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 0x7f0a700b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5c9390 0x561fab991ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3939520856 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab655650 con 0x561fab5c11b0 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 0x561fab991460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a877fe640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 0x7f0a700b8cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0ccae0 tx=0x7f0a7c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.802+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a80010400 tx=0x7f0a800108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78584f05fa34f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.803+0000 7f0a6dffb640 1 -- 172.21.15.83:0/3939520856 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80003b70 con 0x561fab5c11b0 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.803+0000 7f0a6dffb640 1 -- 172.21.15.83:0/3939520856 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a80003d10 con 0x561fab5c11b0 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.803+0000 7f0a6dffb640 1 -- 172.21.15.83:0/3939520856 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011ad0 con 0x561fab5c11b0 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3939520856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 msgr2=0x561fab9919a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a80010400 tx=0x7f0a800108c0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3939520856 shutdown_connections 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5c9390 0x561fab991ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab9919a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3939520856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5ea070 0x561fab991460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3939520856 >> 172.21.15.83:0/3939520856 conn(0x561fab5de460 msgr2=0x561fab98f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3939520856 shutdown_connections 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.804+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3939520856 wait complete. 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.805+0000 7f0a93fc4740 1 -- start start 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.805+0000 7f0a93fc4740 1 Processor -- start 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c9390 0x561fab99d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fab758700 con 0x561fab5ea070 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561fab892160 con 0x561fab5c9390 2024-09-21T03:20:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fab9b0ca0 con 0x561fab5c11b0 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a93fc4740 10 client.0 _ll_register_callbacks cb 0x561fab8ab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a867fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a867fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39106/0 (socket says 172.21.15.83:39106) 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.806+0000 7f0a867fc640 1 -- 172.21.15.83:0/912932090 learned_addr learned my addr 172.21.15.83:0/912932090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 -- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c9390 msgr2=0x561fab99d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c9390 0x561fab99d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 -- 172.21.15.83:0/912932090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 msgr2=0x561fab9b9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 -- 172.21.15.83:0/912932090 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fab8bfc90 con 0x561fab5c11b0 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a867fc640 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009fc0 tx=0x7f0a78012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471760b0033d5265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.807+0000 7f0a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.808+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7801f060 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.808+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a80003d10 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.808+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0a78010ce0 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.808+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0a78012e20 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.808+0000 7f0a93fc4740 10 client.25729 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561fab9bc970 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a80003d10 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7f0a78023430 con 0x561fab5c11b0 2024-09-21T03:20:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 10 client.25729 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 20 client.25729 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab5a89f0 0x561fab9c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 4 client.25729 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.809+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fab909df0 con 0x561fab5a89f0 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.810+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab5a89f0 0x561fab9c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.810+0000 7f0a66ffd640 10 client.25729 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.810+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab5a89f0 0x561fab9c0290 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc644f5c699ea862 server_cookie=65d3309100e7ed9b in_seq=0 out_seq=0 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.811+0000 7f0a66ffd640 1 -- 172.21.15.83:0/912932090 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fab909df0 con 0x561fab5a89f0 2024-09-21T03:20:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.811+0000 7f0a66ffd640 10 client.25729 handle_command_reply: tid=0 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.811+0000 7f0a93fc4740 1 client.25729 shutdown 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 msgr2=0x561fab99cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009fc0 tx=0x7f0a78012bd0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 shutdown_connections 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab5a89f0 0x561fab9c0290 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c9390 0x561fab99d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/912932090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c11b0 0x561fab99cad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 >> 172.21.15.83:0/912932090 conn(0x561fab5de460 msgr2=0x561fab9945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a66ffd640 0 client.25729 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 shutdown_connections 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 1 -- 172.21.15.83:0/912932090 wait complete. 2024-09-21T03:20:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.812+0000 7f0a93fc4740 20 client.25729 trim_cache size 0 max 16384 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 Processor -- start 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 -- start start 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab99cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab98eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88075440 con 0x561fab5ea070 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881aee80 con 0x561fab5c11b0 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a93fc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881ad9b0 con 0x561fab5c9390 2024-09-21T03:20:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.835+0000 7f0a86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39110/0 (socket says 172.21.15.83:39110) 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3211399519 learned_addr learned my addr 172.21.15.83:0/3211399519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab98eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab99cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3211399519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 msgr2=0x561fab99cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab99cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3211399519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 msgr2=0x561fab98eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab98eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 -- 172.21.15.83:0/3211399519 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fab9c0860 con 0x561fab5c9390 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a867fc640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab98eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.836+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a80010900 tx=0x7f0a80010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00ee973c5c8ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.837+0000 7f0a6d7fa640 1 -- 172.21.15.83:0/3211399519 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80009c10 con 0x561fab5c9390 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.837+0000 7f0a6d7fa640 1 -- 172.21.15.83:0/3211399519 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a80004e40 con 0x561fab5c9390 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.837+0000 7f0a6d7fa640 1 -- 172.21.15.83:0/3211399519 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80005880 con 0x561fab5c9390 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3211399519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 msgr2=0x561fab98e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a80010900 tx=0x7f0a80010fd0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3211399519 shutdown_connections 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab98eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab98e980 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/3211399519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab99cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3211399519 >> 172.21.15.83:0/3211399519 conn(0x561fab5de460 msgr2=0x561fab99c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3211399519 shutdown_connections 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.838+0000 7f0a93fc4740 1 -- 172.21.15.83:0/3211399519 wait complete. 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.839+0000 7f0a93fc4740 1 -- start start 2024-09-21T03:20:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.839+0000 7f0a93fc4740 1 Processor -- start 2024-09-21T03:20:59.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.839+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.839+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab9b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a93fc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41246/0 (socket says 172.21.15.83:41246) 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 -- 172.21.15.83:0/2493839645 learned_addr learned my addr 172.21.15.83:0/2493839645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881ae7b0 con 0x561fab5ea070 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881ae0b0 con 0x561fab5c11b0 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88073420 con 0x561fab5c9390 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab9b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a93fc4740 10 client.0 _ll_register_callbacks cb 0x7f0a78009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 -- 172.21.15.83:0/2493839645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 msgr2=0x561fab9b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab9b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 -- 172.21.15.83:0/2493839645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 msgr2=0x561fab9b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:20:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.840+0000 7f0a867fc640 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fab9b2390 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.841+0000 7f0a867fc640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a78024020 tx=0x7f0a780097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e8d4000dcaa8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.841+0000 7f0a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.841+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7801f020 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.841+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0a7801a430 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fab9058a0 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0a78029080 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 10 client.25754 fetch_fsmap learned FSMap version 38 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561faba2a950 con 0x561fab5c11b0 2024-09-21T03:20:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561fab9058a0 con 0x561fab5c11b0 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7f0a78024540 con 0x561fab5c11b0 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 10 client.25754 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 20 client.25754 populate_metadata read hostname 'smithi083' 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab57dda0 0x561faba2d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 4 client.25754 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.842+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fab902cd0 con 0x561fab57dda0 2024-09-21T03:20:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.843+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab57dda0 0x561faba2d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.845+0000 7f0a86ffd640 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab57dda0 0x561faba2d140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305dd45a7eca6692 server_cookie=4b0848e3edb31178 in_seq=0 out_seq=0 2024-09-21T03:20:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.845+0000 7f0a66ffd640 10 client.25754 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.845+0000 7f0a66ffd640 1 -- 172.21.15.83:0/2493839645 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fab902cd0 con 0x561fab57dda0 2024-09-21T03:20:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.845+0000 7f0a66ffd640 10 client.25754 handle_command_reply: tid=0 2024-09-21T03:20:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.845+0000 7f0a93fc4740 1 client.25754 shutdown 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 msgr2=0x561fab9b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a78024020 tx=0x7f0a780097d0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 shutdown_connections 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561fab57dda0 0x561faba2d140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fab5ea070 0x561fab9b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a66ffd640 0 client.25754 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:20:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fab5c9390 0x561fab9b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 --2- 172.21.15.83:0/2493839645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fab5c11b0 0x561fab9b6460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 >> 172.21.15.83:0/2493839645 conn(0x561fab5de460 msgr2=0x561fab992390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 shutdown_connections 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.846+0000 7f0a93fc4740 1 -- 172.21.15.83:0/2493839645 wait complete. 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.847+0000 7f0a93fc4740 20 client.25754 trim_cache size 0 max 16384 2024-09-21T03:20:59.849 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 msgr2=0x7f0a700b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 0x7f0a700b8cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0ccae0 tx=0x7f0a7c005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 msgr2=0x7f0a881a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0303f0 tx=0x7f0a7c032d20 comp rx=0 tx=0).stop 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 shutdown_connections 2024-09-21T03:20:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a700b67b0 0x7f0a700b8cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88110250 0x7f0a881a6140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8810f900 0x7f0a881a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 --2- 172.21.15.83:0/3784410048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a88109fc0 0x7f0a881a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 >> 172.21.15.83:0/3784410048 conn(0x7f0a880795c0 msgr2=0x7f0a88079d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 shutdown_connections 2024-09-21T03:20:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:20:59.848+0000 7f0a87fff640 1 -- 172.21.15.83:0/3784410048 wait complete. 2024-09-21T03:21:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:00 smithi193 ceph-mon[35224]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-21T03:21:00.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:00 smithi083 ceph-mon[31219]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-21T03:21:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:00 smithi142 ceph-mon[34512]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-21T03:21:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 220.0 s, 2512.6 tps, lat 35.896 ms stddev 42.371 2024-09-21T03:21:01.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:21:01.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:21:02.096 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:02.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:02 smithi083 ceph-mon[31219]: pgmap v456: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 904 op/s 2024-09-21T03:21:02.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:02 smithi193 ceph-mon[35224]: pgmap v456: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 904 op/s 2024-09-21T03:21:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:02 smithi142 ceph-mon[34512]: pgmap v456: 97 pgs: 97 active+clean; 3.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 904 op/s 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.836+0000 7fdbde104640 1 -- 172.21.15.83:0/3458327657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 msgr2=0x7fdbd81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.836+0000 7fdbde104640 1 --2- 172.21.15.83:0/3458327657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd81098e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009860 tx=0x7fdbcc02a360 comp rx=0 tx=0).stop 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 -- 172.21.15.83:0/3458327657 shutdown_connections 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 --2- 172.21.15.83:0/3458327657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8109e20 0x7fdbd8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 --2- 172.21.15.83:0/3458327657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd81098e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 --2- 172.21.15.83:0/3458327657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8103b90 0x7fdbd8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 -- 172.21.15.83:0/3458327657 >> 172.21.15.83:0/3458327657 conn(0x7fdbd80ff600 msgr2=0x7fdbd8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 -- 172.21.15.83:0/3458327657 shutdown_connections 2024-09-21T03:21:02.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.837+0000 7fdbde104640 1 -- 172.21.15.83:0/3458327657 wait complete. 2024-09-21T03:21:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.838+0000 7fdbde104640 1 Processor -- start 2024-09-21T03:21:02.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbde104640 1 -- start start 2024-09-21T03:21:02.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbde104640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbde104640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbde104640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8109e20 0x7fdbd819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbde104640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8113ae0 con 0x7fdbd8109e20 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39144/0 (socket says 172.21.15.83:39144) 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.839+0000 7fdbd77fe640 1 -- 172.21.15.83:0/2077531817 learned_addr learned my addr 172.21.15.83:0/2077531817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8113960 con 0x7fdbd81094d0 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8113c60 con 0x7fdbd8103b90 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8109e20 0x7fdbd819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd77fe640 1 -- 172.21.15.83:0/2077531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 msgr2=0x7fdbd819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd77fe640 1 -- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8109e20 msgr2=0x7fdbd819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8109e20 0x7fdbd819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.840+0000 7fdbd77fe640 1 -- 172.21.15.83:0/2077531817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81a03e0 con 0x7fdbd8103b90 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000efb0 tx=0x7fdbd000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbd4ff9640 1 -- 172.21.15.83:0/2077531817 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0020070 con 0x7fdbd8103b90 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81a06d0 con 0x7fdbd8103b90 2024-09-21T03:21:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbd4ff9640 1 -- 172.21.15.83:0/2077531817 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd0013d10 con 0x7fdbd8103b90 2024-09-21T03:21:02.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd80755f0 con 0x7fdbd8103b90 2024-09-21T03:21:02.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.841+0000 7fdbd4ff9640 1 -- 172.21.15.83:0/2077531817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000b360 con 0x7fdbd8103b90 2024-09-21T03:21:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.846+0000 7fdbe4678740 1 Processor -- start 2024-09-21T03:21:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.846+0000 7fdbd4ff9640 1 -- 172.21.15.83:0/2077531817 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdbd0023070 con 0x7fdbd8103b90 2024-09-21T03:21:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.846+0000 7fdbe4678740 1 -- start start 2024-09-21T03:21:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d016430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d016430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41276/0 (socket says 172.21.15.83:41276) 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbd77fe640 1 -- 172.21.15.83:0/2853509753 learned_addr learned my addr 172.21.15.83:0/2853509753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d016eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c98cf8ce60 con 0x55c98cc6f350 2024-09-21T03:21:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c98cf7c290 con 0x55c98cc46490 2024-09-21T03:21:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.847+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c98cf800a0 con 0x55c98cc4e670 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.848+0000 7fdbd4ff9640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 0x7fdbc00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.848+0000 7fdbd4ff9640 1 -- 172.21.15.83:0/2077531817 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdbd00ebc50 con 0x7fdbd8103b90 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2853509753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 msgr2=0x55c98d016970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2853509753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 msgr2=0x55c98d016430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d016430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2853509753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98ccda930 con 0x55c98cc4e670 2024-09-21T03:21:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.849+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.850+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 0x7fdbc00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.850+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d016eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc02a880 tx=0x7fdbcc005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88224ddee7f0c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.851+0000 7fdbbdffb640 1 -- 172.21.15.83:0/2853509753 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc041020 con 0x55c98cc4e670 2024-09-21T03:21:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.851+0000 7fdbbdffb640 1 -- 172.21.15.83:0/2853509753 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc030e50 con 0x55c98cc4e670 2024-09-21T03:21:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.851+0000 7fdbbdffb640 1 -- 172.21.15.83:0/2853509753 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037040 con 0x55c98cc4e670 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 msgr2=0x55c98d016eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 0x7fdbc00b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8013640 tx=0x7fdbc8004730 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d016eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc02a880 tx=0x7fdbcc005d90 comp rx=0 tx=0).stop 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 shutdown_connections 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d016eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d016970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2853509753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d016430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 >> 172.21.15.83:0/2853509753 conn(0x55c98cc63740 msgr2=0x55c98d014320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 shutdown_connections 2024-09-21T03:21:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.852+0000 7fdbe4678740 1 -- 172.21.15.83:0/2853509753 wait complete. 2024-09-21T03:21:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.854+0000 7fdbe4678740 1 -- start start 2024-09-21T03:21:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.854+0000 7fdbe4678740 1 Processor -- start 2024-09-21T03:21:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d02a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc4e670 0x55c98d02ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 0x55c98d036690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c98cddd890 con 0x55c98cc4e670 2024-09-21T03:21:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d02a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d02a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2024-09-21T03:21:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c98cf16d40 con 0x55c98cc46490 2024-09-21T03:21:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c98d02d360 con 0x55c98cc6f350 2024-09-21T03:21:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd77fe640 1 -- 172.21.15.83:0/4274729712 learned_addr learned my addr 172.21.15.83:0/4274729712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 0x55c98d036690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc4e670 0x55c98d02ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbe4678740 10 client.0 _ll_register_callbacks cb 0x55c98cf7ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 -- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 msgr2=0x55c98d02a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d02a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 -- 172.21.15.83:0/4274729712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc4e670 msgr2=0x55c98d02ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc4e670 0x55c98d02ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.855+0000 7fdbd7fff640 1 -- 172.21.15.83:0/4274729712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98d036bd0 con 0x55c98cc6f350 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 0x55c98d036690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8013ed0 tx=0x7fdbc80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1c57e6a130100b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8025020 con 0x55c98cc6f350 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdbc80119c0 con 0x55c98cc6f350 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc030e50 con 0x55c98cc6f350 2024-09-21T03:21:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdbc8019070 con 0x55c98cc6f350 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbe4678740 10 client.25747 fetch_fsmap learned FSMap version 38 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c98d039540 con 0x55c98cc6f350 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.857+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc030e50 con 0x55c98cc6f350 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fdbc80103d0 con 0x55c98cc6f350 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 10 client.25747 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 20 client.25747 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc2dcd0 0x55c98d03ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 4 client.25747 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c98cf8edb0 con 0x55c98cc2dcd0 2024-09-21T03:21:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.858+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc2dcd0 0x55c98d03ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.859+0000 7fdbb6ffd640 10 client.25747 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.859+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc2dcd0 0x55c98d03ca10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5db0ae207e6893d server_cookie=801c34c9772c6c0d in_seq=0 out_seq=0 2024-09-21T03:21:02.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.861+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/4274729712 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c98cf8edb0 con 0x55c98cc2dcd0 2024-09-21T03:21:02.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.861+0000 7fdbb6ffd640 10 client.25747 handle_command_reply: tid=0 2024-09-21T03:21:02.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.861+0000 7fdbe4678740 1 client.25747 shutdown 2024-09-21T03:21:02.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 msgr2=0x55c98d036690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 0x55c98d036690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8013ed0 tx=0x7fdbc80046c0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 shutdown_connections 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc2dcd0 0x55c98d03ca10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc6f350 0x55c98d036690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc4e670 0x55c98d02ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 --2- 172.21.15.83:0/4274729712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d02a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 >> 172.21.15.83:0/4274729712 conn(0x55c98cc63740 msgr2=0x55c98d012850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.862+0000 7fdbb6ffd640 0 client.25747 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:02.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.863+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 shutdown_connections 2024-09-21T03:21:02.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.863+0000 7fdbe4678740 1 -- 172.21.15.83:0/4274729712 wait complete. 2024-09-21T03:21:02.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.863+0000 7fdbe4678740 20 client.25747 trim_cache size 0 max 16384 2024-09-21T03:21:02.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.885+0000 7fdbe4678740 1 Processor -- start 2024-09-21T03:21:02.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.885+0000 7fdbe4678740 1 -- start start 2024-09-21T03:21:02.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.885+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d012f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.885+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.885+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d027820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbe4678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ad350 con 0x55c98cc6f350 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbe4678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81aeef0 con 0x55c98cc46490 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbe4678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ada20 con 0x55c98cc4e670 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d027820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39176/0 (socket says 172.21.15.83:39176) 2024-09-21T03:21:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2327485433 learned_addr learned my addr 172.21.15.83:0/2327485433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2327485433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 msgr2=0x55c98d012f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d012f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2327485433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 msgr2=0x55c98d027820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d027820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.886+0000 7fdbd6ffd640 1 -- 172.21.15.83:0/2327485433 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98d02d250 con 0x55c98cc4e670 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.887+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc030de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593628f1809ccc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.887+0000 7fdbbd7fa640 1 -- 172.21.15.83:0/2327485433 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc041040 con 0x55c98cc4e670 2024-09-21T03:21:02.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.887+0000 7fdbbd7fa640 1 -- 172.21.15.83:0/2327485433 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc030900 con 0x55c98cc4e670 2024-09-21T03:21:02.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.887+0000 7fdbbd7fa640 1 -- 172.21.15.83:0/2327485433 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037040 con 0x55c98cc4e670 2024-09-21T03:21:02.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- 172.21.15.83:0/2327485433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 msgr2=0x55c98d029ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc030de0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- 172.21.15.83:0/2327485433 shutdown_connections 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d027820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc4e670 0x55c98d029ec0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2327485433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc46490 0x55c98d012f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- 172.21.15.83:0/2327485433 >> 172.21.15.83:0/2327485433 conn(0x55c98cc63740 msgr2=0x55c98d012850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- 172.21.15.83:0/2327485433 shutdown_connections 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- 172.21.15.83:0/2327485433 wait complete. 2024-09-21T03:21:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.889+0000 7fdbe4678740 1 -- start start 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.890+0000 7fdbe4678740 1 Processor -- start 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc46490 0x55c98d032b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ae820 con 0x55c98cc6f350 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ae120 con 0x55c98cc4e670 2024-09-21T03:21:02.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81ab330 con 0x55c98cc46490 2024-09-21T03:21:02.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbe4678740 10 client.0 _ll_register_callbacks cb 0x7fdbc800bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60128/0 (socket says 172.21.15.83:60128) 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 -- 172.21.15.83:0/2997966353 learned_addr learned my addr 172.21.15.83:0/2997966353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41320/0 (socket says 172.21.15.83:41320) 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc46490 0x55c98d032b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 -- 172.21.15.83:0/2997966353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc46490 msgr2=0x55c98d032b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc46490 0x55c98d032b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 -- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 msgr2=0x55c98d02f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd7fff640 1 -- 172.21.15.83:0/2997966353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98d02faa0 con 0x55c98cc6f350 2024-09-21T03:21:02.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.891+0000 7fdbd6ffd640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbd7fff640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80108c0 tx=0x7fdbc8010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad2876b3c29f466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8038070 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc030900 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fdbc8011bd0 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdbc8025050 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbe4678740 10 client.16236 fetch_fsmap learned FSMap version 38 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.892+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c98d0a8070 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc030900 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7fdbc803b030 con 0x55c98cc6f350 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 10 client.16236 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 20 client.16236 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc03080 0x55c98d0aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 4 client.16236 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:21:02.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c98cf8a860 con 0x55c98cc03080 2024-09-21T03:21:02.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.893+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc03080 0x55c98d0aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.894+0000 7fdbd77fe640 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc03080 0x55c98d0aa7d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d6ec53a111c0e9 server_cookie=561b5e14650f0e59 in_seq=0 out_seq=0 2024-09-21T03:21:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.894+0000 7fdbb6ffd640 10 client.16236 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbb6ffd640 1 -- 172.21.15.83:0/2997966353 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c98cf8a860 con 0x55c98cc03080 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbb6ffd640 10 client.16236 handle_command_reply: tid=0 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 client.16236 shutdown 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 msgr2=0x55c98d02f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80108c0 tx=0x7fdbc8010680 comp rx=0 tx=0).stop 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 shutdown_connections 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c98cc03080 0x55c98d0aa7d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c98cc6f350 0x55c98d02f560 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c98cc4e670 0x55c98d02f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 --2- 172.21.15.83:0/2997966353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c98cc46490 0x55c98d032b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 >> 172.21.15.83:0/2997966353 conn(0x55c98cc63740 msgr2=0x55c98d017460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbb6ffd640 0 client.16236 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 shutdown_connections 2024-09-21T03:21:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 1 -- 172.21.15.83:0/2997966353 wait complete. 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.896+0000 7fdbe4678740 20 client.16236 trim_cache size 0 max 16384 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "b80ab80a-2e5b-4614-b429-33354044a838", 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:21:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 msgr2=0x7fdbc00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 0x7fdbc00b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8013640 tx=0x7fdbc8004730 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 msgr2=0x7fdbd819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000efb0 tx=0x7fdbd000c800 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 shutdown_connections 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdbc00b6700 0x7fdbc00b8c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8109e20 0x7fdbd819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbd81094d0 0x7fdbd819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 --2- 172.21.15.83:0/2077531817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd8103b90 0x7fdbd819a470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 >> 172.21.15.83:0/2077531817 conn(0x7fdbd80ff600 msgr2=0x7fdbd8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 shutdown_connections 2024-09-21T03:21:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:02.897+0000 7fdbde104640 1 -- 172.21.15.83:0/2077531817 wait complete. 2024-09-21T03:21:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:21:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:03 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:03 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:03 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:03.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:21:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:04 smithi083 ceph-mon[31219]: pgmap v457: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2024-09-21T03:21:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:04 smithi142 ceph-mon[34512]: pgmap v457: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:04 smithi193 ceph-mon[35224]: pgmap v457: 97 pgs: 97 active+clean; 3.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2024-09-21T03:21:04.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 -- 172.21.15.83:0/1046161981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 msgr2=0x7fc86c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 --2- 172.21.15.83:0/1046161981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c103f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc860009900 tx=0x7fc86002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 -- 172.21.15.83:0/1046161981 shutdown_connections 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 --2- 172.21.15.83:0/1046161981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 0x7fc86c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 --2- 172.21.15.83:0/1046161981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 --2- 172.21.15.83:0/1046161981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c103f90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.953+0000 7fc871775640 1 -- 172.21.15.83:0/1046161981 >> 172.21.15.83:0/1046161981 conn(0x7fc86c07b8d0 msgr2=0x7fc86c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.954+0000 7fc871775640 1 -- 172.21.15.83:0/1046161981 shutdown_connections 2024-09-21T03:21:04.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.954+0000 7fc871775640 1 -- 172.21.15.83:0/1046161981 wait complete. 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.954+0000 7fc871775640 1 Processor -- start 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 -- start start 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 0x7fc86c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113ad0 con 0x7fc86c1094c0 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113950 con 0x7fc86c103b80 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc871775640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113c50 con 0x7fc86c109e10 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc86a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc86a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60132/0 (socket says 172.21.15.83:60132) 2024-09-21T03:21:04.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc86a7fc640 1 -- 172.21.15.83:0/355722097 learned_addr learned my addr 172.21.15.83:0/355722097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc86affd640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.955+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 0x7fc86c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86a7fc640 1 -- 172.21.15.83:0/355722097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 msgr2=0x7fc86c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 0x7fc86c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86a7fc640 1 -- 172.21.15.83:0/355722097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 msgr2=0x7fc86c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86a7fc640 1 -- 172.21.15.83:0/355722097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c1a8860 con 0x7fc86c1094c0 2024-09-21T03:21:04.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.956+0000 7fc86affd640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:04.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.957+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e970 tx=0x7fc85c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.958+0000 7fc853fff640 1 -- 172.21.15.83:0/355722097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cd80 con 0x7fc86c1094c0 2024-09-21T03:21:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.958+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a8b50 con 0x7fc86c1094c0 2024-09-21T03:21:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.958+0000 7fc853fff640 1 -- 172.21.15.83:0/355722097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c0035f0 con 0x7fc86c1094c0 2024-09-21T03:21:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.958+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a9090 con 0x7fc86c1094c0 2024-09-21T03:21:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.958+0000 7fc853fff640 1 -- 172.21.15.83:0/355722097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cd80 con 0x7fc86c1094c0 2024-09-21T03:21:04.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.961+0000 7fc877ce9740 1 Processor -- start 2024-09-21T03:21:04.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.961+0000 7fc877ce9740 1 -- start start 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c844eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c84460e50 con 0x559c84143070 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c84450280 con 0x559c84122390 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc877ce9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c84454090 con 0x559c8411a1b0 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60140/0 (socket says 172.21.15.83:60140) 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86b7fe640 1 -- 172.21.15.83:0/3750216527 learned_addr learned my addr 172.21.15.83:0/3750216527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc853fff640 1 -- 172.21.15.83:0/355722097 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc85c004000 con 0x7fc86c1094c0 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 -- 172.21.15.83:0/3750216527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 msgr2=0x559c844eacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c844eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c844eacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 -- 172.21.15.83:0/3750216527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 msgr2=0x559c844ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86affd640 1 -- 172.21.15.83:0/3750216527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c841ae650 con 0x559c8411a1b0 2024-09-21T03:21:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.962+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.963+0000 7fc86affd640 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844ea7b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc86c077160 tx=0x7fc860005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f878263f4e515ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.963+0000 7fc853fff640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 0x7fc8540b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.963+0000 7fc8517fa640 1 -- 172.21.15.83:0/3750216527 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860040070 con 0x559c8411a1b0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.963+0000 7fc8517fa640 1 -- 172.21.15.83:0/3750216527 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860031720 con 0x559c8411a1b0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.964+0000 7fc853fff640 1 -- 172.21.15.83:0/355722097 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc85c0e75a0 con 0x7fc86c1094c0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.964+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 0x7fc8540b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.964+0000 7fc8517fa640 1 -- 172.21.15.83:0/3750216527 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860035040 con 0x559c8411a1b0 2024-09-21T03:21:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 -- 172.21.15.83:0/3750216527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 msgr2=0x559c844ea7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844ea7b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc86c077160 tx=0x7fc860005e20 comp rx=0 tx=0).stop 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 -- 172.21.15.83:0/3750216527 shutdown_connections 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c844eacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844ea7b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3750216527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84143070 0x559c844ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 -- 172.21.15.83:0/3750216527 >> 172.21.15.83:0/3750216527 conn(0x559c84137460 msgr2=0x559c844e8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 0x7fc8540b8d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc864009fd0 tx=0x7fc864009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.965+0000 7fc877ce9740 1 -- 172.21.15.83:0/3750216527 shutdown_connections 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.966+0000 7fc877ce9740 1 -- 172.21.15.83:0/3750216527 wait complete. 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.966+0000 7fc877ce9740 1 -- start start 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.966+0000 7fc877ce9740 1 Processor -- start 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84122390 0x559c844f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 0x559c84501d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c842b1700 con 0x559c84122390 2024-09-21T03:21:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c843eb110 con 0x559c84143070 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c844f8a40 con 0x559c8411a1b0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39234/0 (socket says 172.21.15.83:39234) 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 -- 172.21.15.83:0/124294348 learned_addr learned my addr 172.21.15.83:0/124294348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 0x559c84501d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc877ce9740 10 client.0 _ll_register_callbacks cb 0x559c84404990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84122390 0x559c844f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 -- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 msgr2=0x559c84501d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 0x559c84501d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 -- 172.21.15.83:0/124294348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84122390 msgr2=0x559c844f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84122390 0x559c844f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86affd640 1 -- 172.21.15.83:0/124294348 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c84418c40 con 0x559c8411a1b0 2024-09-21T03:21:04.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.967+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 0x559c84501d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc86affd640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc860004620 tx=0x7fc86002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2eaca087d46674 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc84eff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860053060 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc86003c650 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc860031720 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc86004a050 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.968+0000 7fc877ce9740 10 client.25765 fetch_fsmap learned FSMap version 39 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c84504720 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86003cd40 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc860058ac0 con 0x559c8411a1b0 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 10 client.25765 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 20 client.25765 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c841019f0 0x559c84507a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 4 client.25765 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.969+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc860031720 con 0x559c841019f0 2024-09-21T03:21:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.970+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c841019f0 0x559c84507a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.970+0000 7fc84eff5640 10 client.25765 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.970+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c841019f0 0x559c84507a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9383b6030a3888b9 server_cookie=938f7506905c8a48 in_seq=0 out_seq=0 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.971+0000 7fc84eff5640 1 -- 172.21.15.83:0/124294348 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc860031720 con 0x559c841019f0 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.971+0000 7fc84eff5640 10 client.25765 handle_command_reply: tid=0 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.971+0000 7fc877ce9740 1 client.25765 shutdown 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.971+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 msgr2=0x559c844f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.971+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc860004620 tx=0x7fc86002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 shutdown_connections 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c841019f0 0x559c84507a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc84eff5640 0 client.25765 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84143070 0x559c84501d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c84122390 0x559c844f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 --2- 172.21.15.83:0/124294348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c844f58d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.973+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 >> 172.21.15.83:0/124294348 conn(0x559c84137460 msgr2=0x559c844ed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:04.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.974+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 shutdown_connections 2024-09-21T03:21:04.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.974+0000 7fc877ce9740 1 -- 172.21.15.83:0/124294348 wait complete. 2024-09-21T03:21:04.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.974+0000 7fc877ce9740 20 client.25765 trim_cache size 0 max 16384 2024-09-21T03:21:05.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.999+0000 7fc877ce9740 1 Processor -- start 2024-09-21T03:21:05.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:04.999+0000 7fc877ce9740 1 -- start start 2024-09-21T03:21:05.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 0x559c844f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 0x559c844e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 0x559c844f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 0x559c844e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 0x559c844e8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60154/0 (socket says 172.21.15.83:60154) 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 0x559c844f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39238/0 (socket says 172.21.15.83:39238) 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41372/0 (socket says 172.21.15.83:41372) 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc86a7fc640 1 -- 172.21.15.83:0/2243459849 learned_addr learned my addr 172.21.15.83:0/2243459849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c075390 con 0x559c8411a1b0 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1aacd0 con 0x559c84122390 2024-09-21T03:21:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.000+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1a97a0 con 0x559c841019f0 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 -- 172.21.15.83:0/2243459849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 msgr2=0x559c844f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 0x559c844f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 -- 172.21.15.83:0/2243459849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 msgr2=0x559c844e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 0x559c844e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 -- 172.21.15.83:0/2243459849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c844ec790 con 0x559c84122390 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc86400f790 tx=0x7fc864016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2b5ab793f835fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc850ff9640 1 -- 172.21.15.83:0/2243459849 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86400fe10 con 0x559c84122390 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc850ff9640 1 -- 172.21.15.83:0/2243459849 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc864003970 con 0x559c84122390 2024-09-21T03:21:05.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.001+0000 7fc850ff9640 1 -- 172.21.15.83:0/2243459849 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864004030 con 0x559c84122390 2024-09-21T03:21:05.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 msgr2=0x559c84501cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc86400f790 tx=0x7fc864016000 comp rx=0 tx=0).stop 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 shutdown_connections 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84501cf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c8411a1b0 0x559c844e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 --2- 172.21.15.83:0/2243459849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c841019f0 0x559c844f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 >> 172.21.15.83:0/2243459849 conn(0x559c84137460 msgr2=0x559c844e7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 shutdown_connections 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.002+0000 7fc877ce9740 1 -- 172.21.15.83:0/2243459849 wait complete. 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 -- start start 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 Processor -- start 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c841019f0 0x559c8450b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c84508560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1aa600 con 0x559c841019f0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1a9ea0 con 0x559c84122390 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c073370 con 0x559c8411a1b0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc877ce9740 10 client.0 _ll_register_callbacks cb 0x7fc864009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41384/0 (socket says 172.21.15.83:41384) 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc86b7fe640 1 -- 172.21.15.83:0/3128435028 learned_addr learned my addr 172.21.15.83:0/3128435028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc86affd640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c841019f0 0x559c8450b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.003+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c84508560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 -- 172.21.15.83:0/3128435028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 msgr2=0x559c84508560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c84508560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 -- 172.21.15.83:0/3128435028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c841019f0 msgr2=0x559c8450b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:05.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c841019f0 0x559c8450b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 -- 172.21.15.83:0/3128435028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c84508fe0 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86a7fc640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c84508560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc86b7fe640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc86402a040 tx=0x7fc864004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ecc3ff02bc350bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc84eff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864035020 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc864005410 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc864003970 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc864040050 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc877ce9740 10 client.25784 fetch_fsmap learned FSMap version 39 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.004+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c84571f00 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc864003970 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc8640319b0 con 0x559c84122390 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 10 client.25784 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T03:21:05.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:05.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 20 client.25784 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:05.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c84143070 0x559c84574650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:05.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 4 client.25784 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:21:05.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c84462da0 con 0x559c84143070 2024-09-21T03:21:05.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc86affd640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c84143070 0x559c84574650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.005+0000 7fc84eff5640 10 client.25784 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.006+0000 7fc86affd640 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c84143070 0x559c84574650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f87ba6b31a25e93 server_cookie=e0196e2c4abe14b9 in_seq=0 out_seq=0 2024-09-21T03:21:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.006+0000 7fc84eff5640 1 -- 172.21.15.83:0/3128435028 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x559c84462da0 con 0x559c84143070 2024-09-21T03:21:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.006+0000 7fc84eff5640 10 client.25784 handle_command_reply: tid=0 2024-09-21T03:21:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.006+0000 7fc877ce9740 1 client.25784 shutdown 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 msgr2=0x559c84508aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc86402a040 tx=0x7fc864004f20 comp rx=0 tx=0).stop 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 shutdown_connections 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559c84143070 0x559c84574650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c84122390 0x559c84508aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc84eff5640 0 client.25784 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c8411a1b0 0x559c84508560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 --2- 172.21.15.83:0/3128435028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c841019f0 0x559c8450b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 >> 172.21.15.83:0/3128435028 conn(0x559c84137460 msgr2=0x559c844ed370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.007+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 shutdown_connections 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc877ce9740 1 -- 172.21.15.83:0/3128435028 wait complete. 2024-09-21T03:21:05.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc877ce9740 20 client.25784 trim_cache size 0 max 16384 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "b80ab80a-2e5b-4614-b429-33354044a838", 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:21:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:05.017 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 msgr2=0x7fc8540b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 0x7fc8540b8d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc864009fd0 tx=0x7fc864009500 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 msgr2=0x7fc86c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e970 tx=0x7fc85c00ee30 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.008+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 shutdown_connections 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.009+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc8540b6880 0x7fc8540b8d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.009+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc86c109e10 0x7fc86c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.009+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc86c1094c0 0x7fc86c075c50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.009+0000 7fc871775640 1 --2- 172.21.15.83:0/355722097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc86c103b80 0x7fc86c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:05.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.009+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 >> 172.21.15.83:0/355722097 conn(0x7fc86c07b8d0 msgr2=0x7fc86c107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:05.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.010+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 shutdown_connections 2024-09-21T03:21:05.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:05.010+0000 7fc871775640 1 -- 172.21.15.83:0/355722097 wait complete. 2024-09-21T03:21:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:05 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:05 smithi083 ceph-mon[31219]: pgmap v458: 97 pgs: 97 active+clean; 3.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 25 MiB/s wr, 1.73k op/s 2024-09-21T03:21:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:05 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:05 smithi142 ceph-mon[34512]: pgmap v458: 97 pgs: 97 active+clean; 3.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 25 MiB/s wr, 1.73k op/s 2024-09-21T03:21:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:05 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:05 smithi193 ceph-mon[35224]: pgmap v458: 97 pgs: 97 active+clean; 3.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 25 MiB/s wr, 1.73k op/s 2024-09-21T03:21:05.940 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 225.0 s, 3251.3 tps, lat 34.054 ms stddev 75.071 2024-09-21T03:21:05.971 INFO:tasks.cephfs.filesystem:scrub status for tag:b80ab80a-2e5b-4614-b429-33354044a838 - {'path': '/', 'tag': 'b80ab80a-2e5b-4614-b429-33354044a838', 'options': 'recursive,force'} 2024-09-21T03:21:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:07 smithi142 ceph-mon[34512]: pgmap v459: 97 pgs: 97 active+clean; 3.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2024-09-21T03:21:07.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:07 smithi193 ceph-mon[35224]: pgmap v459: 97 pgs: 97 active+clean; 3.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2024-09-21T03:21:07.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:07 smithi083 ceph-mon[31219]: pgmap v459: 97 pgs: 97 active+clean; 3.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2024-09-21T03:21:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:08 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:21:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:08 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:21:08.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:08 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:21:08.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:08 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:21:08.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:08 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:21:08.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:08 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:21:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:21:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:21:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:21:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:09 smithi142 ceph-mon[34512]: pgmap v460: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 32 MiB/s wr, 2.10k op/s 2024-09-21T03:21:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:21:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:21:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:21:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:09 smithi193 ceph-mon[35224]: pgmap v460: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 32 MiB/s wr, 2.10k op/s 2024-09-21T03:21:09.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:21:09.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:21:09.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:21:09.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:09 smithi083 ceph-mon[31219]: pgmap v460: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 32 MiB/s wr, 2.10k op/s 2024-09-21T03:21:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:10 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:10.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:10 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:10 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:21:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 230.0 s, 3548.4 tps, lat 28.221 ms stddev 31.190 2024-09-21T03:21:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:11 smithi142 ceph-mon[34512]: pgmap v461: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-21T03:21:11.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:11 smithi193 ceph-mon[35224]: pgmap v461: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-21T03:21:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:11 smithi083 ceph-mon[31219]: pgmap v461: 97 pgs: 97 active+clean; 3.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2024-09-21T03:21:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:13 smithi142 ceph-mon[34512]: pgmap v462: 97 pgs: 97 active+clean; 3.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 30 MiB/s wr, 1.94k op/s 2024-09-21T03:21:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:13 smithi193 ceph-mon[35224]: pgmap v462: 97 pgs: 97 active+clean; 3.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 30 MiB/s wr, 1.94k op/s 2024-09-21T03:21:13.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:13 smithi083 ceph-mon[31219]: pgmap v462: 97 pgs: 97 active+clean; 3.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 30 MiB/s wr, 1.94k op/s 2024-09-21T03:21:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:15 smithi142 ceph-mon[34512]: pgmap v463: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-09-21T03:21:15.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:15 smithi193 ceph-mon[35224]: pgmap v463: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-09-21T03:21:15.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:15 smithi083 ceph-mon[31219]: pgmap v463: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-09-21T03:21:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 235.0 s, 3071.6 tps, lat 32.418 ms stddev 35.015 2024-09-21T03:21:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:17 smithi142 ceph-mon[34512]: pgmap v464: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 933 op/s 2024-09-21T03:21:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:17 smithi193 ceph-mon[35224]: pgmap v464: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 933 op/s 2024-09-21T03:21:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:17 smithi083 ceph-mon[31219]: pgmap v464: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 933 op/s 2024-09-21T03:21:19.673 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:19 smithi083 ceph-mon[31219]: pgmap v465: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-21T03:21:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:19 smithi142 ceph-mon[34512]: pgmap v465: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-21T03:21:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:19 smithi193 ceph-mon[35224]: pgmap v465: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-21T03:21:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 240.0 s, 2607.2 tps, lat 38.546 ms stddev 42.644 2024-09-21T03:21:21.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:21 smithi083 ceph-mon[31219]: pgmap v466: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2024-09-21T03:21:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:21 smithi142 ceph-mon[34512]: pgmap v466: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2024-09-21T03:21:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:21 smithi193 ceph-mon[35224]: pgmap v466: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2024-09-21T03:21:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:23 smithi142 ceph-mon[34512]: pgmap v467: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 752 op/s 2024-09-21T03:21:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:23 smithi193 ceph-mon[35224]: pgmap v467: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 752 op/s 2024-09-21T03:21:23.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:23 smithi083 ceph-mon[31219]: pgmap v467: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.0 MiB/s wr, 752 op/s 2024-09-21T03:21:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:25 smithi142 ceph-mon[34512]: pgmap v468: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 649 op/s 2024-09-21T03:21:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:25 smithi193 ceph-mon[35224]: pgmap v468: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 649 op/s 2024-09-21T03:21:25.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:25 smithi083 ceph-mon[31219]: pgmap v468: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 649 op/s 2024-09-21T03:21:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 245.0 s, 2491.2 tps, lat 40.071 ms stddev 43.018 2024-09-21T03:21:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:27 smithi142 ceph-mon[34512]: pgmap v469: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-21T03:21:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:27 smithi193 ceph-mon[35224]: pgmap v469: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-21T03:21:27.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:27 smithi083 ceph-mon[31219]: pgmap v469: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-21T03:21:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:29 smithi142 ceph-mon[34512]: pgmap v470: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-09-21T03:21:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:29 smithi193 ceph-mon[35224]: pgmap v470: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-09-21T03:21:29.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:29 smithi083 ceph-mon[31219]: pgmap v470: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-09-21T03:21:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 250.0 s, 2921.6 tps, lat 34.083 ms stddev 36.558 2024-09-21T03:21:31.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:31 smithi142 ceph-mon[34512]: pgmap v471: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-21T03:21:31.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:31 smithi083 ceph-mon[31219]: pgmap v471: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-21T03:21:31.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:31 smithi193 ceph-mon[35224]: pgmap v471: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 702 op/s 2024-09-21T03:21:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:33 smithi142 ceph-mon[34512]: pgmap v472: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.5 MiB/s wr, 794 op/s 2024-09-21T03:21:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:33 smithi193 ceph-mon[35224]: pgmap v472: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.5 MiB/s wr, 794 op/s 2024-09-21T03:21:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:33 smithi083 ceph-mon[31219]: pgmap v472: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.5 MiB/s wr, 794 op/s 2024-09-21T03:21:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:35 smithi142 ceph-mon[34512]: pgmap v473: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 706 op/s 2024-09-21T03:21:35.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:35 smithi193 ceph-mon[35224]: pgmap v473: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 706 op/s 2024-09-21T03:21:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:35 smithi083 ceph-mon[31219]: pgmap v473: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 706 op/s 2024-09-21T03:21:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 255.0 s, 3185.6 tps, lat 31.602 ms stddev 33.641 2024-09-21T03:21:35.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:21:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 -- 172.21.15.83:0/582914067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310105270 msgr2=0x7f531010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 --2- 172.21.15.83:0/582914067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310105270 0x7f531010d8b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5300009900 tx=0x7f530002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 -- 172.21.15.83:0/582914067 shutdown_connections 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 --2- 172.21.15.83:0/582914067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f5310110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 --2- 172.21.15.83:0/582914067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310105270 0x7f531010d8b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 --2- 172.21.15.83:0/582914067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310104840 0x7f5310104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.966+0000 7f53186b7640 1 -- 172.21.15.83:0/582914067 >> 172.21.15.83:0/582914067 conn(0x7f5310079730 msgr2=0x7f5310079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:36.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.967+0000 7f53186b7640 1 -- 172.21.15.83:0/582914067 shutdown_connections 2024-09-21T03:21:36.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.967+0000 7f53186b7640 1 -- 172.21.15.83:0/582914067 wait complete. 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 Processor -- start 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 -- start start 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310104840 0x7f531019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310105270 0x7f531019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5310113ac0 con 0x7f5310104840 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5310113940 con 0x7f531010ddf0 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f53186b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5310113c40 con 0x7f5310105270 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58828/0 (socket says 172.21.15.83:58828) 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 -- 172.21.15.83:0/1351895882 learned_addr learned my addr 172.21.15.83:0/1351895882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316453640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310104840 0x7f531019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5315c52640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310105270 0x7f531019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 -- 172.21.15.83:0/1351895882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310105270 msgr2=0x7f531019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310105270 0x7f531019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 -- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310104840 msgr2=0x7f531019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310104840 0x7f531019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.969+0000 7f5316c54640 1 -- 172.21.15.83:0/1351895882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53101a0440 con 0x7f531010ddf0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f5316c54640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f530800eb10 tx=0x7f530800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f53077fe640 1 -- 172.21.15.83:0/1351895882 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308004b20 con 0x7f531010ddf0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f53077fe640 1 -- 172.21.15.83:0/1351895882 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5308004cc0 con 0x7f531010ddf0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53101a0730 con 0x7f531010ddf0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f53077fe640 1 -- 172.21.15.83:0/1351895882 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308010670 con 0x7f531010ddf0 2024-09-21T03:21:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.970+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101a6e90 con 0x7f531010ddf0 2024-09-21T03:21:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.973+0000 7f53077fe640 1 -- 172.21.15.83:0/1351895882 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5308005020 con 0x7f531010ddf0 2024-09-21T03:21:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.973+0000 7f531ec2b740 1 Processor -- start 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 -- start start 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c130a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c130b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594c1280e50 con 0x5594c0f3a1b0 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594c1270280 con 0x5594c0f42390 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f531ec2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594c1274090 con 0x5594c0f63070 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f5316453640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f5316453640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46516/0 (socket says 172.21.15.83:46516) 2024-09-21T03:21:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f5316453640 1 -- 172.21.15.83:0/1558798186 learned_addr learned my addr 172.21.15.83:0/1558798186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f5315c52640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c130b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.974+0000 7f5315c52640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c130a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 -- 172.21.15.83:0/1558798186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 msgr2=0x5594c130a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c130a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 -- 172.21.15.83:0/1558798186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 msgr2=0x5594c130b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c130b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 -- 172.21.15.83:0/1558798186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c0fce650 con 0x5594c0f3a1b0 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f53077fe640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 0x7f52fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5315c52640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c130a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5316453640 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f530c00bcb0 tx=0x7f530c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c37cf44b75ada6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5315c52640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 0x7f52fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5304ff9640 1 -- 172.21.15.83:0/1558798186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c01c480 con 0x5594c0f3a1b0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f53077fe640 1 -- 172.21.15.83:0/1351895882 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f53080e7c70 con 0x7f531010ddf0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5304ff9640 1 -- 172.21.15.83:0/1558798186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c01cad0 con 0x5594c0f3a1b0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.975+0000 7f5304ff9640 1 -- 172.21.15.83:0/1558798186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c011bc0 con 0x5594c0f3a1b0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.976+0000 7f531ec2b740 1 -- 172.21.15.83:0/1558798186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 msgr2=0x5594c130ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.976+0000 7f531ec2b740 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f530c00bcb0 tx=0x7f530c0107a0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f5315c52640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 0x7f52fc0b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f530002aa00 tx=0x7f5300041040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 -- 172.21.15.83:0/1558798186 shutdown_connections 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c130b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c130ab90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 --2- 172.21.15.83:0/1558798186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c130a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 -- 172.21.15.83:0/1558798186 >> 172.21.15.83:0/1558798186 conn(0x5594c0f57460 msgr2=0x5594c1308540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 -- 172.21.15.83:0/1558798186 shutdown_connections 2024-09-21T03:21:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 -- 172.21.15.83:0/1558798186 wait complete. 2024-09-21T03:21:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.977+0000 7f531ec2b740 1 -- start start 2024-09-21T03:21:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 Processor -- start 2024-09-21T03:21:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c1320330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c1317fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594c10d1700 con 0x5594c0f3a1b0 2024-09-21T03:21:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594c120b110 con 0x5594c0f42390 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594c1321550 con 0x5594c0f63070 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.978+0000 7f531ec2b740 10 client.0 _ll_register_callbacks cb 0x5594c1224990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58842/0 (socket says 172.21.15.83:58842) 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 -- 172.21.15.83:0/2176496404 learned_addr learned my addr 172.21.15.83:0/2176496404 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5316c54640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c1317fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 -- 172.21.15.83:0/2176496404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 msgr2=0x5594c1317fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c1317fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 -- 172.21.15.83:0/2176496404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 msgr2=0x5594c1320330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c1320330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 -- 172.21.15.83:0/2176496404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c1238c40 con 0x5594c0f42390 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.979+0000 7f5315c52640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5300004650 tx=0x7f530002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd94e369fd6cd0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f52f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300050070 con 0x5594c0f42390 2024-09-21T03:21:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c01cad0 con 0x5594c0f42390 2024-09-21T03:21:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f53000316a0 con 0x5594c0f42390 2024-09-21T03:21:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f530005c050 con 0x5594c0f42390 2024-09-21T03:21:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f531ec2b740 10 client.25796 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5594c1313b40 con 0x5594c0f42390 2024-09-21T03:21:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.980+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c01cad0 con 0x5594c0f42390 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f530003d8c0 con 0x5594c0f42390 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 10 client.25796 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 20 client.25796 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0f219f0 0x5594c13395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 4 client.25796 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c1282da0 con 0x5594c0f219f0 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.981+0000 7f5316453640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0f219f0 0x5594c13395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.982+0000 7f52f9ffb640 10 client.25796 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.982+0000 7f5316453640 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0f219f0 0x5594c13395a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862e4bdbfa35d6c6 server_cookie=4a4ecf934f018c4f in_seq=0 out_seq=0 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2176496404 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594c1282da0 con 0x5594c0f219f0 2024-09-21T03:21:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f52f9ffb640 10 client.25796 handle_command_reply: tid=0 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 client.25796 shutdown 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 msgr2=0x5594c1320910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5300004650 tx=0x7f530002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 shutdown_connections 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0f219f0 0x5594c13395a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f63070 0x5594c1317fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f42390 0x5594c1320910 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5300004650 tx=0x7f530002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2176496404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c1320330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 >> 172.21.15.83:0/2176496404 conn(0x5594c0f57460 msgr2=0x5594c130d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f52f9ffb640 0 client.25796 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 shutdown_connections 2024-09-21T03:21:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 1 -- 172.21.15.83:0/2176496404 wait complete. 2024-09-21T03:21:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:36.983+0000 7f531ec2b740 20 client.25796 trim_cache size 0 max 16384 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.009+0000 7f531ec2b740 1 Processor -- start 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.009+0000 7f531ec2b740 1 -- start start 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 0x5594c131d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f3a1b0 0x5594c131da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ad360 con 0x5594c0f57460 2024-09-21T03:21:37.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53101aef90 con 0x5594c0f63070 2024-09-21T03:21:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f531ec2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ada30 con 0x5594c0f3a1b0 2024-09-21T03:21:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f5316c54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f5316c54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58856/0 (socket says 172.21.15.83:58856) 2024-09-21T03:21:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f5316c54640 1 -- 172.21.15.83:0/2720156061 learned_addr learned my addr 172.21.15.83:0/2720156061 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f5316453640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 0x5594c131d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.010+0000 7f5315c52640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f3a1b0 0x5594c131da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316c54640 1 -- 172.21.15.83:0/2720156061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f3a1b0 msgr2=0x5594c131da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316c54640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f3a1b0 0x5594c131da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316c54640 1 -- 172.21.15.83:0/2720156061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 msgr2=0x5594c131d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316c54640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 0x5594c131d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316c54640 1 -- 172.21.15.83:0/2720156061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c1308e30 con 0x5594c0f63070 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.011+0000 7f5316453640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 0x5594c131d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.012+0000 7f5316c54640 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53080049d0 tx=0x7f530800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2930c3765c1ed14e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.012+0000 7f52fbfff640 1 -- 172.21.15.83:0/2720156061 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53080ad2b0 con 0x5594c0f63070 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.012+0000 7f52fbfff640 1 -- 172.21.15.83:0/2720156061 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53080ad650 con 0x5594c0f63070 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f52fbfff640 1 -- 172.21.15.83:0/2720156061 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53080b2200 con 0x5594c0f63070 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 -- 172.21.15.83:0/2720156061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 msgr2=0x5594c130d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53080049d0 tx=0x7f530800cee0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 -- 172.21.15.83:0/2720156061 shutdown_connections 2024-09-21T03:21:37.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c130d750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f3a1b0 0x5594c131da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2720156061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f57460 0x5594c131d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.013+0000 7f531ec2b740 1 -- 172.21.15.83:0/2720156061 >> 172.21.15.83:0/2720156061 conn(0x5594c13186a0 msgr2=0x5594c131e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:37.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.015+0000 7f531ec2b740 1 -- 172.21.15.83:0/2720156061 shutdown_connections 2024-09-21T03:21:37.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.015+0000 7f531ec2b740 1 -- 172.21.15.83:0/2720156061 wait complete. 2024-09-21T03:21:37.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.015+0000 7f531ec2b740 1 -- start start 2024-09-21T03:21:37.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 Processor -- start 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c134b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c1347310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ae8c0 con 0x5594c0f3a1b0 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ae130 con 0x5594c0f63070 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53101ab360 con 0x5594c0f57460 2024-09-21T03:21:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f531ec2b740 10 client.0 _ll_register_callbacks cb 0x7f5300004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:37.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f5315c52640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f5315c52640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59682/0 (socket says 172.21.15.83:59682) 2024-09-21T03:21:37.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f5315c52640 1 -- 172.21.15.83:0/2966677400 learned_addr learned my addr 172.21.15.83:0/2966677400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:37.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f5316453640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c134b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.016+0000 7f5316c54640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c1347310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 -- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 msgr2=0x5594c1347310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:37.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c1347310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 -- 172.21.15.83:0/2966677400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 msgr2=0x5594c134b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c134b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 -- 172.21.15.83:0/2966677400 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c13a2f30 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5316c54640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c1347310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f5315c52640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5300005ab0 tx=0x7f53000095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e99bdcd0493327c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f52f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300050040 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53080ad650 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.017+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f530003d860 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.018+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5300046070 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.018+0000 7f531ec2b740 10 client.25789 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.018+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5594c13a3fd0 con 0x5594c0f57460 2024-09-21T03:21:37.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.018+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53080ad650 con 0x5594c0f57460 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f53000314a0 con 0x5594c0f57460 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 10 client.25789 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 20 client.25789 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0ef6da0 0x5594c13a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 4 client.25789 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594c127e850 con 0x5594c0ef6da0 2024-09-21T03:21:37.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.019+0000 7f5316453640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0ef6da0 0x5594c13a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:37.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f52f9ffb640 10 client.25789 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:37.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f5316453640 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0ef6da0 0x5594c13a6740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d765e9b26a0f5288 server_cookie=ebeecdfa66727e0e in_seq=0 out_seq=0 2024-09-21T03:21:37.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f52f9ffb640 1 -- 172.21.15.83:0/2966677400 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x5594c127e850 con 0x5594c0ef6da0 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f52f9ffb640 10 client.25789 handle_command_reply: tid=0 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 client.25789 shutdown 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 msgr2=0x5594c1346c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5300005ab0 tx=0x7f53000095a0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 shutdown_connections 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5594c0ef6da0 0x5594c13a6740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594c0f63070 0x5594c1347310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f52f9ffb640 0 client.25789 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c0f57460 0x5594c1346c00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 --2- 172.21.15.83:0/2966677400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594c0f3a1b0 0x5594c134b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 >> 172.21.15.83:0/2966677400 conn(0x5594c13186a0 msgr2=0x5594c130d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.020+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 shutdown_connections 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f531ec2b740 1 -- 172.21.15.83:0/2966677400 wait complete. 2024-09-21T03:21:37.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f531ec2b740 20 client.25789 trim_cache size 0 max 16384 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.027 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:21:37.028 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:37.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 msgr2=0x7f52fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:37.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 0x7f52fc0b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f530002aa00 tx=0x7f5300041040 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 msgr2=0x7f531019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f530800eb10 tx=0x7f530800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 shutdown_connections 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f52fc0b6680 0x7f52fc0b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531010ddf0 0x7f531019fd10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5310105270 0x7f531019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 --2- 172.21.15.83:0/1351895882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310104840 0x7f531019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 >> 172.21.15.83:0/1351895882 conn(0x7f5310079730 msgr2=0x7f5310107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 shutdown_connections 2024-09-21T03:21:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:37.021+0000 7f53186b7640 1 -- 172.21.15.83:0/1351895882 wait complete. 2024-09-21T03:21:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:37 smithi142 ceph-mon[34512]: pgmap v474: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 688 op/s 2024-09-21T03:21:37.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:37 smithi193 ceph-mon[35224]: pgmap v474: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 688 op/s 2024-09-21T03:21:37.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:37 smithi083 ceph-mon[31219]: pgmap v474: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 688 op/s 2024-09-21T03:21:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:39 smithi142 ceph-mon[34512]: pgmap v475: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-09-21T03:21:39.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:39 smithi083 ceph-mon[31219]: pgmap v475: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-09-21T03:21:39.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:21:39.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:21:39.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:39 smithi193 ceph-mon[35224]: pgmap v475: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-09-21T03:21:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1909296810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4108cf0 msgr2=0x7fc1e410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1909296810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4108cf0 0x7fc1e410f520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009be0 tx=0x7fc1dc02a6f0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1909296810 shutdown_connections 2024-09-21T03:21:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1909296810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4108cf0 0x7fc1e410f520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1909296810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 0x7fc1e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1909296810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4102a60 0x7fc1e4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1909296810 >> 172.21.15.83:0/1909296810 conn(0x7fc1e4079730 msgr2=0x7fc1e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1909296810 shutdown_connections 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.778+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1909296810 wait complete. 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.779+0000 7fc1e8c4d640 1 Processor -- start 2024-09-21T03:21:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.779+0000 7fc1e8c4d640 1 -- start start 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 0x7fc1e41050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46918/0 (socket says 172.21.15.83:46918) 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 0x7fc1e4105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e259c640 1 -- 172.21.15.83:0/1432294133 learned_addr learned my addr 172.21.15.83:0/1432294133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e41129c0 con 0x7fc1e4102a60 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112840 con 0x7fc1e41083a0 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.780+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112b40 con 0x7fc1e4108cf0 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e2d9d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 0x7fc1e4105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e1d9b640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 0x7fc1e41050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 -- 172.21.15.83:0/1432294133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 msgr2=0x7fc1e4105620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 0x7fc1e4105620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 -- 172.21.15.83:0/1432294133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 msgr2=0x7fc1e41050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 0x7fc1e41050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e4105e20 con 0x7fc1e4102a60 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e2d9d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 0x7fc1e4105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e259c640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d950 tx=0x7fc1d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4017070 con 0x7fc1e4102a60 2024-09-21T03:21:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41ace80 con 0x7fc1e4102a60 2024-09-21T03:21:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.781+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d4009c30 con 0x7fc1e4102a60 2024-09-21T03:21:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.782+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400f620 con 0x7fc1e4102a60 2024-09-21T03:21:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.782+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41ad400 con 0x7fc1e4102a60 2024-09-21T03:21:40.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.783+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4102e70 con 0x7fc1e4102a60 2024-09-21T03:21:40.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.786+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc1d4010460 con 0x7fc1e4102a60 2024-09-21T03:21:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.787+0000 7fc1d37fe640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 0x7fc1cc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.787+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc1d40e7660 con 0x7fc1e4102a60 2024-09-21T03:21:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.787+0000 7fc1e1d9b640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 0x7fc1cc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.788+0000 7fc1e1d9b640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 0x7fc1cc0b8bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8006fd0 tx=0x7fc1d8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:40.794+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1d40adbd0 con 0x7fc1e4102a60 2024-09-21T03:21:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 260.0 s, 3140.4 tps, lat 31.778 ms stddev 33.169 2024-09-21T03:21:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.140+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e402e640 con 0x7fc1e4102a60 2024-09-21T03:21:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.141+0000 7fc1d37fe640 1 -- 172.21.15.83:0/1432294133 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11990 (secure 0 0 0) 0x7fc1d4009de0 con 0x7fc1e4102a60 2024-09-21T03:21:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:21:41.143 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":40,"btime":"2024-09-21T03:21:09:378043+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:21:09.043180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:21:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 40 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 msgr2=0x7fc1cc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 0x7fc1cc0b8bc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8006fd0 tx=0x7fc1d8008040 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 msgr2=0x7fc1e4106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d950 tx=0x7fc1d400de10 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 shutdown_connections 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1cc0b66b0 0x7fc1cc0b8bc0 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8006fd0 tx=0x7fc1d8008040 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e4108cf0 0x7fc1e4105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e41083a0 0x7fc1e41050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 --2- 172.21.15.83:0/1432294133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1e4102a60 0x7fc1e4106af0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.143+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 >> 172.21.15.83:0/1432294133 conn(0x7fc1e4079730 msgr2=0x7fc1e406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.144+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 shutdown_connections 2024-09-21T03:21:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:41.144+0000 7fc1e8c4d640 1 -- 172.21.15.83:0/1432294133 wait complete. 2024-09-21T03:21:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1432294133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:21:41.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1432294133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:21:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1432294133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:21:41.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:21:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:42.462 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:42 smithi083 ceph-mon[31219]: pgmap v476: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-21T03:21:42.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:42 smithi193 ceph-mon[35224]: pgmap v476: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-21T03:21:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:42 smithi142 ceph-mon[34512]: pgmap v476: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 -- 172.21.15.83:0/444860613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c075dd0 msgr2=0x7f0d0c0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/444860613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c0761e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009900 tx=0x7f0cfc02a410 comp rx=0 tx=0).stop 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 -- 172.21.15.83:0/444860613 shutdown_connections 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/444860613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/444860613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c0761e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/444860613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c077380 0x7f0d0c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.881+0000 7f0d10a6d640 1 -- 172.21.15.83:0/444860613 >> 172.21.15.83:0/444860613 conn(0x7f0d0c0ff5e0 msgr2=0x7f0d0c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.882+0000 7f0d10a6d640 1 -- 172.21.15.83:0/444860613 shutdown_connections 2024-09-21T03:21:42.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.883+0000 7f0d10a6d640 1 -- 172.21.15.83:0/444860613 wait complete. 2024-09-21T03:21:42.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 Processor -- start 2024-09-21T03:21:42.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 -- start start 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c11a070 con 0x7f0d0c077380 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c119ef0 con 0x7f0d0c075dd0 2024-09-21T03:21:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d10a6d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c11a1f0 con 0x7f0d0c114440 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50906/0 (socket says 172.21.15.83:50906) 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d0a59c640 1 -- 172.21.15.83:0/561978787 learned_addr learned my addr 172.21.15.83:0/561978787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46932/0 (socket says 172.21.15.83:46932) 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.884+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 -- 172.21.15.83:0/561978787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 msgr2=0x7f0d0c1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 -- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 msgr2=0x7f0d0c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 -- 172.21.15.83:0/561978787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c1a6a50 con 0x7f0d0c075dd0 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:42.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:42.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000e9f0 tx=0x7f0d0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0cfb7fe640 1 -- 172.21.15.83:0/561978787 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000cd80 con 0x7f0d0c075dd0 2024-09-21T03:21:42.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1a6d40 con 0x7f0d0c075dd0 2024-09-21T03:21:42.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1ad4a0 con 0x7f0d0c075dd0 2024-09-21T03:21:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0cfb7fe640 1 -- 172.21.15.83:0/561978787 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d000108e0 con 0x7f0d0c075dd0 2024-09-21T03:21:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.885+0000 7f0cfb7fe640 1 -- 172.21.15.83:0/561978787 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d000188e0 con 0x7f0d0c075dd0 2024-09-21T03:21:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.888+0000 7f0cfb7fe640 1 -- 172.21.15.83:0/561978787 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0d00018a80 con 0x7f0d0c075dd0 2024-09-21T03:21:42.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.888+0000 7f0d16fe1740 1 Processor -- start 2024-09-21T03:21:42.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 -- start start 2024-09-21T03:21:42.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca8212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca821800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca821d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641ca797ea0 con 0x5641ca47a070 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5641ca7872d0 con 0x5641ca4511b0 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641ca78b0e0 con 0x5641ca459390 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d0ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca8212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d0ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca8212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46946/0 (socket says 172.21.15.83:46946) 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d0ad9d640 1 -- 172.21.15.83:0/1816958353 learned_addr learned my addr 172.21.15.83:0/1816958353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca821800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca821d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.889+0000 7f0cfb7fe640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 0x7f0cf40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 0x7f0cf40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 -- 172.21.15.83:0/1816958353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 msgr2=0x5641ca821800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca821800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 -- 172.21.15.83:0/1816958353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 msgr2=0x5641ca8212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca8212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 -- 172.21.15.83:0/1816958353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ca4e5650 con 0x5641ca459390 2024-09-21T03:21:42.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0cfb7fe640 1 -- 172.21.15.83:0/561978787 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0d000e7a30 con 0x7f0d0c075dd0 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca821d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006010 tx=0x7f0cfc005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e695028de76525b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0cf8ff9640 1 -- 172.21.15.83:0/1816958353 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc040070 con 0x5641ca459390 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0cf8ff9640 1 -- 172.21.15.83:0/1816958353 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc031720 con 0x5641ca459390 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0cf8ff9640 1 -- 172.21.15.83:0/1816958353 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc035040 con 0x5641ca459390 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.890+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 0x7f0cf40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005c40 tx=0x7f0d04004ab0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1816958353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 msgr2=0x5641ca821d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca821d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006010 tx=0x7f0cfc005e20 comp rx=0 tx=0).stop 2024-09-21T03:21:42.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1816958353 shutdown_connections 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca821d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca821800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1816958353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca8212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1816958353 >> 172.21.15.83:0/1816958353 conn(0x5641ca46e460 msgr2=0x5641ca81f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1816958353 shutdown_connections 2024-09-21T03:21:42.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.892+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1816958353 wait complete. 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 -- start start 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 Processor -- start 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca82c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca82d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641ca5e8700 con 0x5641ca459390 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5641ca722160 con 0x5641ca47a070 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641ca82fa40 con 0x5641ca4511b0 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d16fe1740 10 client.0 _ll_register_callbacks cb 0x5641ca73b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca82c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d0ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca82d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46954/0 (socket says 172.21.15.83:46954) 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.893+0000 7f0d09d9b640 1 -- 172.21.15.83:0/3149727059 learned_addr learned my addr 172.21.15.83:0/3149727059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 -- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 msgr2=0x5641ca82c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca82c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 -- 172.21.15.83:0/3149727059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 msgr2=0x5641ca82d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca82d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 -- 172.21.15.83:0/3149727059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ca74fc90 con 0x5641ca459390 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0319b0 tx=0x7f0cfc0319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8d8fe5518eb9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0cf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.894+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc04b020 con 0x5641ca459390 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.895+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc031720 con 0x5641ca459390 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.895+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0cfc004250 con 0x5641ca459390 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.895+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc056050 con 0x5641ca459390 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.895+0000 7f0d16fe1740 10 client.16284 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.895+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5641ca83b650 con 0x5641ca459390 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc004450 con 0x5641ca459390 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0cfc03c760 con 0x5641ca459390 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 10 client.16284 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 10 client.16284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 20 client.16284 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca4389f0 0x5641ca83ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 4 client.16284 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.896+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0cfc031720 con 0x5641ca4389f0 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.897+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca4389f0 0x5641ca83ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.898+0000 7f0cf1ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.898+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca4389f0 0x5641ca83ed40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cee46c4ab175137 server_cookie=5fccaf4a6edf33e6 in_seq=0 out_seq=0 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/3149727059 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0cfc031720 con 0x5641ca4389f0 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0cf1ffb640 10 client.16284 handle_command_reply: tid=0 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 client.16284 shutdown 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 msgr2=0x5641ca82ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0319b0 tx=0x7f0cfc0319e0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 shutdown_connections 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca4389f0 0x5641ca83ed40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca82d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca82ce20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/3149727059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca82c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 >> 172.21.15.83:0/3149727059 conn(0x5641ca46e460 msgr2=0x5641ca8244a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0cf1ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:21:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.899+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 shutdown_connections 2024-09-21T03:21:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.900+0000 7f0d16fe1740 1 -- 172.21.15.83:0/3149727059 wait complete. 2024-09-21T03:21:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.900+0000 7f0d16fe1740 20 client.16284 trim_cache size 0 max 16384 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 Processor -- start 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 -- start start 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca822cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b5bf0 con 0x5641ca47a070 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b7820 con 0x5641ca4511b0 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d16fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b62c0 con 0x5641ca459390 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d0ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60158/0 (socket says 172.21.15.83:60158) 2024-09-21T03:21:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca822cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d0ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46956/0 (socket says 172.21.15.83:46956) 2024-09-21T03:21:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.916+0000 7f0d09d9b640 1 -- 172.21.15.83:0/1791598310 learned_addr learned my addr 172.21.15.83:0/1791598310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 -- 172.21.15.83:0/1791598310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 msgr2=0x5641ca824ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 -- 172.21.15.83:0/1791598310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 msgr2=0x5641ca822cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca822cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 -- 172.21.15.83:0/1791598310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ca82dba0 con 0x5641ca47a070 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca822cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009fd0 tx=0x7f0d0400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5495d91721a4e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0cf3fff640 1 -- 172.21.15.83:0/1791598310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04028070 con 0x5641ca47a070 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0cf3fff640 1 -- 172.21.15.83:0/1791598310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d04011df0 con 0x5641ca47a070 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.917+0000 7f0cf3fff640 1 -- 172.21.15.83:0/1791598310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d040233d0 con 0x5641ca47a070 2024-09-21T03:21:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1791598310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 msgr2=0x5641ca829ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009fd0 tx=0x7f0d0400f3c0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1791598310 shutdown_connections 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca47a070 0x5641ca829ad0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca459390 0x5641ca824ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/1791598310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca4511b0 0x5641ca822cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.918+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1791598310 >> 172.21.15.83:0/1791598310 conn(0x5641ca46e460 msgr2=0x5641ca8243c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.919+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1791598310 shutdown_connections 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.919+0000 7f0d16fe1740 1 -- 172.21.15.83:0/1791598310 wait complete. 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.919+0000 7f0d16fe1740 1 -- start start 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.919+0000 7f0d16fe1740 1 Processor -- start 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca8328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60172/0 (socket says 172.21.15.83:60172) 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d0a59c640 1 -- 172.21.15.83:0/4160850915 learned_addr learned my addr 172.21.15.83:0/4160850915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca832df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b7150 con 0x5641ca459390 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b69c0 con 0x5641ca47a070 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca8328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b3bd0 con 0x5641ca4511b0 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d0ad9d640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca832df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.920+0000 7f0d16fe1740 10 client.0 _ll_register_callbacks cb 0x7f0d04009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 -- 172.21.15.83:0/4160850915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 msgr2=0x5641ca832df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca832df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 -- 172.21.15.83:0/4160850915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 msgr2=0x5641ca8328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca8328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ca8334e0 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d04011df0 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d0a59c640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d000aabc0 tx=0x7f0d00009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76627c6a9e2437f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0cf1ffb640 10 client.25813 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/4160850915 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d000b2170 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/4160850915 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f0d000e4d60 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/4160850915 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d000ad460 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d16fe1740 10 client.25813 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.921+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5641ca8a94b0 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/4160850915 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0d000d4020 con 0x5641ca4511b0 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 10 client.25813 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 10 client.25813 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:21:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 20 client.25813 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca40dda0 0x5641ca8abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 4 client.25813 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:21:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641ca799df0 con 0x5641ca40dda0 2024-09-21T03:21:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.922+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca40dda0 0x5641ca8abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0cf1ffb640 10 client.25813 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:21:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d09d9b640 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca40dda0 0x5641ca8abc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffdb896ecccd5fab server_cookie=c25a794bfa3c4bd1 in_seq=0 out_seq=0 2024-09-21T03:21:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0cf1ffb640 1 -- 172.21.15.83:0/4160850915 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641ca799df0 con 0x5641ca40dda0 2024-09-21T03:21:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0cf1ffb640 10 client.25813 handle_command_reply: tid=0 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 client.25813 shutdown 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 msgr2=0x5641ca8353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d000aabc0 tx=0x7f0d00009700 comp rx=0 tx=0).stop 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 shutdown_connections 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5641ca40dda0 0x5641ca8abc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641ca47a070 0x5641ca832df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ca459390 0x5641ca8328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 --2- 172.21.15.83:0/4160850915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641ca4511b0 0x5641ca8353e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 >> 172.21.15.83:0/4160850915 conn(0x5641ca46e460 msgr2=0x5641ca82ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0cf1ffb640 0 client.25813 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 shutdown_connections 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.923+0000 7f0d16fe1740 1 -- 172.21.15.83:0/4160850915 wait complete. 2024-09-21T03:21:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.924+0000 7f0d16fe1740 20 client.25813 trim_cache size 0 max 16384 2024-09-21T03:21:42.930 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:21:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.925+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 msgr2=0x7f0cf40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.925+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 0x7f0cf40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005c40 tx=0x7f0d04004ab0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.925+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 msgr2=0x7f0d0c1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.925+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000e9f0 tx=0x7f0d0000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 shutdown_connections 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0cf40b6680 0x7f0cf40b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d0c114440 0x7f0d0c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d0c077380 0x7f0d0c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 --2- 172.21.15.83:0/561978787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d0c075dd0 0x7f0d0c1a0ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 >> 172.21.15.83:0/561978787 conn(0x7f0d0c0ff5e0 msgr2=0x7f0d0c110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 shutdown_connections 2024-09-21T03:21:42.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:42.926+0000 7f0d10a6d640 1 -- 172.21.15.83:0/561978787 wait complete. 2024-09-21T03:21:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:43 smithi142 ceph-mon[34512]: pgmap v477: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 798 op/s 2024-09-21T03:21:43.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:43 smithi083 ceph-mon[31219]: pgmap v477: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 798 op/s 2024-09-21T03:21:43.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:21:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:43 smithi193 ceph-mon[35224]: pgmap v477: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 798 op/s 2024-09-21T03:21:44.134 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.900+0000 7f9582d77640 1 -- 172.21.15.83:0/1620985475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c102aa0 msgr2=0x7f957c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.900+0000 7f9582d77640 1 --2- 172.21.15.83:0/1620985475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c102aa0 0x7f957c102eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9570009900 tx=0x7f957002a400 comp rx=0 tx=0).stop 2024-09-21T03:21:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 -- 172.21.15.83:0/1620985475 shutdown_connections 2024-09-21T03:21:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 --2- 172.21.15.83:0/1620985475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c108d30 0x7f957c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 --2- 172.21.15.83:0/1620985475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c1083e0 0x7f957c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 --2- 172.21.15.83:0/1620985475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c102aa0 0x7f957c102eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 -- 172.21.15.83:0/1620985475 >> 172.21.15.83:0/1620985475 conn(0x7f957c0797e0 msgr2=0x7f957c079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 -- 172.21.15.83:0/1620985475 shutdown_connections 2024-09-21T03:21:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.901+0000 7f9582d77640 1 -- 172.21.15.83:0/1620985475 wait complete. 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 Processor -- start 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 -- start start 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c102aa0 0x7f957c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 0x7f957c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1118a0 con 0x7f957c102aa0 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f957c111720 con 0x7f957c1083e0 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.902+0000 7f9582d77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f957c111a20 con 0x7f957c108d30 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f9581314640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 0x7f957c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50954/0 (socket says 172.21.15.83:50954) 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 -- 172.21.15.83:0/2814054154 learned_addr learned my addr 172.21.15.83:0/2814054154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f9580b13640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c102aa0 0x7f957c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 -- 172.21.15.83:0/2814054154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 msgr2=0x7f957c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 0x7f957c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 -- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c102aa0 msgr2=0x7f957c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c102aa0 0x7f957c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 -- 172.21.15.83:0/2814054154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c19e200 con 0x7f957c1083e0 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f9581314640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 0x7f957c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.903+0000 7f957bfff640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f956c012f90 tx=0x7f956c010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.904+0000 7f9579ffb640 1 -- 172.21.15.83:0/2814054154 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c00de50 con 0x7f957c1083e0 2024-09-21T03:21:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.904+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c19e490 con 0x7f957c1083e0 2024-09-21T03:21:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.904+0000 7f9579ffb640 1 -- 172.21.15.83:0/2814054154 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956c00de50 con 0x7f957c1083e0 2024-09-21T03:21:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.904+0000 7f9579ffb640 1 -- 172.21.15.83:0/2814054154 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0055f0 con 0x7f957c1083e0 2024-09-21T03:21:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.905+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a4c50 con 0x7f957c1083e0 2024-09-21T03:21:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.908+0000 7f9579ffb640 1 -- 172.21.15.83:0/2814054154 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f956c01f020 con 0x7f957c1083e0 2024-09-21T03:21:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.908+0000 7f95892eb740 1 Processor -- start 2024-09-21T03:21:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.908+0000 7f95892eb740 1 -- start start 2024-09-21T03:21:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.908+0000 7f95892eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be329b6070 0x55be32d5d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.908+0000 7f95892eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 0x55be32d5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f95892eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f95892eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be32cd3ea0 con 0x55be3298d1b0 2024-09-21T03:21:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f95892eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be32cc32d0 con 0x55be32995390 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f95892eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be32cc70e0 con 0x55be329b6070 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9579ffb640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 0x7f95640b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9580b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 0x55be32d5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9580b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 0x55be32d5dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46990/0 (socket says 172.21.15.83:46990) 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9580b13640 1 -- 172.21.15.83:0/1637402129 learned_addr learned my addr 172.21.15.83:0/1637402129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9581314640 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be329b6070 0x55be32d5d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9579ffb640 1 -- 172.21.15.83:0/2814054154 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f956c01d020 con 0x7f957c1083e0 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 -- 172.21.15.83:0/1637402129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be329b6070 msgr2=0x55be32d5d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be329b6070 0x55be32d5d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 -- 172.21.15.83:0/1637402129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 msgr2=0x55be32d5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 0x55be32d5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 -- 172.21.15.83:0/1637402129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be32a21650 con 0x55be32995390 2024-09-21T03:21:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f9581314640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 0x7f95640b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.909+0000 7f957bfff640 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5e120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f956c0e8210 tx=0x7f956c00d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda8229abc52d612 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.910+0000 7f9562ffd640 1 -- 172.21.15.83:0/1637402129 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b0420 con 0x55be32995390 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.910+0000 7f9562ffd640 1 -- 172.21.15.83:0/1637402129 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956c0b1010 con 0x55be32995390 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.910+0000 7f9562ffd640 1 -- 172.21.15.83:0/1637402129 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b0b70 con 0x55be32995390 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 -- 172.21.15.83:0/1637402129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 msgr2=0x55be32d5e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5e120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f956c0e8210 tx=0x7f956c00d000 comp rx=0 tx=0).stop 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f9581314640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 0x7f95640b8f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95740049d0 tx=0x7f9574004a40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 -- 172.21.15.83:0/1637402129 shutdown_connections 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5e120 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be3298d1b0 0x55be32d5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 --2- 172.21.15.83:0/1637402129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be329b6070 0x55be32d5d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 -- 172.21.15.83:0/1637402129 >> 172.21.15.83:0/1637402129 conn(0x55be329aa460 msgr2=0x55be32d5b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 -- 172.21.15.83:0/1637402129 shutdown_connections 2024-09-21T03:21:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.911+0000 7f95892eb740 1 -- 172.21.15.83:0/1637402129 wait complete. 2024-09-21T03:21:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.912+0000 7f95892eb740 1 -- start start 2024-09-21T03:21:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.912+0000 7f95892eb740 1 Processor -- start 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be32b24700 con 0x55be329b6070 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be32c5e160 con 0x55be3298d1b0 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be32d6be20 con 0x55be32995390 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f9580b13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f957bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f9580b13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50966/0 (socket says 172.21.15.83:50966) 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f957bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d69300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60206/0 (socket says 172.21.15.83:60206) 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f9580b13640 1 -- 172.21.15.83:0/2006221787 learned_addr learned my addr 172.21.15.83:0/2006221787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.913+0000 7f95892eb740 10 client.0 _ll_register_callbacks cb 0x55be32c779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 -- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 msgr2=0x55be32d69300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d69300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 -- 172.21.15.83:0/2006221787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 msgr2=0x55be32d75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 -- 172.21.15.83:0/2006221787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be32c8bc90 con 0x55be3298d1b0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f9580b13640 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9570000c00 tx=0x7f9570005f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f76c27d7de7324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f955bfff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570035040 con 0x55be3298d1b0 2024-09-21T03:21:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f9570031720 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.914+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956c0b1010 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9570031920 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 10 client.25835 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55be32d77b90 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956c0b1010 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f95700047e0 con 0x55be3298d1b0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 10 client.25835 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 10 client.25835 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 20 client.25835 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be329749f0 0x55be32d7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 4 client.25835 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.915+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be32cd5df0 con 0x55be329749f0 2024-09-21T03:21:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.916+0000 7f957bfff640 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be329749f0 0x55be32d7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.916+0000 7f955bfff640 10 client.25835 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:21:44.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.916+0000 7f957bfff640 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be329749f0 0x55be32d7b080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1080335f5f590aaf server_cookie=d2916a71c784836f in_seq=0 out_seq=0 2024-09-21T03:21:44.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f955bfff640 1 -- 172.21.15.83:0/2006221787 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be32cd5df0 con 0x55be329749f0 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f955bfff640 10 client.25835 handle_command_reply: tid=0 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 client.25835 shutdown 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 msgr2=0x55be32d68dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9570000c00 tx=0x7f9570005f20 comp rx=0 tx=0).stop 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 shutdown_connections 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be329749f0 0x55be32d7b080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d69300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 --2- 172.21.15.83:0/2006221787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d68dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 >> 172.21.15.83:0/2006221787 conn(0x55be329aa460 msgr2=0x55be32d60830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f955bfff640 0 client.25835 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.917+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 shutdown_connections 2024-09-21T03:21:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.918+0000 7f95892eb740 1 -- 172.21.15.83:0/2006221787 wait complete. 2024-09-21T03:21:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.918+0000 7f95892eb740 20 client.25835 trim_cache size 0 max 16384 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 Processor -- start 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 -- start start 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be3298d1b0 0x55be32d6aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f956800e620 con 0x55be329b6070 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9568010250 con 0x55be32995390 2024-09-21T03:21:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f95892eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f956800ecf0 con 0x55be3298d1b0 2024-09-21T03:21:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f9581314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f9581314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47010/0 (socket says 172.21.15.83:47010) 2024-09-21T03:21:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.937+0000 7f9581314640 1 -- 172.21.15.83:0/964589052 learned_addr learned my addr 172.21.15.83:0/964589052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9580b13640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be3298d1b0 0x55be32d6aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9581314640 1 -- 172.21.15.83:0/964589052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be3298d1b0 msgr2=0x55be32d6aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9581314640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be3298d1b0 0x55be32d6aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f957bfff640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9581314640 1 -- 172.21.15.83:0/964589052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 msgr2=0x55be32d5ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9581314640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.938+0000 7f9581314640 1 -- 172.21.15.83:0/964589052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be32d69e80 con 0x55be329b6070 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.939+0000 7f957bfff640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.939+0000 7f9581314640 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95740148f0 tx=0x7f9574004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7e955e4109ac7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.939+0000 7f95627fc640 1 -- 172.21.15.83:0/964589052 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574017d30 con 0x55be329b6070 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.940+0000 7f95627fc640 1 -- 172.21.15.83:0/964589052 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9574029070 con 0x55be329b6070 2024-09-21T03:21:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.940+0000 7f95627fc640 1 -- 172.21.15.83:0/964589052 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95740165e0 con 0x55be329b6070 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 -- 172.21.15.83:0/964589052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 msgr2=0x55be32d5abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95740148f0 tx=0x7f9574004ab0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 -- 172.21.15.83:0/964589052 shutdown_connections 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d5abc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be32995390 0x55be32d5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 --2- 172.21.15.83:0/964589052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be3298d1b0 0x55be32d6aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 -- 172.21.15.83:0/964589052 >> 172.21.15.83:0/964589052 conn(0x55be329aa460 msgr2=0x55be32d5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 -- 172.21.15.83:0/964589052 shutdown_connections 2024-09-21T03:21:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.941+0000 7f95892eb740 1 -- 172.21.15.83:0/964589052 wait complete. 2024-09-21T03:21:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.942+0000 7f95892eb740 1 -- start start 2024-09-21T03:21:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.942+0000 7f95892eb740 1 Processor -- start 2024-09-21T03:21:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d71530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f9580b13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d71530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f9580b13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d71530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50988/0 (socket says 172.21.15.83:50988) 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f9580b13640 1 -- 172.21.15.83:0/3579117233 learned_addr learned my addr 172.21.15.83:0/3579117233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f956800fb80 con 0x55be329b6070 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f956800f3f0 con 0x55be3298d1b0 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f956800c600 con 0x55be32995390 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f9581314640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.943+0000 7f957bfff640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f95892eb740 10 client.0 _ll_register_callbacks cb 0x7f9570009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 -- 172.21.15.83:0/3579117233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 msgr2=0x55be32d71530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d71530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 -- 172.21.15.83:0/3579117233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 msgr2=0x55be32d6cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d6cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be32d6d460 con 0x55be32995390 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f957bfff640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d6c9e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f956c0b0ad0 tx=0x7f956c0b0080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c4d06ef58f63e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f955bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b1010 con 0x55be32995390 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f956c0055d0 con 0x55be32995390 2024-09-21T03:21:44.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.944+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9574029070 con 0x55be32995390 2024-09-21T03:21:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f956c0b53f0 con 0x55be32995390 2024-09-21T03:21:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 10 client.25831 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55be32de5640 con 0x55be32995390 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9574029070 con 0x55be32995390 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f956c0b0b00 con 0x55be32995390 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 10 client.25831 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 10 client.25831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 20 client.25831 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be32949da0 0x55be32de78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 4 client.25831 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be32cd18a0 con 0x55be32949da0 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.945+0000 7f9580b13640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be32949da0 0x55be32de78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.946+0000 7f955bfff640 10 client.25831 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:21:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.947+0000 7f9580b13640 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be32949da0 0x55be32de78e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d386446059d08edb server_cookie=3bfacc0eaceb6a29 in_seq=0 out_seq=0 2024-09-21T03:21:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f955bfff640 1 -- 172.21.15.83:0/3579117233 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be32cd18a0 con 0x55be32949da0 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f955bfff640 10 client.25831 handle_command_reply: tid=0 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 client.25831 shutdown 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 msgr2=0x55be32d6c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d6c9e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f956c0b0ad0 tx=0x7f956c0b0080 comp rx=0 tx=0).stop 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 shutdown_connections 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be329b6070 0x55be32d6cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55be32949da0 0x55be32de78e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be32995390 0x55be32d6c9e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 --2- 172.21.15.83:0/3579117233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be3298d1b0 0x55be32d71530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 >> 172.21.15.83:0/3579117233 conn(0x55be329aa460 msgr2=0x55be32d6aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f955bfff640 0 client.25831 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.948+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 shutdown_connections 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f95892eb740 1 -- 172.21.15.83:0/3579117233 wait complete. 2024-09-21T03:21:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f95892eb740 20 client.25831 trim_cache size 0 max 16384 2024-09-21T03:21:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:21:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 msgr2=0x7f95640b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 0x7f95640b8f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95740049d0 tx=0x7f9574004a40 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 msgr2=0x7f957c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.949+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f956c012f90 tx=0x7f956c010490 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 shutdown_connections 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95640b6a80 0x7f95640b8f90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f957c108d30 0x7f957c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c1083e0 0x7f957c198830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 --2- 172.21.15.83:0/2814054154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957c102aa0 0x7f957c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 >> 172.21.15.83:0/2814054154 conn(0x7f957c0797e0 msgr2=0x7f957c101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 shutdown_connections 2024-09-21T03:21:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:44.950+0000 7f9582d77640 1 -- 172.21.15.83:0/2814054154 wait complete. 2024-09-21T03:21:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:45 smithi083 ceph-mon[31219]: pgmap v478: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 674 op/s 2024-09-21T03:21:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:45 smithi142 ceph-mon[34512]: pgmap v478: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 674 op/s 2024-09-21T03:21:45.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:21:45.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:45 smithi193 ceph-mon[35224]: pgmap v478: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 674 op/s 2024-09-21T03:21:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 265.0 s, 3165.8 tps, lat 31.357 ms stddev 33.348 2024-09-21T03:21:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 -- 172.21.15.83:0/716902266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c105af0 msgr2=0x7f203c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 --2- 172.21.15.83:0/716902266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c105af0 0x7f203c108000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2034009ca0 tx=0x7f203402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 -- 172.21.15.83:0/716902266 shutdown_connections 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 --2- 172.21.15.83:0/716902266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c105af0 0x7f203c108000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 --2- 172.21.15.83:0/716902266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c07cf30 0x7f203c07d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 --2- 172.21.15.83:0/716902266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0808b0 0x7f203c07c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.669+0000 7f2040d09640 1 -- 172.21.15.83:0/716902266 >> 172.21.15.83:0/716902266 conn(0x7f203c0795e0 msgr2=0x7f203c07ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.670+0000 7f2040d09640 1 -- 172.21.15.83:0/716902266 shutdown_connections 2024-09-21T03:21:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.670+0000 7f2040d09640 1 -- 172.21.15.83:0/716902266 wait complete. 2024-09-21T03:21:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 Processor -- start 2024-09-21T03:21:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 -- start start 2024-09-21T03:21:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 0x7f203c113060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c105af0 0x7f203c113600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10b770 con 0x7f203c1ae650 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10b5f0 con 0x7f203c07cf30 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2040d09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f203c10b8f0 con 0x7f203c105af0 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47030/0 (socket says 172.21.15.83:47030) 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f203ad9d640 1 -- 172.21.15.83:0/1944015606 learned_addr learned my addr 172.21.15.83:0/1944015606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f2039d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c105af0 0x7f203c113600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.671+0000 7f203a59c640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 0x7f203c113060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 -- 172.21.15.83:0/1944015606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c105af0 msgr2=0x7f203c113600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c105af0 0x7f203c113600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 -- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 msgr2=0x7f203c1aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 -- 172.21.15.83:0/1944015606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c1af0d0 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203ad9d640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f203a59c640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 0x7f203c113060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f203000c970 tx=0x7f203000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f202b7fe640 1 -- 172.21.15.83:0/1944015606 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030007bb0 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c1af3c0 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f202b7fe640 1 -- 172.21.15.83:0/1944015606 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2030007d50 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f202b7fe640 1 -- 172.21.15.83:0/1944015606 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030018780 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.672+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c1b5ae0 con 0x7f203c07cf30 2024-09-21T03:21:46.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.675+0000 7f202b7fe640 1 -- 172.21.15.83:0/1944015606 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f203000f450 con 0x7f203c07cf30 2024-09-21T03:21:46.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.675+0000 7f204727d740 1 Processor -- start 2024-09-21T03:21:46.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.677+0000 7f202b7fe640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 0x7f20240b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.677+0000 7f202b7fe640 1 -- 172.21.15.83:0/1944015606 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f20300e76a0 con 0x7f203c07cf30 2024-09-21T03:21:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.678+0000 7f204727d740 1 -- start start 2024-09-21T03:21:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.678+0000 7f203ad9d640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 0x7f20240b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.678+0000 7f204727d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 0x5588adde06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f204727d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f204727d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada18390 0x5588adde1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f204727d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588add56ea0 con 0x5588ada39070 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f204727d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5588add462d0 con 0x5588ada18390 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f204727d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5588add4a0e0 con 0x5588ada101b0 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60242/0 (socket says 172.21.15.83:60242) 2024-09-21T03:21:46.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 -- 172.21.15.83:0/1733834407 learned_addr learned my addr 172.21.15.83:0/1733834407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f203a59c640 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 0x5588adde06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 -- 172.21.15.83:0/1733834407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada18390 msgr2=0x5588adde1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada18390 0x5588adde1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 -- 172.21.15.83:0/1733834407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 msgr2=0x5588adde06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 0x5588adde06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f2039d9b640 1 -- 172.21.15.83:0/1733834407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588adaa4650 con 0x5588ada101b0 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.679+0000 7f203a59c640 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 0x5588adde06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.680+0000 7f2039d9b640 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f202c009c20 tx=0x7f202c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61abbedc1e516c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.680+0000 7f203ad9d640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 0x7f20240b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2034005e30 tx=0x7f2034005c50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.680+0000 7f2028ff9640 1 -- 172.21.15.83:0/1733834407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c004230 con 0x5588ada101b0 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.680+0000 7f2028ff9640 1 -- 172.21.15.83:0/1733834407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c0043d0 con 0x5588ada101b0 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.680+0000 7f2028ff9640 1 -- 172.21.15.83:0/1733834407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c019750 con 0x5588ada101b0 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- 172.21.15.83:0/1733834407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 msgr2=0x5588adde0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f202c009c20 tx=0x7f202c011810 comp rx=0 tx=0).stop 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- 172.21.15.83:0/1733834407 shutdown_connections 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada18390 0x5588adde1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588adde0be0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 --2- 172.21.15.83:0/1733834407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada39070 0x5588adde06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- 172.21.15.83:0/1733834407 >> 172.21.15.83:0/1733834407 conn(0x5588ada2d460 msgr2=0x5588addde590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- 172.21.15.83:0/1733834407 shutdown_connections 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- 172.21.15.83:0/1733834407 wait complete. 2024-09-21T03:21:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.681+0000 7f204727d740 1 -- start start 2024-09-21T03:21:46.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 Processor -- start 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada39070 0x5588addf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588adba7700 con 0x5588ada18390 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5588adce1160 con 0x5588ada101b0 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5588addeee80 con 0x5588ada39070 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f204727d740 10 client.0 _ll_register_callbacks cb 0x5588adcfa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:46.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f203a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f203a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:51020/0 (socket says 172.21.15.83:51020) 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f203a59c640 1 -- 172.21.15.83:0/858971009 learned_addr learned my addr 172.21.15.83:0/858971009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f2039d9b640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.682+0000 7f203ad9d640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada39070 0x5588addf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 -- 172.21.15.83:0/858971009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada39070 msgr2=0x5588addf8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada39070 0x5588addf8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 -- 172.21.15.83:0/858971009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 msgr2=0x5588addec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addec320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 -- 172.21.15.83:0/858971009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588add0ec90 con 0x5588ada101b0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f203a59c640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20300aa7e0 tx=0x7f20300acce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f46bdc75dbcc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f2021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300ac9d0 con 0x5588ada101b0 2024-09-21T03:21:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202c0043d0 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f20300a8ec0 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f20300b2050 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f204727d740 10 client.25847 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.683+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5588addfac30 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c0043d0 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f20300b3300 con 0x5588ada101b0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 10 client.25847 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 10 client.25847 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 20 client.25847 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9f79f0 0x5588addfe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 4 client.25847 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588add58df0 con 0x5588ad9f79f0 2024-09-21T03:21:46.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.684+0000 7f2039d9b640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9f79f0 0x5588addfe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.685+0000 7f2021ffb640 10 client.25847 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:21:46.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.685+0000 7f2039d9b640 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9f79f0 0x5588addfe050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b073dc5ecb4512a6 server_cookie=b2c67ce7bf4d97a2 in_seq=0 out_seq=0 2024-09-21T03:21:46.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.686+0000 7f2021ffb640 1 -- 172.21.15.83:0/858971009 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588add58df0 con 0x5588ad9f79f0 2024-09-21T03:21:46.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.686+0000 7f2021ffb640 10 client.25847 handle_command_reply: tid=0 2024-09-21T03:21:46.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.686+0000 7f204727d740 1 client.25847 shutdown 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.686+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 msgr2=0x5588addebde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.686+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20300aa7e0 tx=0x7f20300acce0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 shutdown_connections 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9f79f0 0x5588addfe050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada39070 0x5588addf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 --2- 172.21.15.83:0/858971009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada101b0 0x5588addebde0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 >> 172.21.15.83:0/858971009 conn(0x5588ada2d460 msgr2=0x5588adde3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f2021ffb640 0 client.25847 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 shutdown_connections 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.687+0000 7f204727d740 1 -- 172.21.15.83:0/858971009 wait complete. 2024-09-21T03:21:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.688+0000 7f204727d740 20 client.25847 trim_cache size 0 max 16384 2024-09-21T03:21:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 Processor -- start 2024-09-21T03:21:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 -- start start 2024-09-21T03:21:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588addec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addf8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1be280 con 0x5588ada18390 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1bfeb0 con 0x5588ada39070 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.706+0000 7f204727d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1be950 con 0x5588ada101b0 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588addec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:51022/0 (socket says 172.21.15.83:51022) 2024-09-21T03:21:46.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203ad9d640 1 -- 172.21.15.83:0/3650677625 learned_addr learned my addr 172.21.15.83:0/3650677625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 -- 172.21.15.83:0/3650677625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 msgr2=0x5588addf85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f2039d9b640 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addf8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 -- 172.21.15.83:0/3650677625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 msgr2=0x5588addf8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addf8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 -- 172.21.15.83:0/3650677625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588addfe5b0 con 0x5588ada101b0 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f2039d9b640 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addf8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.707+0000 7f203a59c640 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588addec2a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20300e4f20 tx=0x7f20300a9cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1023b827f19005c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.708+0000 7f2023fff640 1 -- 172.21.15.83:0/3650677625 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300b33e0 con 0x5588ada101b0 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.708+0000 7f2023fff640 1 -- 172.21.15.83:0/3650677625 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20300ac7d0 con 0x5588ada101b0 2024-09-21T03:21:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.708+0000 7f2023fff640 1 -- 172.21.15.83:0/3650677625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300ac4a0 con 0x5588ada101b0 2024-09-21T03:21:46.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- 172.21.15.83:0/3650677625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 msgr2=0x5588addec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588addec2a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20300e4f20 tx=0x7f20300a9cf0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- 172.21.15.83:0/3650677625 shutdown_connections 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada18390 0x5588addf8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 --2- 172.21.15.83:0/3650677625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada101b0 0x5588addec2a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- 172.21.15.83:0/3650677625 >> 172.21.15.83:0/3650677625 conn(0x5588ada2d460 msgr2=0x5588addebb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- 172.21.15.83:0/3650677625 shutdown_connections 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- 172.21.15.83:0/3650677625 wait complete. 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 -- start start 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.709+0000 7f204727d740 1 Processor -- start 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada101b0 0x5588addf4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 0x5588addefcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1bf7e0 con 0x5588ada101b0 2024-09-21T03:21:46.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1bf050 con 0x5588ada39070 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1bc260 con 0x5588ada18390 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f204727d740 10 client.0 _ll_register_callbacks cb 0x7f203000a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:51032/0 (socket says 172.21.15.83:51032) 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada101b0 0x5588addf4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 -- 172.21.15.83:0/3868708388 learned_addr learned my addr 172.21.15.83:0/3868708388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f2039d9b640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 0x5588addefcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 -- 172.21.15.83:0/3868708388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 msgr2=0x5588addefcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 0x5588addefcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 -- 172.21.15.83:0/3868708388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada101b0 msgr2=0x5588addf4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada101b0 0x5588addf4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 -- 172.21.15.83:0/3868708388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588addf08e0 con 0x5588ada39070 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f2039d9b640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 0x5588addefcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:46.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f203ad9d640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2034003c20 tx=0x7f203402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b07bea886d1ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.710+0000 7f2021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034056060 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20300ac7d0 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f20340047c0 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f203402f660 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 10 client.25853 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5588ade68800 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20300ac7d0 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f2034044040 con 0x5588ada39070 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 10 client.25853 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 10 client.25853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 20 client.25853 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9ccda0 0x5588ade6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 4 client.25853 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588add548a0 con 0x5588ad9ccda0 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.711+0000 7f203a59c640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9ccda0 0x5588ade6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f2021ffb640 10 client.25853 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f203a59c640 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9ccda0 0x5588ade6b050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9360a2f50d549af server_cookie=39f7daed16ad4e44 in_seq=0 out_seq=0 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f2021ffb640 1 -- 172.21.15.83:0/3868708388 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588add548a0 con 0x5588ad9ccda0 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f2021ffb640 10 client.25853 handle_command_reply: tid=0 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 client.25853 shutdown 2024-09-21T03:21:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 msgr2=0x5588addf01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2034003c20 tx=0x7f203402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 shutdown_connections 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5588ad9ccda0 0x5588ade6b050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588ada39070 0x5588addf01f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588ada18390 0x5588addefcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.712+0000 7f2021ffb640 0 client.25853 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f204727d740 1 --2- 172.21.15.83:0/3868708388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ada101b0 0x5588addf4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 >> 172.21.15.83:0/3868708388 conn(0x5588ada2d460 msgr2=0x5588added0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 shutdown_connections 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f204727d740 1 -- 172.21.15.83:0/3868708388 wait complete. 2024-09-21T03:21:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f204727d740 20 client.25853 trim_cache size 0 max 16384 2024-09-21T03:21:46.720 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:21:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 msgr2=0x7f20240b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 0x7f20240b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2034005e30 tx=0x7f2034005c50 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 msgr2=0x7f203c113060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 0x7f203c113060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f203000c970 tx=0x7f203000ce30 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 shutdown_connections 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f20240b66b0 0x7f20240b8bc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203c1ae650 0x7f203c1aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c105af0 0x7f203c113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 --2- 172.21.15.83:0/1944015606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f203c07cf30 0x7f203c113060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.713+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 >> 172.21.15.83:0/1944015606 conn(0x7f203c0795e0 msgr2=0x7f203c0799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.715+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 shutdown_connections 2024-09-21T03:21:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:46.715+0000 7f2040d09640 1 -- 172.21.15.83:0/1944015606 wait complete. 2024-09-21T03:21:47.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:47 smithi083 ceph-mon[31219]: pgmap v479: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-09-21T03:21:47.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:21:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:47 smithi142 ceph-mon[34512]: pgmap v479: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-09-21T03:21:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:47 smithi193 ceph-mon[35224]: pgmap v479: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-09-21T03:21:47.930 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1200241423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105ac0 msgr2=0x7fb29c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1200241423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105ac0 0x7fb29c10f530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb294009ca0 tx=0x7fb29402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1200241423 shutdown_connections 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1200241423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105ac0 0x7fb29c10f530 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1200241423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105170 0x7fb29c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1200241423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 0x7fb29c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1200241423 >> 172.21.15.83:0/1200241423 conn(0x7fb29c077c60 msgr2=0x7fb29c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1200241423 shutdown_connections 2024-09-21T03:21:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.425+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1200241423 wait complete. 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 Processor -- start 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 -- start start 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 0x7fb29c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105ac0 0x7fb29c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1129c0 con 0x7fb29c105170 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112840 con 0x7fb29c105ac0 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a29c1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112b40 con 0x7fb29c1031a0 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47090/0 (socket says 172.21.15.83:47090) 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 0x7fb29c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb293fff640 1 -- 172.21.15.83:0/1504568931 learned_addr learned my addr 172.21.15.83:0/1504568931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.426+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105ac0 0x7fb29c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.427+0000 7fb293fff640 1 -- 172.21.15.83:0/1504568931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 msgr2=0x7fb29c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.427+0000 7fb293fff640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 0x7fb29c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.427+0000 7fb293fff640 1 -- 172.21.15.83:0/1504568931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105ac0 msgr2=0x7fb29c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.427+0000 7fb293fff640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105ac0 0x7fb29c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.427+0000 7fb293fff640 1 -- 172.21.15.83:0/1504568931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c1a6520 con 0x7fb29c105170 2024-09-21T03:21:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.429+0000 7fb293fff640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb28400e9f0 tx=0x7fb28400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.429+0000 7fb299ffb640 1 -- 172.21.15.83:0/1504568931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28400ccc0 con 0x7fb29c105170 2024-09-21T03:21:48.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.429+0000 7fb299ffb640 1 -- 172.21.15.83:0/1504568931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840108e0 con 0x7fb29c105170 2024-09-21T03:21:48.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.429+0000 7fb299ffb640 1 -- 172.21.15.83:0/1504568931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284018860 con 0x7fb29c105170 2024-09-21T03:21:48.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.430+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c1a6810 con 0x7fb29c105170 2024-09-21T03:21:48.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.430+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a6d00 con 0x7fb29c105170 2024-09-21T03:21:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.433+0000 7fb2a8f35740 1 Processor -- start 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 -- start start 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f1abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b52390 0x55d565f1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d565e90ea0 con 0x55d565b73070 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d565e802d0 con 0x55d565b4a1b0 2024-09-21T03:21:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d565e840e0 con 0x55d565b52390 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb299ffb640 1 -- 172.21.15.83:0/1504568931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb284010430 con 0x7fb29c105170 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a0f5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a0f5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47092/0 (socket says 172.21.15.83:47092) 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.434+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/3496426141 learned_addr learned my addr 172.21.15.83:0/3496426141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/3496426141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b52390 msgr2=0x55d565f1b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b52390 0x55d565f1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/3496426141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 msgr2=0x55d565f1abe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f1abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/3496426141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d565bde650 con 0x55d565b73070 2024-09-21T03:21:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.435+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb294035000 tx=0x7fb29402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c4d819137faf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.436+0000 7fb299ffb640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 0x7fb2880b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.436+0000 7fb299ffb640 1 -- 172.21.15.83:0/1504568931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb2840e7880 con 0x7fb29c105170 2024-09-21T03:21:48.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.447+0000 7fb2927fc640 1 -- 172.21.15.83:0/3496426141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294031cf0 con 0x55d565b73070 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.447+0000 7fb2927fc640 1 -- 172.21.15.83:0/3496426141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb294036030 con 0x55d565b73070 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.447+0000 7fb2927fc640 1 -- 172.21.15.83:0/3496426141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294004040 con 0x55d565b73070 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 -- 172.21.15.83:0/3496426141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 msgr2=0x55d565f1a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb294035000 tx=0x7fb29402f410 comp rx=0 tx=0).stop 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 -- 172.21.15.83:0/3496426141 shutdown_connections 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b52390 0x55d565f1b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f1abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/3496426141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b73070 0x55d565f1a6a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb2a8f35740 1 -- 172.21.15.83:0/3496426141 >> 172.21.15.83:0/3496426141 conn(0x55d565b67460 msgr2=0x55d565f18590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.448+0000 7fb29bfff640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 0x7fb2880b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.449+0000 7fb2a8f35740 1 -- 172.21.15.83:0/3496426141 shutdown_connections 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.451+0000 7fb2a8f35740 1 -- 172.21.15.83:0/3496426141 wait complete. 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.451+0000 7fb2a8f35740 1 -- start start 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb29bfff640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 0x7fb2880b8d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d565f19320 tx=0x7fb28c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 Processor -- start 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b4a1b0 0x55d565f264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 0x55d565f26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d565ce1700 con 0x55d565b4a1b0 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d565e1b160 con 0x55d565b52390 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d565f1d680 con 0x55d565b73070 2024-09-21T03:21:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.452+0000 7fb2a8f35740 10 client.0 _ll_register_callbacks cb 0x55d565e349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb293fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 0x55d565f26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb2a0f5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb2a0f5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60300/0 (socket says 172.21.15.83:60300) 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb293fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 0x55d565f26aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:51040/0 (socket says 172.21.15.83:51040) 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/1576634842 learned_addr learned my addr 172.21.15.83:0/1576634842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.454+0000 7fb29bfff640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b4a1b0 0x55d565f264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.455+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/1576634842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 msgr2=0x55d565f26aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.455+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 0x55d565f26aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.455+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/1576634842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b4a1b0 msgr2=0x55d565f264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.455+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b4a1b0 0x55d565f264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.455+0000 7fb2a0f5e640 1 -- 172.21.15.83:0/1576634842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d565e48c90 con 0x55d565b73070 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.456+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb294005e30 tx=0x7fb294031680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cff362dda818f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.456+0000 7fb27b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.456+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29404a040 con 0x55d565b73070 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.456+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb29402f660 con 0x55d565b73070 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.456+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb294036030 con 0x55d565b73070 2024-09-21T03:21:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.457+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb294009360 con 0x55d565b73070 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.457+0000 7fb2a8f35740 10 client.25855 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.457+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d565f2c8b0 con 0x55d565b73070 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb294036030 con 0x55d565b73070 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb29404d030 con 0x55d565b73070 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 10 client.25855 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 10 client.25855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 20 client.25855 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b319f0 0x55d565f48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 4 client.25855 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d565e92df0 con 0x55d565b319f0 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.458+0000 7fb29bfff640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b319f0 0x55d565f48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.459+0000 7fb27b7fe640 10 client.25855 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:21:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.459+0000 7fb29bfff640 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b319f0 0x55d565f48e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f819dc36f35142b server_cookie=a2f51d97090186fe in_seq=0 out_seq=0 2024-09-21T03:21:48.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.460+0000 7fb27b7fe640 1 -- 172.21.15.83:0/1576634842 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d565e92df0 con 0x55d565b319f0 2024-09-21T03:21:48.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.460+0000 7fb27b7fe640 10 client.25855 handle_command_reply: tid=0 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.460+0000 7fb2a8f35740 1 client.25855 shutdown 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 msgr2=0x55d565f30c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb294005e30 tx=0x7fb294031680 comp rx=0 tx=0).stop 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 shutdown_connections 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b319f0 0x55d565f48e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f30c70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb27b7fe640 0 client.25855 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b52390 0x55d565f26aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/1576634842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b4a1b0 0x55d565f264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 >> 172.21.15.83:0/1576634842 conn(0x55d565b67460 msgr2=0x55d565f1b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 shutdown_connections 2024-09-21T03:21:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 1 -- 172.21.15.83:0/1576634842 wait complete. 2024-09-21T03:21:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.461+0000 7fb2a8f35740 20 client.25855 trim_cache size 0 max 16384 2024-09-21T03:21:48.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.478+0000 7fb2a8f35740 1 Processor -- start 2024-09-21T03:21:48.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.478+0000 7fb2a8f35740 1 -- start start 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ad240 con 0x55d565b52390 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1aee70 con 0x55d565b4a1b0 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ad910 con 0x55d565b73070 2024-09-21T03:21:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47110/0 (socket says 172.21.15.83:47110) 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 -- 172.21.15.83:0/246053178 learned_addr learned my addr 172.21.15.83:0/246053178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb29bfff640 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 -- 172.21.15.83:0/246053178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 msgr2=0x55d565f18370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f18370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 -- 172.21.15.83:0/246053178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 msgr2=0x55d565f20820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f20820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 -- 172.21.15.83:0/246053178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d565f1cbc0 con 0x55d565b52390 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.479+0000 7fb293fff640 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2840a9d70 tx=0x7fb28400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd2e16d5c50f6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.480+0000 7fb291ffb640 1 -- 172.21.15.83:0/246053178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2840ad2b0 con 0x55d565b52390 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.480+0000 7fb291ffb640 1 -- 172.21.15.83:0/246053178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840b21d0 con 0x55d565b52390 2024-09-21T03:21:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.480+0000 7fb291ffb640 1 -- 172.21.15.83:0/246053178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2840ac6d0 con 0x55d565b52390 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.480+0000 7fb2a8f35740 1 -- 172.21.15.83:0/246053178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 msgr2=0x55d565f2b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.480+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2840a9d70 tx=0x7fb28400cef0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 -- 172.21.15.83:0/246053178 shutdown_connections 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f18370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f2b050 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/246053178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f20820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 -- 172.21.15.83:0/246053178 >> 172.21.15.83:0/246053178 conn(0x55d565b67460 msgr2=0x55d565f2a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 -- 172.21.15.83:0/246053178 shutdown_connections 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 -- 172.21.15.83:0/246053178 wait complete. 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 -- start start 2024-09-21T03:21:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.481+0000 7fb2a8f35740 1 Processor -- start 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ae7a0 con 0x55d565b52390 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ae010 con 0x55d565b4a1b0 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ab240 con 0x55d565b73070 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a8f35740 10 client.0 _ll_register_callbacks cb 0x7fb294009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47124/0 (socket says 172.21.15.83:47124) 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb293fff640 1 -- 172.21.15.83:0/561196213 learned_addr learned my addr 172.21.15.83:0/561196213 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb2a0f5e640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 -- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 msgr2=0x55d565f552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 -- 172.21.15.83:0/561196213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 msgr2=0x55d565f5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f5a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 -- 172.21.15.83:0/561196213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d565f55800 con 0x55d565b4a1b0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb29bfff640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f5a310 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55d565e803c0 tx=0x7fb28c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde99cf690dce675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb27b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.482+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01e420 con 0x55d565b4a1b0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.483+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2840b21d0 con 0x55d565b4a1b0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.483+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fb28c01e5c0 con 0x55d565b4a1b0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.483+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb28c01e7e0 con 0x55d565b4a1b0 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.483+0000 7fb2a8f35740 10 client.25868 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.483+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d565fb3c00 con 0x55d565b4a1b0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840b21d0 con 0x55d565b4a1b0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb28c039980 con 0x55d565b4a1b0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 10 client.25868 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 10 client.25868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 20 client.25868 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b06da0 0x55d565fb6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 4 client.25868 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d565e8e8a0 con 0x55d565b06da0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.485+0000 7fb293fff640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b06da0 0x55d565fb6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.486+0000 7fb27b7fe640 10 client.25868 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.486+0000 7fb293fff640 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b06da0 0x55d565fb6360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f54f1d92651ae2b server_cookie=661f58e30f8d8b9d in_seq=0 out_seq=0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.486+0000 7fb27b7fe640 1 -- 172.21.15.83:0/561196213 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d565e8e8a0 con 0x55d565b06da0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.486+0000 7fb27b7fe640 10 client.25868 handle_command_reply: tid=0 2024-09-21T03:21:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.486+0000 7fb2a8f35740 1 client.25868 shutdown 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 msgr2=0x55d565f5a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f5a310 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55d565e803c0 tx=0x7fb28c014000 comp rx=0 tx=0).stop 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 shutdown_connections 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d565b06da0 0x55d565fb6360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d565b73070 0x55d565f552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d565b52390 0x55d565f5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 --2- 172.21.15.83:0/561196213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d565b4a1b0 0x55d565f5a310 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 >> 172.21.15.83:0/561196213 conn(0x55d565b67460 msgr2=0x55d565f27790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb27b7fe640 0 client.25868 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 shutdown_connections 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 1 -- 172.21.15.83:0/561196213 wait complete. 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.487+0000 7fb2a8f35740 20 client.25868 trim_cache size 0 max 16384 2024-09-21T03:21:48.491 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 msgr2=0x7fb2880b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 0x7fb2880b8d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d565f19320 tx=0x7fb28c005f90 comp rx=0 tx=0).stop 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 msgr2=0x7fb29c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb28400e9f0 tx=0x7fb28400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 shutdown_connections 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2880b6850 0x7fb2880b8d60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb29c105ac0 0x7fb29c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c105170 0x7fb29c19b750 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 --2- 172.21.15.83:0/1504568931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb29c1031a0 0x7fb29c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 >> 172.21.15.83:0/1504568931 conn(0x7fb29c077c60 msgr2=0x7fb29c1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 shutdown_connections 2024-09-21T03:21:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:48.488+0000 7fb2a29c1640 1 -- 172.21.15.83:0/1504568931 wait complete. 2024-09-21T03:21:49.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:21:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:49 smithi142 ceph-mon[34512]: pgmap v480: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-09-21T03:21:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:49 smithi193 ceph-mon[35224]: pgmap v480: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-09-21T03:21:49.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:49 smithi083 ceph-mon[31219]: pgmap v480: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 796 op/s 2024-09-21T03:21:50.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.364+0000 7f3449d75640 1 -- 172.21.15.83:0/1107143430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344410a9e0 msgr2=0x7f344410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.364+0000 7f3449d75640 1 --2- 172.21.15.83:0/1107143430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344410a9e0 0x7f344410adf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3434009900 tx=0x7f343402a420 comp rx=0 tx=0).stop 2024-09-21T03:21:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.370+0000 7f3449d75640 1 -- 172.21.15.83:0/1107143430 shutdown_connections 2024-09-21T03:21:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.370+0000 7f3449d75640 1 --2- 172.21.15.83:0/1107143430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344411cc30 0x7f344411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.370+0000 7f3449d75640 1 --2- 172.21.15.83:0/1107143430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344410a9e0 0x7f344410adf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.370+0000 7f3449d75640 1 --2- 172.21.15.83:0/1107143430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3444111f40 0x7f344410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.370+0000 7f3449d75640 1 -- 172.21.15.83:0/1107143430 >> 172.21.15.83:0/1107143430 conn(0x7f3444070ec0 msgr2=0x7f34440712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.371+0000 7f3449d75640 1 -- 172.21.15.83:0/1107143430 shutdown_connections 2024-09-21T03:21:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.375+0000 7f3449d75640 1 -- 172.21.15.83:0/1107143430 wait complete. 2024-09-21T03:21:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 Processor -- start 2024-09-21T03:21:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 -- start start 2024-09-21T03:21:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344410a9e0 0x7f344411a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3444111f40 0x7f34441136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3444122960 con 0x7f344411cc30 2024-09-21T03:21:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34441227e0 con 0x7f3444111f40 2024-09-21T03:21:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.376+0000 7f3449d75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3444122ae0 con 0x7f344410a9e0 2024-09-21T03:21:50.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.377+0000 7f3443fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.377+0000 7f3443fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47844/0 (socket says 172.21.15.83:47844) 2024-09-21T03:21:50.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.377+0000 7f3443fff640 1 -- 172.21.15.83:0/3452364048 learned_addr learned my addr 172.21.15.83:0/3452364048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:50.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f34437fe640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344410a9e0 0x7f344411a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 -- 172.21.15.83:0/3452364048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344410a9e0 msgr2=0x7f344411a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344410a9e0 0x7f344411a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 -- 172.21.15.83:0/3452364048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3444111f40 msgr2=0x7f34441136f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3444111f40 0x7f34441136f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 -- 172.21.15.83:0/3452364048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34441144b0 con 0x7f344411cc30 2024-09-21T03:21:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.378+0000 7f3443fff640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f343800b3f0 tx=0x7f343800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.379+0000 7f3440ff9640 1 -- 172.21.15.83:0/3452364048 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438015020 con 0x7f344411cc30 2024-09-21T03:21:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.379+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3444119150 con 0x7f344411cc30 2024-09-21T03:21:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.379+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3444119760 con 0x7f344411cc30 2024-09-21T03:21:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.380+0000 7f3440ff9640 1 -- 172.21.15.83:0/3452364048 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3438011ce0 con 0x7f344411cc30 2024-09-21T03:21:50.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.380+0000 7f3440ff9640 1 -- 172.21.15.83:0/3452364048 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438010e10 con 0x7f344411cc30 2024-09-21T03:21:50.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.383+0000 7f34502e9740 1 Processor -- start 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 -- start start 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc6735b1b0 0x55fc6772bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6772c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fc676a1ea0 con 0x55fc67363390 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fc676912d0 con 0x55fc67384070 2024-09-21T03:21:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f34502e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc676950e0 con 0x55fc6735b1b0 2024-09-21T03:21:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.387+0000 7f3440ff9640 1 -- 172.21.15.83:0/3452364048 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3438018020 con 0x7f344411cc30 2024-09-21T03:21:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.388+0000 7f3442ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.388+0000 7f3442ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49294/0 (socket says 172.21.15.83:49294) 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.388+0000 7f3442ffd640 1 -- 172.21.15.83:0/2991829731 learned_addr learned my addr 172.21.15.83:0/2991829731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.388+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6772c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.388+0000 7f34437fe640 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc6735b1b0 0x55fc6772bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 -- 172.21.15.83:0/2991829731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc6735b1b0 msgr2=0x55fc6772bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc6735b1b0 0x55fc6772bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 -- 172.21.15.83:0/2991829731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 msgr2=0x55fc6772c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6772c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 -- 172.21.15.83:0/2991829731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc673ef650 con 0x55fc67384070 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.389+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3434009850 tx=0x7f3434031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53de632a0b0884c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.390+0000 7f3429ffb640 1 -- 172.21.15.83:0/2991829731 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434042070 con 0x55fc67384070 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.390+0000 7f3429ffb640 1 -- 172.21.15.83:0/2991829731 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3434031be0 con 0x55fc67384070 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.390+0000 7f3440ff9640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 0x7f342c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.391+0000 7f3440ff9640 1 -- 172.21.15.83:0/3452364048 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f34380e6cc0 con 0x7f344411cc30 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.391+0000 7f34502e9740 1 -- 172.21.15.83:0/2991829731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 msgr2=0x55fc6772b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.391+0000 7f34502e9740 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3434009850 tx=0x7f3434031400 comp rx=0 tx=0).stop 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.392+0000 7f34502e9740 1 -- 172.21.15.83:0/2991829731 shutdown_connections 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.392+0000 7f34502e9740 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6772c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.392+0000 7f34502e9740 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc6735b1b0 0x55fc6772bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.392+0000 7f34502e9740 1 --2- 172.21.15.83:0/2991829731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc67384070 0x55fc6772b6a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.392+0000 7f34502e9740 1 -- 172.21.15.83:0/2991829731 >> 172.21.15.83:0/2991829731 conn(0x55fc67378460 msgr2=0x55fc67729590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.395+0000 7f34502e9740 1 -- 172.21.15.83:0/2991829731 shutdown_connections 2024-09-21T03:21:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.395+0000 7f34437fe640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 0x7f342c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.395+0000 7f34502e9740 1 -- 172.21.15.83:0/2991829731 wait complete. 2024-09-21T03:21:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.395+0000 7f34502e9740 1 -- start start 2024-09-21T03:21:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 Processor -- start 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 0x55fc67737280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67384070 0x55fc67743240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fc674f2700 con 0x55fc67384070 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fc6762c160 con 0x55fc6735b1b0 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc67739e70 con 0x55fc67363390 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.396+0000 7f34502e9740 10 client.0 _ll_register_callbacks cb 0x55fc676459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.397+0000 7f34437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.397+0000 7f34437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49300/0 (socket says 172.21.15.83:49300) 2024-09-21T03:21:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.397+0000 7f34437fe640 1 -- 172.21.15.83:0/2396386746 learned_addr learned my addr 172.21.15.83:0/2396386746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.397+0000 7f34437fe640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 0x7f342c0b8cc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f343c005c40 tx=0x7f343c005bd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.397+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 0x55fc67737280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f34437fe640 1 -- 172.21.15.83:0/2396386746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 msgr2=0x55fc67737280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f34437fe640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 0x55fc67737280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f34437fe640 1 -- 172.21.15.83:0/2396386746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67384070 msgr2=0x55fc67743240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f34437fe640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67384070 0x55fc67743240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f34437fe640 1 -- 172.21.15.83:0/2396386746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67659c90 con 0x55fc6735b1b0 2024-09-21T03:21:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.398+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 0x55fc67737280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:50.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.400+0000 7f34437fe640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f343c009fa0 tx=0x7f343c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5251e785b58118 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.401+0000 7f3422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.401+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c023070 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.401+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3434031be0 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.401+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f343c01f860 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.402+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f343c01fa60 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.402+0000 7f34502e9740 10 client.25874 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.402+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fc67745cd0 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3434031be0 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f343c00f040 con 0x55fc6735b1b0 2024-09-21T03:21:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 10 client.25874 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 20 client.25874 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc673429f0 0x55fc67749610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 4 client.25874 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.403+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc676a3df0 con 0x55fc673429f0 2024-09-21T03:21:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.405+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc673429f0 0x55fc67749610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.405+0000 7f3442ffd640 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc673429f0 0x55fc67749610 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f6c3df3f9065b3 server_cookie=241a4683e1ae54d7 in_seq=0 out_seq=0 2024-09-21T03:21:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.406+0000 7f3422ffd640 10 client.25874 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.406+0000 7f3422ffd640 1 -- 172.21.15.83:0/2396386746 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc676a3df0 con 0x55fc673429f0 2024-09-21T03:21:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.406+0000 7f3422ffd640 10 client.25874 handle_command_reply: tid=0 2024-09-21T03:21:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 client.25874 shutdown 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 msgr2=0x55fc67736d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f343c009fa0 tx=0x7f343c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 shutdown_connections 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc673429f0 0x55fc67749610 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67384070 0x55fc67743240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67363390 0x55fc67737280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 --2- 172.21.15.83:0/2396386746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67736d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 >> 172.21.15.83:0/2396386746 conn(0x55fc67378460 msgr2=0x55fc6772e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f3422ffd640 0 client.25874 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:50.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 shutdown_connections 2024-09-21T03:21:50.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.407+0000 7f34502e9740 1 -- 172.21.15.83:0/2396386746 wait complete. 2024-09-21T03:21:50.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.408+0000 7f34502e9740 20 client.25874 trim_cache size 0 max 16384 2024-09-21T03:21:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.429+0000 7f34502e9740 1 Processor -- start 2024-09-21T03:21:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.429+0000 7f34502e9740 1 -- start start 2024-09-21T03:21:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc67743140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc67733f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f343000e780 con 0x55fc67363390 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34300103b0 con 0x55fc6735b1b0 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34502e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f343000ee50 con 0x55fc67384070 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49310/0 (socket says 172.21.15.83:49310) 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 -- 172.21.15.83:0/1630335305 learned_addr learned my addr 172.21.15.83:0/1630335305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f3442ffd640 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc67743140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 -- 172.21.15.83:0/1630335305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 msgr2=0x55fc67733f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc67733f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 -- 172.21.15.83:0/1630335305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 msgr2=0x55fc67743140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc67743140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.430+0000 7f34437fe640 1 -- 172.21.15.83:0/1630335305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6772f7b0 con 0x55fc6735b1b0 2024-09-21T03:21:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.431+0000 7f34437fe640 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f343c003f10 tx=0x7f343c003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437b05b6f57d97b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.432+0000 7f34297fa640 1 -- 172.21.15.83:0/1630335305 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c059020 con 0x55fc6735b1b0 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.432+0000 7f34297fa640 1 -- 172.21.15.83:0/1630335305 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343c027860 con 0x55fc6735b1b0 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.432+0000 7f34297fa640 1 -- 172.21.15.83:0/1630335305 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c023070 con 0x55fc6735b1b0 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- 172.21.15.83:0/1630335305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 msgr2=0x55fc67737200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f343c003f10 tx=0x7f343c003a30 comp rx=0 tx=0).stop 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- 172.21.15.83:0/1630335305 shutdown_connections 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc67733f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc67743140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 --2- 172.21.15.83:0/1630335305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc67737200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- 172.21.15.83:0/1630335305 >> 172.21.15.83:0/1630335305 conn(0x55fc67378460 msgr2=0x55fc67736af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- 172.21.15.83:0/1630335305 shutdown_connections 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- 172.21.15.83:0/1630335305 wait complete. 2024-09-21T03:21:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.433+0000 7f34502e9740 1 -- start start 2024-09-21T03:21:50.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 Processor -- start 2024-09-21T03:21:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc6773b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6773b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f343000fce0 con 0x55fc67363390 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f343000f550 con 0x55fc6735b1b0 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f343000c760 con 0x55fc67384070 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f34502e9740 10 client.0 _ll_register_callbacks cb 0x7f343c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f3443fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f3443fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44610/0 (socket says 172.21.15.83:44610) 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.434+0000 7f3443fff640 1 -- 172.21.15.83:0/3955639659 learned_addr learned my addr 172.21.15.83:0/3955639659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f34437fe640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc6773b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 -- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 msgr2=0x55fc6773b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc6773b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 -- 172.21.15.83:0/3955639659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 msgr2=0x55fc6773b760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6773b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 -- 172.21.15.83:0/3955639659 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc677b30c0 con 0x55fc67384070 2024-09-21T03:21:50.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3443fff640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34380e4490 tx=0x7f34380aca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944731954af6395b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380ade60 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.435+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f34380b13b0 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343c027860 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f34380a8290 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 10 client.25873 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fc677b4340 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f343c027860 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f34380e4b80 con 0x55fc67384070 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 10 client.25873 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 20 client.25873 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc67317da0 0x55fc677b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 4 client.25873 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.436+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc6769f8a0 con 0x55fc67317da0 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.437+0000 7f34437fe640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc67317da0 0x55fc677b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.437+0000 7f3422ffd640 10 client.25873 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.437+0000 7f34437fe640 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc67317da0 0x55fc677b6b50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681888e0a524256a server_cookie=f5a5cf7ddb1f4288 in_seq=0 out_seq=0 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f3422ffd640 1 -- 172.21.15.83:0/3955639659 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc6769f8a0 con 0x55fc67317da0 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f3422ffd640 10 client.25873 handle_command_reply: tid=0 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 client.25873 shutdown 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 msgr2=0x55fc6773bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34380e4490 tx=0x7f34380aca30 comp rx=0 tx=0).stop 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 shutdown_connections 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fc67317da0 0x55fc677b6b50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc67384070 0x55fc6773bca0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc67363390 0x55fc6773b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 --2- 172.21.15.83:0/3955639659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc6735b1b0 0x55fc6773b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 >> 172.21.15.83:0/3955639659 conn(0x55fc67378460 msgr2=0x55fc6772e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f3422ffd640 0 client.25873 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 shutdown_connections 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 1 -- 172.21.15.83:0/3955639659 wait complete. 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.438+0000 7f34502e9740 20 client.25873 trim_cache size 0 max 16384 2024-09-21T03:21:50.444 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 msgr2=0x7f342c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 0x7f342c0b8cc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f343c005c40 tx=0x7f343c005bd0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 msgr2=0x7f3444113c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f343800b3f0 tx=0x7f343800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 shutdown_connections 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f342c0b67b0 0x7f342c0b8cc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f344411cc30 0x7f3444113c90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3444111f40 0x7f34441136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 --2- 172.21.15.83:0/3452364048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344410a9e0 0x7f344411a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 >> 172.21.15.83:0/3452364048 conn(0x7f3444070ec0 msgr2=0x7f344411e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 shutdown_connections 2024-09-21T03:21:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:50.439+0000 7f3449d75640 1 -- 172.21.15.83:0/3452364048 wait complete. 2024-09-21T03:21:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 270.0 s, 2132.8 tps, lat 47.267 ms stddev 48.524 2024-09-21T03:21:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:51 smithi083 ceph-mon[31219]: pgmap v481: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T03:21:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:51 smithi142 ceph-mon[34512]: pgmap v481: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T03:21:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:51 smithi193 ceph-mon[35224]: pgmap v481: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 726 op/s 2024-09-21T03:21:52.279 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:21:52.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:21:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 -- 172.21.15.83:0/1884491942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c102a60 msgr2=0x7fe19c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 --2- 172.21.15.83:0/1884491942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c102a60 0x7fe19c102e70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe198009900 tx=0x7fe19802a410 comp rx=0 tx=0).stop 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 -- 172.21.15.83:0/1884491942 shutdown_connections 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 --2- 172.21.15.83:0/1884491942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c108cf0 0x7fe19c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 --2- 172.21.15.83:0/1884491942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 --2- 172.21.15.83:0/1884491942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c102a60 0x7fe19c102e70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 -- 172.21.15.83:0/1884491942 >> 172.21.15.83:0/1884491942 conn(0x7fe19c079730 msgr2=0x7fe19c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 -- 172.21.15.83:0/1884491942 shutdown_connections 2024-09-21T03:21:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.200+0000 7fe1a458e640 1 -- 172.21.15.83:0/1884491942 wait complete. 2024-09-21T03:21:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 Processor -- start 2024-09-21T03:21:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 -- start start 2024-09-21T03:21:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c102a60 0x7fe19c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c108cf0 0x7fe19c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1129b0 con 0x7fe19c108cf0 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c112830 con 0x7fe19c102a60 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.201+0000 7fe1a458e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c112b30 con 0x7fe19c1083a0 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44628/0 (socket says 172.21.15.83:44628) 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2438388597 learned_addr learned my addr 172.21.15.83:0/2438388597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c108cf0 0x7fe19c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a232a640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c102a60 0x7fe19c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2438388597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c102a60 msgr2=0x7fe19c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c102a60 0x7fe19c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c108cf0 msgr2=0x7fe19c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c108cf0 0x7fe19c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2438388597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c1a0320 con 0x7fe19c1083a0 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.202+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00e9f0 tx=0x7fe18c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.203+0000 7fe1937fe640 1 -- 172.21.15.83:0/2438388597 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00ccc0 con 0x7fe19c1083a0 2024-09-21T03:21:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.203+0000 7fe1937fe640 1 -- 172.21.15.83:0/2438388597 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c0054d0 con 0x7fe19c1083a0 2024-09-21T03:21:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.203+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1a0610 con 0x7fe19c1083a0 2024-09-21T03:21:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.203+0000 7fe1937fe640 1 -- 172.21.15.83:0/2438388597 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c010430 con 0x7fe19c1083a0 2024-09-21T03:21:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.203+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a6e30 con 0x7fe19c1083a0 2024-09-21T03:21:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.206+0000 7fe1937fe640 1 -- 172.21.15.83:0/2438388597 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe18c0105d0 con 0x7fe19c1083a0 2024-09-21T03:21:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1937fe640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 0x7fe1880b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1a232a640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 0x7fe1880b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1937fe640 1 -- 172.21.15.83:0/2438388597 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe18c0046c0 con 0x7fe19c1083a0 2024-09-21T03:21:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1aab02740 1 Processor -- start 2024-09-21T03:21:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1aab02740 1 -- start start 2024-09-21T03:21:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.207+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da0074430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 0x555da0074eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1aab02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d9ffeae60 con 0x555d9fcac670 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1aab02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555d9ffda290 con 0x555d9fccd350 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1aab02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555d9ffde0a0 con 0x555d9fca4490 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 0x555da0074eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a1b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a1b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44642/0 (socket says 172.21.15.83:44642) 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a1b29640 1 -- 172.21.15.83:0/196446364 learned_addr learned my addr 172.21.15.83:0/196446364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a232a640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da0074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a232a640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 0x7fe1880b8c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe198005ab0 tx=0x7fe198005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 -- 172.21.15.83:0/196446364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 msgr2=0x555da0074970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 -- 172.21.15.83:0/196446364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 msgr2=0x555da0074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da0074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 -- 172.21.15.83:0/196446364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9fd38930 con 0x555d9fcac670 2024-09-21T03:21:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a232a640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da0074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 0x555da0074eb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe194007e40 tx=0x7fe19400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dccc952e85b6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe190ff9640 1 -- 172.21.15.83:0/196446364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194003b70 con 0x555d9fcac670 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.208+0000 7fe190ff9640 1 -- 172.21.15.83:0/196446364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe194003d10 con 0x555d9fcac670 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 -- 172.21.15.83:0/196446364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 msgr2=0x555da0074eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 0x555da0074eb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe194007e40 tx=0x7fe19400f790 comp rx=0 tx=0).stop 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 -- 172.21.15.83:0/196446364 shutdown_connections 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fcac670 0x555da0074eb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 --2- 172.21.15.83:0/196446364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da0074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 -- 172.21.15.83:0/196446364 >> 172.21.15.83:0/196446364 conn(0x555d9fcc1740 msgr2=0x555da0072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 -- 172.21.15.83:0/196446364 shutdown_connections 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.209+0000 7fe1aab02740 1 -- 172.21.15.83:0/196446364 wait complete. 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.210+0000 7fe1aab02740 1 -- start start 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.210+0000 7fe1aab02740 1 Processor -- start 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.210+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da0080420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fccd350 0x555da0090400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d9fe3b890 con 0x555d9fca4490 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555d9ff74d40 con 0x555d9fcac670 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555da0082cb0 con 0x555d9fccd350 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1aab02740 10 client.0 _ll_register_callbacks cb 0x555d9ffd8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47890/0 (socket says 172.21.15.83:47890) 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1a232a640 1 -- 172.21.15.83:0/4026586655 learned_addr learned my addr 172.21.15.83:0/4026586655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.211+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fccd350 0x555da0090400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da0080420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 -- 172.21.15.83:0/4026586655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fccd350 msgr2=0x555da0090400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fccd350 0x555da0090400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 -- 172.21.15.83:0/4026586655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 msgr2=0x555da007fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 -- 172.21.15.83:0/4026586655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da0090940 con 0x555d9fcac670 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da0080420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0ae740 tx=0x7fe18c0b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6277ea7fd3adc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0ae2c0 con 0x555d9fcac670 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe18c0adc20 con 0x555d9fcac670 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.212+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe194003d10 con 0x555d9fcac670 2024-09-21T03:21:53.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe18c005680 con 0x555d9fcac670 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 10 client.25889 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555da0093240 con 0x555d9fcac670 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe194003d10 con 0x555d9fcac670 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe18c0aa100 con 0x555d9fcac670 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 10 client.25889 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 20 client.25889 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc8bcd0 0x555da0096700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 4 client.25889 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d9ffecdb0 con 0x555d9fc8bcd0 2024-09-21T03:21:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.213+0000 7fe1a232a640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc8bcd0 0x555da0096700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.214+0000 7fe185ffb640 10 client.25889 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.214+0000 7fe1a232a640 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc8bcd0 0x555da0096700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717a6decf3a5ca27 server_cookie=81969add06ca2b5a in_seq=0 out_seq=0 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.214+0000 7fe185ffb640 1 -- 172.21.15.83:0/4026586655 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d9ffecdb0 con 0x555d9fc8bcd0 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.214+0000 7fe185ffb640 10 client.25889 handle_command_reply: tid=0 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 client.25889 shutdown 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 msgr2=0x555da0080420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da0080420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0ae740 tx=0x7fe18c0b8040 comp rx=0 tx=0).stop 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 shutdown_connections 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc8bcd0 0x555da0096700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fccd350 0x555da0090400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da0080420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 --2- 172.21.15.83:0/4026586655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 >> 172.21.15.83:0/4026586655 conn(0x555d9fcc1740 msgr2=0x555da0077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe185ffb640 0 client.25889 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 shutdown_connections 2024-09-21T03:21:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 1 -- 172.21.15.83:0/4026586655 wait complete. 2024-09-21T03:21:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.215+0000 7fe1aab02740 20 client.25889 trim_cache size 0 max 16384 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 Processor -- start 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 -- start start 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 0x555da0081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ad320 con 0x555d9fca4490 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1aef50 con 0x555d9fccd350 2024-09-21T03:21:53.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.240+0000 7fe1aab02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ad9f0 con 0x555d9fcac670 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47900/0 (socket says 172.21.15.83:47900) 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a232a640 1 -- 172.21.15.83:0/1043328220 learned_addr learned my addr 172.21.15.83:0/1043328220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 0x555da0081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a2b2b640 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 -- 172.21.15.83:0/1043328220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 msgr2=0x555da007ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da007ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 -- 172.21.15.83:0/1043328220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 msgr2=0x555da007f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a1b29640 1 -- 172.21.15.83:0/1043328220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9fc8bf00 con 0x555d9fcac670 2024-09-21T03:21:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.241+0000 7fe1a232a640 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:53.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.242+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 0x555da0081910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0aa5d0 tx=0x7fe18c004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980268b2ac484436 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.243+0000 7fe187fff640 1 -- 172.21.15.83:0/1043328220 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0ae100 con 0x555d9fcac670 2024-09-21T03:21:53.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.243+0000 7fe187fff640 1 -- 172.21.15.83:0/1043328220 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c0ae2c0 con 0x555d9fcac670 2024-09-21T03:21:53.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.243+0000 7fe187fff640 1 -- 172.21.15.83:0/1043328220 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0adc20 con 0x555d9fcac670 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 -- 172.21.15.83:0/1043328220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 msgr2=0x555da0081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 0x555da0081910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0aa5d0 tx=0x7fe18c004a60 comp rx=0 tx=0).stop 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 -- 172.21.15.83:0/1043328220 shutdown_connections 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fccd350 0x555da007ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fcac670 0x555da0081910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 --2- 172.21.15.83:0/1043328220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fca4490 0x555da007f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 -- 172.21.15.83:0/1043328220 >> 172.21.15.83:0/1043328220 conn(0x555d9fcc1740 msgr2=0x555da0081200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 -- 172.21.15.83:0/1043328220 shutdown_connections 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.244+0000 7fe1aab02740 1 -- 172.21.15.83:0/1043328220 wait complete. 2024-09-21T03:21:53.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.245+0000 7fe1aab02740 1 -- start start 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.245+0000 7fe1aab02740 1 Processor -- start 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.245+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0088bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da00890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fccd350 0x555da01000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0088bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ae880 con 0x555d9fccd350 2024-09-21T03:21:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ae0f0 con 0x555d9fcac670 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ab320 con 0x555d9fca4490 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a232a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0088bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44658/0 (socket says 172.21.15.83:44658) 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a232a640 1 -- 172.21.15.83:0/2683916481 learned_addr learned my addr 172.21.15.83:0/2683916481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1aab02740 10 client.0 _ll_register_callbacks cb 0x7fe18c0aaef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da00890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2683916481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 msgr2=0x555da0088bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0088bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2683916481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fccd350 msgr2=0x555da01000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fccd350 0x555da01000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.246+0000 7fe1a1b29640 1 -- 172.21.15.83:0/2683916481 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da0100680 con 0x555d9fcac670 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe1a1b29640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da00890f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x555da0075410 tx=0x7fe18c0ade30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9846e15049ff044 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0ce070 con 0x555d9fcac670 2024-09-21T03:21:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe18c0d4050 con 0x555d9fcac670 2024-09-21T03:21:53.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c0ae2c0 con 0x555d9fcac670 2024-09-21T03:21:53.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe18c0ae100 con 0x555d9fcac670 2024-09-21T03:21:53.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe1aab02740 10 client.25895 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:53.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555da0101920 con 0x555d9fcac670 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.247+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c0b3290 con 0x555d9fcac670 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe18c0d02c0 con 0x555d9fcac670 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 10 client.25895 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 20 client.25895 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc61080 0x555da0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 4 client.25895 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe18c0ae2c0 con 0x555d9fc61080 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.248+0000 7fe1a232a640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc61080 0x555da0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.249+0000 7fe1a232a640 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc61080 0x555da0103f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fc5c670a3d1994 server_cookie=d11e86be91c9d58f in_seq=0 out_seq=0 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.249+0000 7fe185ffb640 10 client.25895 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe185ffb640 1 -- 172.21.15.83:0/2683916481 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe18c0ae2c0 con 0x555d9fc61080 2024-09-21T03:21:53.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe185ffb640 10 client.25895 handle_command_reply: tid=0 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe1aab02740 1 client.25895 shutdown 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 msgr2=0x555da00890f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da00890f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x555da0075410 tx=0x7fe18c0ade30 comp rx=0 tx=0).stop 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 shutdown_connections 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.251+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555d9fc61080 0x555da0103f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9fccd350 0x555da01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe185ffb640 0 client.25895 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d9fcac670 0x555da00890f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 --2- 172.21.15.83:0/2683916481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d9fca4490 0x555da0088bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 >> 172.21.15.83:0/2683916481 conn(0x555d9fcc1740 msgr2=0x555da0080fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 shutdown_connections 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 1 -- 172.21.15.83:0/2683916481 wait complete. 2024-09-21T03:21:53.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.252+0000 7fe1aab02740 20 client.25895 trim_cache size 0 max 16384 2024-09-21T03:21:53.257 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:21:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:21:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "172a5c6b-cfa1-4e83-87f7-40e01466199c", 2024-09-21T03:21:53.257 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:21:53.257 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.253+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 msgr2=0x7fe1880b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.253+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 0x7fe1880b8c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe198005ab0 tx=0x7fe198005a20 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 msgr2=0x7fe19c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00e9f0 tx=0x7fe18c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 shutdown_connections 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe1880b6730 0x7fe1880b8c40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c108cf0 0x7fe19c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe19c1083a0 0x7fe19c19aa10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 --2- 172.21.15.83:0/2438388597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe19c102a60 0x7fe19c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 >> 172.21.15.83:0/2438388597 conn(0x7fe19c079730 msgr2=0x7fe19c106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 shutdown_connections 2024-09-21T03:21:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:53.254+0000 7fe1a458e640 1 -- 172.21.15.83:0/2438388597 wait complete. 2024-09-21T03:21:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:53 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:21:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:53 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:53 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:53.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:53 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:53.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:53 smithi083 ceph-mon[31219]: pgmap v482: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-09-21T03:21:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:53 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:21:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:53 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:53.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:53 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:53.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:53 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:53.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:53 smithi142 ceph-mon[34512]: pgmap v482: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-09-21T03:21:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:53 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:21:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:53 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:53 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:53 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:53.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:53 smithi193 ceph-mon[35224]: pgmap v482: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 701 op/s 2024-09-21T03:21:53.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:21:54.329 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:54.629 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:54.629 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.629 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:54 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:54 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:54 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:21:54.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 -- 172.21.15.83:0/1978196836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810aee0 msgr2=0x7fa79810b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/1978196836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810aee0 0x7fa79810b2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa78800adb0 tx=0x7fa78802b7a0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 -- 172.21.15.83:0/1978196836 shutdown_connections 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/1978196836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810ead0 0x7fa7981174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/1978196836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810e180 0x7fa79810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/1978196836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810aee0 0x7fa79810b2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 -- 172.21.15.83:0/1978196836 >> 172.21.15.83:0/1978196836 conn(0x7fa7980712f0 msgr2=0x7fa798073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 -- 172.21.15.83:0/1978196836 shutdown_connections 2024-09-21T03:21:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.885+0000 7fa79f2c0640 1 -- 172.21.15.83:0/1978196836 wait complete. 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.886+0000 7fa79f2c0640 1 Processor -- start 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.886+0000 7fa79f2c0640 1 -- start start 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 0x7fa7981a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810ead0 0x7fa7981a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa79811a0a0 con 0x7fa79810e180 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa798119f20 con 0x7fa79810ead0 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79f2c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa79811a220 con 0x7fa79810aee0 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44680/0 (socket says 172.21.15.83:44680) 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 -- 172.21.15.83:0/257192663 learned_addr learned my addr 172.21.15.83:0/257192663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79c85b640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 0x7fa7981a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d85d640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810ead0 0x7fa7981a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 -- 172.21.15.83:0/257192663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810ead0 msgr2=0x7fa7981a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810ead0 0x7fa7981a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 -- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 msgr2=0x7fa7981a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 0x7fa7981a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.887+0000 7fa79d05c640 1 -- 172.21.15.83:0/257192663 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7981a8bd0 con 0x7fa79810aee0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa79c85b640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 0x7fa7981a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa79d05c640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa788005e30 tx=0x7fa788003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa7967fc640 1 -- 172.21.15.83:0/257192663 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788040070 con 0x7fa79810aee0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7981a8e60 con 0x7fa79810aee0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa7967fc640 1 -- 172.21.15.83:0/257192663 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa788032810 con 0x7fa79810aee0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7981a0760 con 0x7fa79810aee0 2024-09-21T03:21:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.888+0000 7fa7967fc640 1 -- 172.21.15.83:0/257192663 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788004570 con 0x7fa79810aee0 2024-09-21T03:21:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 Processor -- start 2024-09-21T03:21:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 -- start start 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dec390 0x5644b91b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5644b912ae50 con 0x5644b8dec390 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644b911a280 con 0x5644b8e0d070 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa7a5834740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644b911e090 con 0x5644b8de41b0 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.892+0000 7fa79d85d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49418/0 (socket says 172.21.15.83:49418) 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 -- 172.21.15.83:0/1392079288 learned_addr learned my addr 172.21.15.83:0/1392079288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa7967fc640 1 -- 172.21.15.83:0/257192663 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa788004710 con 0x7fa79810aee0 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d05c640 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 -- 172.21.15.83:0/1392079288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 msgr2=0x5644b91b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 -- 172.21.15.83:0/1392079288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dec390 msgr2=0x5644b91b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dec390 0x5644b91b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 -- 172.21.15.83:0/1392079288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b8e78650 con 0x5644b8e0d070 2024-09-21T03:21:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.893+0000 7fa79d85d640 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa790009ca0 tx=0x7fa790011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903c458d0905a140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa77f7fe640 1 -- 172.21.15.83:0/1392079288 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79001e070 con 0x5644b8e0d070 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa77f7fe640 1 -- 172.21.15.83:0/1392079288 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa790009e50 con 0x5644b8e0d070 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa77f7fe640 1 -- 172.21.15.83:0/1392079288 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000ea50 con 0x5644b8e0d070 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa7967fc640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 0x7fa7800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa79d05c640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 0x7fa7800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.894+0000 7fa7967fc640 1 -- 172.21.15.83:0/257192663 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa788108ae0 con 0x7fa79810aee0 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 -- 172.21.15.83:0/1392079288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 msgr2=0x5644b91b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa79d05c640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 0x7fa7800b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5644b91b32d0 tx=0x7fa788092ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa790009ca0 tx=0x7fa790011810 comp rx=0 tx=0).stop 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 -- 172.21.15.83:0/1392079288 shutdown_connections 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dec390 0x5644b91b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 --2- 172.21.15.83:0/1392079288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8e0d070 0x5644b91b4650 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 -- 172.21.15.83:0/1392079288 >> 172.21.15.83:0/1392079288 conn(0x5644b8e01460 msgr2=0x5644b91b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 -- 172.21.15.83:0/1392079288 shutdown_connections 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.895+0000 7fa7a5834740 1 -- 172.21.15.83:0/1392079288 wait complete. 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.896+0000 7fa7a5834740 1 -- start start 2024-09-21T03:21:54.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.896+0000 7fa7a5834740 1 Processor -- start 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.896+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8e0d070 0x5644b91cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa7a5834740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5644b8f7b700 con 0x5644b8de41b0 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49434/0 (socket says 172.21.15.83:49434) 2024-09-21T03:21:54.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa79c85b640 1 -- 172.21.15.83:0/755480328 learned_addr learned my addr 172.21.15.83:0/755480328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644b90b5110 con 0x5644b8dec390 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa79d85d640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.897+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644b91c2e30 con 0x5644b8e0d070 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 -- 172.21.15.83:0/755480328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8e0d070 msgr2=0x5644b91cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8e0d070 0x5644b91cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 -- 172.21.15.83:0/755480328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 msgr2=0x5644b91bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b90e2c40 con 0x5644b8dec390 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.898+0000 7fa79c85b640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0119c0 tx=0x7fa78c011e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b383fbd06f84a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa77cff9640 1 -- 172.21.15.83:0/755480328 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f960 con 0x5644b8dec390 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa7a5834740 10 client.0 _ll_register_callbacks cb 0x5644b90ce990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:54.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa77cff9640 1 -- 172.21.15.83:0/755480328 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa78c00fb00 con 0x5644b8dec390 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790009e50 con 0x5644b8dec390 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa77cff9640 1 -- 172.21.15.83:0/755480328 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa78c00fd00 con 0x5644b8dec390 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa7a5834740 10 client.25907 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.899+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5644b91cea10 con 0x5644b8dec390 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa77cff9640 1 -- 172.21.15.83:0/755480328 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa78c021070 con 0x5644b8dec390 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 10 client.25907 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 20 client.25907 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8dcb9f0 0x5644b91d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 4 client.25907 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa790009e50 con 0x5644b8dcb9f0 2024-09-21T03:21:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.900+0000 7fa79d85d640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8dcb9f0 0x5644b91d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.901+0000 7fa79d85d640 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8dcb9f0 0x5644b91d1ea0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514b4e7488eba529 server_cookie=92f0ed33c308d10 in_seq=0 out_seq=0 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.901+0000 7fa77cff9640 10 client.25907 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.901+0000 7fa77cff9640 1 -- 172.21.15.83:0/755480328 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa790009e50 con 0x5644b8dcb9f0 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.901+0000 7fa77cff9640 10 client.25907 handle_command_reply: tid=0 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 client.25907 shutdown 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 msgr2=0x5644b91c0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0119c0 tx=0x7fa78c011e80 comp rx=0 tx=0).stop 2024-09-21T03:21:54.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 shutdown_connections 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8dcb9f0 0x5644b91d1ea0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8e0d070 0x5644b91cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa77cff9640 0 client.25907 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c0270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 --2- 172.21.15.83:0/755480328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 >> 172.21.15.83:0/755480328 conn(0x5644b8e01460 msgr2=0x5644b91b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 shutdown_connections 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 1 -- 172.21.15.83:0/755480328 wait complete. 2024-09-21T03:21:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.902+0000 7fa7a5834740 20 client.25907 trim_cache size 0 max 16384 2024-09-21T03:21:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.926+0000 7fa7a5834740 1 Processor -- start 2024-09-21T03:21:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.926+0000 7fa7a5834740 1 -- start start 2024-09-21T03:21:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981b9f90 con 0x5644b8dcb9f0 2024-09-21T03:21:54.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981bbbc0 con 0x5644b8dec390 2024-09-21T03:21:54.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa7a5834740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981ba660 con 0x5644b8de41b0 2024-09-21T03:21:54.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79d85d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79d85d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47950/0 (socket says 172.21.15.83:47950) 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91c1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:44704/0 (socket says 172.21.15.83:44704) 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79d85d640 1 -- 172.21.15.83:0/299012914 learned_addr learned my addr 172.21.15.83:0/299012914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.927+0000 7fa79d85d640 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 -- 172.21.15.83:0/299012914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 msgr2=0x5644b91c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 -- 172.21.15.83:0/299012914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 msgr2=0x5644b91c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 -- 172.21.15.83:0/299012914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b91c04a0 con 0x5644b8dcb9f0 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa79d85d640 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa790011d10 tx=0x7fa79000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad1d635a145eac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa77effd640 1 -- 172.21.15.83:0/299012914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79001e040 con 0x5644b8dcb9f0 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa77effd640 1 -- 172.21.15.83:0/299012914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa79000eb10 con 0x5644b8dcb9f0 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.928+0000 7fa77effd640 1 -- 172.21.15.83:0/299012914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790019a30 con 0x5644b8dcb9f0 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 -- 172.21.15.83:0/299012914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 msgr2=0x5644b91cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa790011d10 tx=0x7fa79000e5f0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 -- 172.21.15.83:0/299012914 shutdown_connections 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dec390 0x5644b91c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8de41b0 0x5644b91c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 --2- 172.21.15.83:0/299012914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8dcb9f0 0x5644b91cc050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 -- 172.21.15.83:0/299012914 >> 172.21.15.83:0/299012914 conn(0x5644b8e01460 msgr2=0x5644b91c1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.929+0000 7fa7a5834740 1 -- 172.21.15.83:0/299012914 shutdown_connections 2024-09-21T03:21:54.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.930+0000 7fa7a5834740 1 -- 172.21.15.83:0/299012914 wait complete. 2024-09-21T03:21:54.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.930+0000 7fa7a5834740 1 -- start start 2024-09-21T03:21:54.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.930+0000 7fa7a5834740 1 Processor -- start 2024-09-21T03:21:54.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 0x5644b91c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa79c85b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47962/0 (socket says 172.21.15.83:47962) 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8dec390 0x5644b92359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa79c85b640 1 -- 172.21.15.83:0/3456536651 learned_addr learned my addr 172.21.15.83:0/3456536651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981bb4f0 con 0x5644b8de41b0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981bad60 con 0x5644b8dcb9f0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa79d85d640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 0x5644b91c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.931+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981b7f70 con 0x5644b8dec390 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79d85d640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8dec390 0x5644b92359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa7a5834740 10 client.0 _ll_register_callbacks cb 0x7fa78c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 -- 172.21.15.83:0/3456536651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8dec390 msgr2=0x5644b92359c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8dec390 0x5644b92359c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 -- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 msgr2=0x5644b91c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 0x5644b91c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b9236030 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79d85d640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 0x5644b91c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa79c85b640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa78c011270 tx=0x7fa78c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311504a671754146 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79000eb10 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa77cff9640 1 -- 172.21.15.83:0/3456536651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c033060 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.932+0000 7fa77cff9640 1 -- 172.21.15.83:0/3456536651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa78c005110 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa77cff9640 1 -- 172.21.15.83:0/3456536651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa78c005330 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 10 client.16377 fetch_fsmap learned FSMap version 40 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5644b923d360 con 0x5644b8de41b0 2024-09-21T03:21:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa77cff9640 1 -- 172.21.15.83:0/3456536651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa78c021040 con 0x5644b8de41b0 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 10 client.16377 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 20 client.16377 populate_metadata read hostname 'smithi083' 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8e0d070 0x5644b923fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 4 client.16377 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644b912cda0 con 0x5644b8e0d070 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.933+0000 7fa79d85d640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8e0d070 0x5644b923fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa77cff9640 10 client.16377 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa79d85d640 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8e0d070 0x5644b923fa10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253f9c65d16bcf58 server_cookie=d6faa40d6669c168 in_seq=0 out_seq=0 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa77cff9640 1 -- 172.21.15.83:0/3456536651 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x5644b912cda0 con 0x5644b8e0d070 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa77cff9640 10 client.16377 handle_command_reply: tid=0 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 client.16377 shutdown 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 msgr2=0x5644b91c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa78c011270 tx=0x7fa78c005f90 comp rx=0 tx=0).stop 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 shutdown_connections 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5644b8e0d070 0x5644b923fa10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644b8dec390 0x5644b92359c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644b8de41b0 0x5644b91c51b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 --2- 172.21.15.83:0/3456536651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b8dcb9f0 0x5644b91c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 >> 172.21.15.83:0/3456536651 conn(0x5644b8e01460 msgr2=0x5644b91b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.934+0000 7fa77cff9640 0 client.16377 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 shutdown_connections 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa7a5834740 1 -- 172.21.15.83:0/3456536651 wait complete. 2024-09-21T03:21:54.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa7a5834740 20 client.16377 trim_cache size 0 max 16384 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "172a5c6b-cfa1-4e83-87f7-40e01466199c", 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:21:54.941 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:21:54.944 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:21:54.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:54.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:21:54.946 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:21:54.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 msgr2=0x7fa7800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 0x7fa7800b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5644b91b32d0 tx=0x7fa788092ee0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 msgr2=0x7fa7981a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.935+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa788005e30 tx=0x7fa788003ce0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 shutdown_connections 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa7800b6700 0x7fa7800b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa79810ead0 0x7fa7981a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79810e180 0x7fa7981a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 --2- 172.21.15.83:0/257192663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa79810aee0 0x7fa7981a2c60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 >> 172.21.15.83:0/257192663 conn(0x7fa7980712f0 msgr2=0x7fa7981102e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 shutdown_connections 2024-09-21T03:21:54.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:21:54.936+0000 7fa79f2c0640 1 -- 172.21.15.83:0/257192663 wait complete. 2024-09-21T03:21:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:55 smithi083 ceph-mon[31219]: pgmap v483: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-09-21T03:21:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:55 smithi142 ceph-mon[34512]: pgmap v483: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-09-21T03:21:55.702 INFO:tasks.cephfs.filesystem:scrub status for tag:172a5c6b-cfa1-4e83-87f7-40e01466199c - {'path': '/', 'tag': '172a5c6b-cfa1-4e83-87f7-40e01466199c', 'options': 'recursive,force'} 2024-09-21T03:21:55.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:55 smithi193 ceph-mon[35224]: pgmap v483: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-09-21T03:21:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 275.0 s, 2346.8 tps, lat 42.581 ms stddev 46.146 2024-09-21T03:21:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:57 smithi142 ceph-mon[34512]: pgmap v484: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-09-21T03:21:57.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:57 smithi193 ceph-mon[35224]: pgmap v484: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-09-21T03:21:57.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:57 smithi083 ceph-mon[31219]: pgmap v484: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-09-21T03:21:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:58 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:58 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:58.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:58 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:21:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:21:59 smithi142 ceph-mon[34512]: pgmap v485: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 780 op/s 2024-09-21T03:21:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:21:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:21:59 smithi193 ceph-mon[35224]: pgmap v485: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 780 op/s 2024-09-21T03:21:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:21:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:21:59 smithi083 ceph-mon[31219]: pgmap v485: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 780 op/s 2024-09-21T03:22:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 280.0 s, 2051.5 tps, lat 41.346 ms stddev 43.234 2024-09-21T03:22:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:01 smithi083 ceph-mon[31219]: pgmap v486: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-21T03:22:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:01 smithi142 ceph-mon[34512]: pgmap v486: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-21T03:22:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:01 smithi193 ceph-mon[35224]: pgmap v486: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-21T03:22:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:03 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:03 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:03 smithi083 ceph-mon[31219]: pgmap v487: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T03:22:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:03 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:03 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:03 smithi142 ceph-mon[34512]: pgmap v487: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T03:22:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:03 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:03 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:03 smithi193 ceph-mon[35224]: pgmap v487: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T03:22:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:04 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:04 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:04 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:04 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:04 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:04 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:04 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:04 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:04 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:05 smithi142 ceph-mon[34512]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T03:22:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:05 smithi193 ceph-mon[35224]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T03:22:05.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:05 smithi083 ceph-mon[31219]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 31 MiB/s wr, 1.38k op/s 2024-09-21T03:22:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 285.0 s, 2468.7 tps, lat 46.698 ms stddev 128.902 2024-09-21T03:22:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:07 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:22:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:07 smithi193 ceph-mon[35224]: pgmap v489: 97 pgs: 97 active+clean; 3.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-09-21T03:22:07.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:07 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:22:07.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:07 smithi083 ceph-mon[31219]: pgmap v489: 97 pgs: 97 active+clean; 3.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-09-21T03:22:07.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:07 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:22:07.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:07 smithi142 ceph-mon[34512]: pgmap v489: 97 pgs: 97 active+clean; 3.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-09-21T03:22:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:09 smithi142 ceph-mon[34512]: pgmap v490: 97 pgs: 97 active+clean; 3.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-09-21T03:22:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:09 smithi193 ceph-mon[35224]: pgmap v490: 97 pgs: 97 active+clean; 3.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-09-21T03:22:09.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:09 smithi083 ceph-mon[31219]: pgmap v490: 97 pgs: 97 active+clean; 3.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-09-21T03:22:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 290.0 s, 2632.2 tps, lat 37.912 ms stddev 39.622 2024-09-21T03:22:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:11 smithi142 ceph-mon[34512]: pgmap v491: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-09-21T03:22:11.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:11 smithi193 ceph-mon[35224]: pgmap v491: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-09-21T03:22:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:11 smithi083 ceph-mon[31219]: pgmap v491: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-09-21T03:22:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:13 smithi142 ceph-mon[34512]: pgmap v492: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-09-21T03:22:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:13 smithi193 ceph-mon[35224]: pgmap v492: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-09-21T03:22:13.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:13 smithi083 ceph-mon[31219]: pgmap v492: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-09-21T03:22:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:15 smithi142 ceph-mon[34512]: pgmap v493: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-21T03:22:15.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:15 smithi193 ceph-mon[35224]: pgmap v493: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-21T03:22:15.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:15 smithi083 ceph-mon[31219]: pgmap v493: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-21T03:22:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 295.0 s, 2754.4 tps, lat 36.315 ms stddev 37.276 2024-09-21T03:22:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:17 smithi142 ceph-mon[34512]: pgmap v494: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-21T03:22:17.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:17 smithi193 ceph-mon[35224]: pgmap v494: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-21T03:22:17.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:17 smithi083 ceph-mon[31219]: pgmap v494: 97 pgs: 97 active+clean; 3.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-21T03:22:19.673 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:19 smithi083 ceph-mon[31219]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T03:22:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:19 smithi142 ceph-mon[34512]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T03:22:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:19 smithi193 ceph-mon[35224]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 770 op/s 2024-09-21T03:22:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 300.0 s, 2340.6 tps, lat 42.564 ms stddev 43.527 2024-09-21T03:22:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:21 smithi083 ceph-mon[31219]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-09-21T03:22:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:21 smithi142 ceph-mon[34512]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-09-21T03:22:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:21 smithi193 ceph-mon[35224]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 647 op/s 2024-09-21T03:22:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:23 smithi142 ceph-mon[34512]: pgmap v497: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 728 op/s 2024-09-21T03:22:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:23 smithi193 ceph-mon[35224]: pgmap v497: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 728 op/s 2024-09-21T03:22:23.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:23 smithi083 ceph-mon[31219]: pgmap v497: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 728 op/s 2024-09-21T03:22:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:25 smithi142 ceph-mon[34512]: pgmap v498: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 632 op/s 2024-09-21T03:22:25.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:22:25.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:25 smithi083 ceph-mon[31219]: pgmap v498: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 632 op/s 2024-09-21T03:22:25.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:25 smithi193 ceph-mon[35224]: pgmap v498: 97 pgs: 97 active+clean; 3.9 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 632 op/s 2024-09-21T03:22:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 305.0 s, 2078.2 tps, lat 47.903 ms stddev 50.800 2024-09-21T03:22:26.054 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:26.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 -- 172.21.15.83:0/623420628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8112dc0 msgr2=0x7f95e81151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/623420628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8112dc0 0x7f95e81151e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009900 tx=0x7f95dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 -- 172.21.15.83:0/623420628 shutdown_connections 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/623420628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8115720 0x7f95e8117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/623420628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8112dc0 0x7f95e81151e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/623420628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 -- 172.21.15.83:0/623420628 >> 172.21.15.83:0/623420628 conn(0x7f95e8070ec0 msgr2=0x7f95e80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 -- 172.21.15.83:0/623420628 shutdown_connections 2024-09-21T03:22:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.656+0000 7f95ecc5e640 1 -- 172.21.15.83:0/623420628 wait complete. 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.657+0000 7f95ecc5e640 1 Processor -- start 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.657+0000 7f95ecc5e640 1 -- start start 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 0x7f95e81a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 0x7f95e81a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e811b180 con 0x7f95e8115720 2024-09-21T03:22:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95e811b000 con 0x7f95e810bff0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95ecc5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95e811b300 con 0x7f95e8112dc0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53696/0 (socket says 172.21.15.83:53696) 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e659c640 1 -- 172.21.15.83:0/3001424631 learned_addr learned my addr 172.21.15.83:0/3001424631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 0x7f95e81a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 0x7f95e81a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 -- 172.21.15.83:0/3001424631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 msgr2=0x7f95e81a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 -- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 msgr2=0x7f95e81a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 0x7f95e81a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e5d9b640 1 -- 172.21.15.83:0/3001424631 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e81a1900 con 0x7f95e8112dc0 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e659c640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.658+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 0x7f95e81a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 0x7f95e81a4050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02a930 tx=0x7f95dc004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95d77fe640 1 -- 172.21.15.83:0/3001424631 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc03f070 con 0x7f95e8112dc0 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81a1b90 con 0x7f95e8112dc0 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95d77fe640 1 -- 172.21.15.83:0/3001424631 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc003ca0 con 0x7f95e8112dc0 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81a2090 con 0x7f95e8112dc0 2024-09-21T03:22:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.659+0000 7f95d77fe640 1 -- 172.21.15.83:0/3001424631 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc035070 con 0x7f95e8112dc0 2024-09-21T03:22:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.663+0000 7f95d77fe640 1 -- 172.21.15.83:0/3001424631 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f95dc043400 con 0x7f95e8112dc0 2024-09-21T03:22:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.664+0000 7f95d77fe640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 0x7f95d00b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.664+0000 7f95d77fe640 1 -- 172.21.15.83:0/3001424631 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f95dc107930 con 0x7f95e8112dc0 2024-09-21T03:22:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.664+0000 7f95e659c640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 0x7f95d00b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 Processor -- start 2024-09-21T03:22:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 -- start start 2024-09-21T03:22:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e659c640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 0x7f95d00b8cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95e810b7b0 tx=0x7f95d8005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 0x559e3eb67b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e3eadde50 con 0x559e3e7c0070 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e3eacd280 con 0x559e3e7971b0 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95f31d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559e3ead1090 con 0x559e3e79f390 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34176/0 (socket says 172.21.15.83:34176) 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e659c640 1 -- 172.21.15.83:0/4277424667 learned_addr learned my addr 172.21.15.83:0/4277424667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 0x559e3eb67b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.665+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 -- 172.21.15.83:0/4277424667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 msgr2=0x559e3eb680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 -- 172.21.15.83:0/4277424667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 msgr2=0x559e3eb67650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 -- 172.21.15.83:0/4277424667 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3e82b650 con 0x559e3e7971b0 2024-09-21T03:22:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e659c640 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:26.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 0x559e3eb67b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1050a0 tx=0x7f95dc0c90f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98d7028a34a8470 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95d4ff9640 1 -- 172.21.15.83:0/4277424667 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc048070 con 0x559e3e7971b0 2024-09-21T03:22:26.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95d4ff9640 1 -- 172.21.15.83:0/4277424667 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc105520 con 0x559e3e7971b0 2024-09-21T03:22:26.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.666+0000 7f95d4ff9640 1 -- 172.21.15.83:0/4277424667 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0d6f90 con 0x559e3e7971b0 2024-09-21T03:22:26.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.667+0000 7f95f31d2740 1 -- 172.21.15.83:0/4277424667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 msgr2=0x559e3eb67b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.667+0000 7f95f31d2740 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 0x559e3eb67b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1050a0 tx=0x7f95dc0c90f0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 -- 172.21.15.83:0/4277424667 shutdown_connections 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7971b0 0x559e3eb67b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 --2- 172.21.15.83:0/4277424667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7c0070 0x559e3eb67650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 -- 172.21.15.83:0/4277424667 >> 172.21.15.83:0/4277424667 conn(0x559e3e7b4460 msgr2=0x559e3eb65540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 -- 172.21.15.83:0/4277424667 shutdown_connections 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 -- 172.21.15.83:0/4277424667 wait complete. 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 -- start start 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.668+0000 7f95f31d2740 1 Processor -- start 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb72d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 0x559e3eb7ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e3e92e700 con 0x559e3e7971b0 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e3ea68110 con 0x559e3e7c0070 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559e3eb75bf0 con 0x559e3e79f390 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95f31d2740 10 client.0 _ll_register_callbacks cb 0x559e3ea81990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb72d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb72d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34182/0 (socket says 172.21.15.83:34182) 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e659c640 1 -- 172.21.15.83:0/1253770868 learned_addr learned my addr 172.21.15.83:0/1253770868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 0x559e3eb7ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 -- 172.21.15.83:0/1253770868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 msgr2=0x559e3eb732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 -- 172.21.15.83:0/1253770868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 msgr2=0x559e3eb72d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb72d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 -- 172.21.15.83:0/1253770868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3ea95c40 con 0x559e3e7c0070 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 0x559e3eb7ee10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f95e0011d30 tx=0x7f95e0013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c97ebdfe1d3744 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.669+0000 7f95cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00107e0 con 0x559e3e7c0070 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f95e0010980 con 0x559e3e7c0070 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc105520 con 0x559e3e7c0070 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95e0010b80 con 0x559e3e7c0070 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 10 client.25925 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559e3eb818d0 con 0x559e3e7c0070 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc105520 con 0x559e3e7c0070 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f95e001a070 con 0x559e3e7c0070 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 10 client.25925 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 20 client.25925 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e77e9f0 0x559e3eb84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 4 client.25925 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3eadfda0 con 0x559e3e77e9f0 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.670+0000 7f95e659c640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e77e9f0 0x559e3eb84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.671+0000 7f95cdffb640 10 client.25925 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.671+0000 7f95e659c640 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e77e9f0 0x559e3eb84e60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845cae61b0167a server_cookie=7c5b795dc18df6fa in_seq=0 out_seq=0 2024-09-21T03:22:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95cdffb640 1 -- 172.21.15.83:0/1253770868 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e3eadfda0 con 0x559e3e77e9f0 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95cdffb640 10 client.25925 handle_command_reply: tid=0 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 client.25925 shutdown 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 msgr2=0x559e3eb7ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 0x559e3eb7ee10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f95e0011d30 tx=0x7f95e0013830 comp rx=0 tx=0).stop 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 shutdown_connections 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e77e9f0 0x559e3eb84e60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.672+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e7c0070 0x559e3eb7ee10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95cdffb640 0 client.25925 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 1 --2- 172.21.15.83:0/1253770868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb72d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 >> 172.21.15.83:0/1253770868 conn(0x559e3e7b4460 msgr2=0x559e3eb6a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 shutdown_connections 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 1 -- 172.21.15.83:0/1253770868 wait complete. 2024-09-21T03:22:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.673+0000 7f95f31d2740 20 client.25925 trim_cache size 0 max 16384 2024-09-21T03:22:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 Processor -- start 2024-09-21T03:22:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 -- start start 2024-09-21T03:22:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb7ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 0x559e3eb6ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95c800e620 con 0x559e3e7971b0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8010220 con 0x559e3e79f390 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95f31d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95c800ecf0 con 0x559e3e77e9f0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95e659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95e5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb7ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95e659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35422/0 (socket says 172.21.15.83:35422) 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.696+0000 7f95e659c640 1 -- 172.21.15.83:0/3493275150 learned_addr learned my addr 172.21.15.83:0/3493275150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 0x559e3eb6ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 -- 172.21.15.83:0/3493275150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 msgr2=0x559e3eb6ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 0x559e3eb6ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 -- 172.21.15.83:0/3493275150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 msgr2=0x559e3eb7ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb7ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 -- 172.21.15.83:0/3493275150 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3eb74040 con 0x559e3e77e9f0 2024-09-21T03:22:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e6d9d640 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 0x559e3eb6ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95e659c640 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009fd0 tx=0x7f95d8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=320872bec9299ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95cffff640 1 -- 172.21.15.83:0/3493275150 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800f9b0 con 0x559e3e77e9f0 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95cffff640 1 -- 172.21.15.83:0/3493275150 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d802a480 con 0x559e3e77e9f0 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.697+0000 7f95cffff640 1 -- 172.21.15.83:0/3493275150 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d801e690 con 0x559e3e77e9f0 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 -- 172.21.15.83:0/3493275150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 msgr2=0x559e3eb73240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009fd0 tx=0x7f95d8014000 comp rx=0 tx=0).stop 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 -- 172.21.15.83:0/3493275150 shutdown_connections 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e79f390 0x559e3eb6ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb7ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 --2- 172.21.15.83:0/3493275150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e77e9f0 0x559e3eb73240 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 -- 172.21.15.83:0/3493275150 >> 172.21.15.83:0/3493275150 conn(0x559e3e7b4460 msgr2=0x559e3eb72b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 -- 172.21.15.83:0/3493275150 shutdown_connections 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.698+0000 7f95f31d2740 1 -- 172.21.15.83:0/3493275150 wait complete. 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 -- start start 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 Processor -- start 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 0x559e3eb7b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95c800fb50 con 0x559e3e7971b0 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95c800f3c0 con 0x559e3e77e9f0 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95c800c600 con 0x559e3e79f390 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95f31d2740 10 client.0 _ll_register_callbacks cb 0x7f95e00113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95e6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95e659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 0x559e3eb7b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95e6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb773b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35436/0 (socket says 172.21.15.83:35436) 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95e6d9d640 1 -- 172.21.15.83:0/2071971817 learned_addr learned my addr 172.21.15.83:0/2071971817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.699+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 -- 172.21.15.83:0/2071971817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 msgr2=0x559e3eb773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb773b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 -- 172.21.15.83:0/2071971817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 msgr2=0x559e3eb76ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb76ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 -- 172.21.15.83:0/2071971817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3ebee860 con 0x559e3e77e9f0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95e659c640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 0x559e3eb7b790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d8004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f821ce970680cd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800fbb0 con 0x559e3e77e9f0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f95d800fd50 con 0x559e3e77e9f0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.700+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d802a480 con 0x559e3e77e9f0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95d802a630 con 0x559e3e77e9f0 2024-09-21T03:22:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 10 client.25931 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559e3ebef9f0 con 0x559e3e77e9f0 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d802a480 con 0x559e3e77e9f0 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f95d8031980 con 0x559e3e77e9f0 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 10 client.25931 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 20 client.25931 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e7c0070 0x559e3ebf2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 4 client.25931 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:22:26.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e3eadb850 con 0x559e3e7c0070 2024-09-21T03:22:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.701+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e7c0070 0x559e3ebf2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.703+0000 7f95e5d9b640 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e7c0070 0x559e3ebf2150 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfb76f6249199a5 server_cookie=b10946856bc775e0 in_seq=0 out_seq=0 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.703+0000 7f95cdffb640 10 client.25931 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95cdffb640 1 -- 172.21.15.83:0/2071971817 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x559e3eadb850 con 0x559e3e7c0070 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95cdffb640 10 client.25931 handle_command_reply: tid=0 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 client.25931 shutdown 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 msgr2=0x559e3eb7b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 0x559e3eb7b790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d8004320 comp rx=0 tx=0).stop 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 shutdown_connections 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559e3e7c0070 0x559e3ebf2150 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559e3e79f390 0x559e3eb773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e3e7971b0 0x559e3eb76ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 --2- 172.21.15.83:0/2071971817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3e77e9f0 0x559e3eb7b790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.704+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 >> 172.21.15.83:0/2071971817 conn(0x559e3e7b4460 msgr2=0x559e3eb68680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.705+0000 7f95cdffb640 0 client.25931 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.705+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 shutdown_connections 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.705+0000 7f95f31d2740 1 -- 172.21.15.83:0/2071971817 wait complete. 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.705+0000 7f95f31d2740 20 client.25931 trim_cache size 0 max 16384 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.709 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.710 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.712 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:22:26.713 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 msgr2=0x7f95d00b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 0x7f95d00b8cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95e810b7b0 tx=0x7f95d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 msgr2=0x7f95e81a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 0x7f95e81a4050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02a930 tx=0x7f95dc004060 comp rx=0 tx=0).stop 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 shutdown_connections 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95d00b67b0 0x7f95d00b8cc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e8115720 0x7f95e81a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95e8112dc0 0x7f95e81a4050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 --2- 172.21.15.83:0/3001424631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e810bff0 0x7f95e81a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 >> 172.21.15.83:0/3001424631 conn(0x7f95e8070ec0 msgr2=0x7f95e8116c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 shutdown_connections 2024-09-21T03:22:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:26.706+0000 7f95ecc5e640 1 -- 172.21.15.83:0/3001424631 wait complete. 2024-09-21T03:22:27.602 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:27 smithi083 ceph-mon[31219]: pgmap v499: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-09-21T03:22:27.649 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:22:27.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:22:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:27 smithi142 ceph-mon[34512]: pgmap v499: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-09-21T03:22:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:27 smithi193 ceph-mon[35224]: pgmap v499: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-09-21T03:22:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:28.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.790+0000 7f187affd640 1 -- 172.21.15.83:0/2932601316 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c0318a0 con 0x7f187c102a90 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 -- 172.21.15.83:0/2932601316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 msgr2=0x7f187c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 --2- 172.21.15.83:0/2932601316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c102ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 -- 172.21.15.83:0/2932601316 shutdown_connections 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 --2- 172.21.15.83:0/2932601316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c108d20 0x7f187c06bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 --2- 172.21.15.83:0/2932601316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c1083d0 0x7f187c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 --2- 172.21.15.83:0/2932601316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c102ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 -- 172.21.15.83:0/2932601316 >> 172.21.15.83:0/2932601316 conn(0x7f187c079730 msgr2=0x7f187c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 -- 172.21.15.83:0/2932601316 shutdown_connections 2024-09-21T03:22:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.791+0000 7f1882043640 1 -- 172.21.15.83:0/2932601316 wait complete. 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 Processor -- start 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 -- start start 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c1155a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 0x7f187c111fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f187b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c1155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1873fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1873fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34212/0 (socket says 172.21.15.83:34212) 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1873fff640 1 -- 172.21.15.83:0/2102262979 learned_addr learned my addr 172.21.15.83:0/2102262979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f187c115cf0 con 0x7f187c1083d0 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.792+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f187c115b70 con 0x7f187c108d20 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f187c115e70 con 0x7f187c102a90 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187bfff640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 0x7f187c111fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 -- 172.21.15.83:0/2102262979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 msgr2=0x7f187c111fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 0x7f187c111fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 -- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 msgr2=0x7f187c111880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1acc40 con 0x7f187c102a90 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f1873fff640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187bfff640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 0x7f187c111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.793+0000 7f187b7fe640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c1155a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186c000c00 tx=0x7f186c0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.794+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c035030 con 0x7f187c102a90 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.794+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1ace10 con 0x7f187c102a90 2024-09-21T03:22:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.794+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186c003cb0 con 0x7f187c102a90 2024-09-21T03:22:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.794+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c0318a0 con 0x7f187c102a90 2024-09-21T03:22:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.794+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c1ad2c0 con 0x7f187c102a90 2024-09-21T03:22:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.795+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c103d10 con 0x7f187c102a90 2024-09-21T03:22:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.797+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f186c03bc90 con 0x7f187c102a90 2024-09-21T03:22:28.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.798+0000 7f18797fa640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 0x7f18680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.798+0000 7f1873fff640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 0x7f18680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.798+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f186c107dd0 con 0x7f187c102a90 2024-09-21T03:22:28.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.798+0000 7f1873fff640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 0x7f18680b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f187c079bf0 tx=0x7f1864005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:28.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:28.801+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f186c041020 con 0x7f187c102a90 2024-09-21T03:22:29.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.127+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f187c1087e0 con 0x7f187c102a90 2024-09-21T03:22:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.128+0000 7f18797fa640 1 -- 172.21.15.83:0/2102262979 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11989 (secure 0 0 0) 0x7f186c0ce3a0 con 0x7f187c102a90 2024-09-21T03:22:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:22:29.130 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"btime":"2024-09-21T03:22:06:485521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:22:06.435378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:22:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 msgr2=0x7f18680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 0x7f18680b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f187c079bf0 tx=0x7f1864005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 msgr2=0x7f187c1155a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c1155a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186c000c00 tx=0x7f186c0314a0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 shutdown_connections 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18680b6680 0x7f18680b8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f187c108d20 0x7f187c111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187c1083d0 0x7f187c111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 --2- 172.21.15.83:0/2102262979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f187c102a90 0x7f187c1155a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 >> 172.21.15.83:0/2102262979 conn(0x7f187c079730 msgr2=0x7f187c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 shutdown_connections 2024-09-21T03:22:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:29.131+0000 7f1882043640 1 -- 172.21.15.83:0/2102262979 wait complete. 2024-09-21T03:22:29.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:29 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2102262979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:22:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:29 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2102262979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:22:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:29 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2102262979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:22:29.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:22:30.321 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:30.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:30 smithi083 ceph-mon[31219]: pgmap v500: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 728 op/s 2024-09-21T03:22:30.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:30 smithi193 ceph-mon[35224]: pgmap v500: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 728 op/s 2024-09-21T03:22:30.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:30 smithi142 ceph-mon[34512]: pgmap v500: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.8 MiB/s wr, 728 op/s 2024-09-21T03:22:30.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.889+0000 7f4a5ffff640 1 -- 172.21.15.83:0/628166978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010af90 msgr2=0x7f4a60115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.889+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/628166978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010af90 0x7f4a60115fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a5002a410 comp rx=0 tx=0).stop 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 -- 172.21.15.83:0/628166978 shutdown_connections 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/628166978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60116510 0x7f4a60118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/628166978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010af90 0x7f4a60115fd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/628166978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a6010a640 0x7f4a6010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 -- 172.21.15.83:0/628166978 >> 172.21.15.83:0/628166978 conn(0x7f4a6006e9f0 msgr2=0x7f4a6006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 -- 172.21.15.83:0/628166978 shutdown_connections 2024-09-21T03:22:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 -- 172.21.15.83:0/628166978 wait complete. 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.890+0000 7f4a5ffff640 1 Processor -- start 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 -- start start 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6010af90 0x7f4a601a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60116510 0x7f4a601aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010c230 con 0x7f4a6010a640 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010c0b0 con 0x7f4a60116510 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5ffff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6010c3b0 con 0x7f4a6010af90 2024-09-21T03:22:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60116510 0x7f4a601aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54688/0 (socket says 172.21.15.83:54688) 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5effd640 1 -- 172.21.15.83:0/2251346757 learned_addr learned my addr 172.21.15.83:0/2251346757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.891+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6010af90 0x7f4a601a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 -- 172.21.15.83:0/2251346757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6010af90 msgr2=0x7f4a601a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6010af90 0x7f4a601a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 -- 172.21.15.83:0/2251346757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60116510 msgr2=0x7f4a601aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60116510 0x7f4a601aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 -- 172.21.15.83:0/2251346757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a601aac20 con 0x7f4a6010a640 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5effd640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800ebc0 tx=0x7f4a5800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a648e6640 1 -- 172.21.15.83:0/2251346757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58010630 con 0x7f4a6010a640 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a648e6640 1 -- 172.21.15.83:0/2251346757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a58010c80 con 0x7f4a6010a640 2024-09-21T03:22:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a648e6640 1 -- 172.21.15.83:0/2251346757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58018b60 con 0x7f4a6010a640 2024-09-21T03:22:30.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a601aaf10 con 0x7f4a6010a640 2024-09-21T03:22:30.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.892+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601b1690 con 0x7f4a6010a640 2024-09-21T03:22:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 Processor -- start 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 -- start start 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a47eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4423390 0x55b8a47ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8a4761ea0 con 0x55b8a441b1b0 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b8a47512d0 con 0x55b8a4444070 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.896+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8a47550e0 con 0x55b8a4423390 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59846/0 (socket says 172.21.15.83:59846) 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/3225829367 learned_addr learned my addr 172.21.15.83:0/3225829367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4423390 0x55b8a47ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/3225829367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4423390 msgr2=0x55b8a47ebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4423390 0x55b8a47ebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/3225829367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 msgr2=0x55b8a47eb9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:22:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a47eb9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/3225829367 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a44af650 con 0x55b8a4444070 2024-09-21T03:22:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b700 tx=0x7f4a5400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bccd93a1c607b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a45ffb640 1 -- 172.21.15.83:0/3225829367 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54003dd0 con 0x55b8a4444070 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a648e6640 1 -- 172.21.15.83:0/2251346757 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4a58018d00 con 0x7f4a6010a640 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a45ffb640 1 -- 172.21.15.83:0/3225829367 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a54015070 con 0x55b8a4444070 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.897+0000 7f4a45ffb640 1 -- 172.21.15.83:0/3225829367 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540106a0 con 0x55b8a4444070 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a648e6640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 0x7f4a480b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a6be5c740 1 -- 172.21.15.83:0/3225829367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 msgr2=0x55b8a47eb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b700 tx=0x7f4a5400bbc0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a5effd640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 0x7f4a480b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a6be5c740 1 -- 172.21.15.83:0/3225829367 shutdown_connections 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.898+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4423390 0x55b8a47ebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a47eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/3225829367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4444070 0x55b8a47eb460 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 -- 172.21.15.83:0/3225829367 >> 172.21.15.83:0/3225829367 conn(0x55b8a4438460 msgr2=0x55b8a47e9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 -- 172.21.15.83:0/3225829367 shutdown_connections 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 -- 172.21.15.83:0/3225829367 wait complete. 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a648e6640 1 -- 172.21.15.83:0/2251346757 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4a580e7c90 con 0x7f4a6010a640 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a5effd640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 0x7f4a480b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b8a47ea0e0 tx=0x7f4a580b1040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 -- start start 2024-09-21T03:22:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.899+0000 7f4a6be5c740 1 Processor -- start 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a4800290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4444070 0x55b8a48052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8a45b2700 con 0x55b8a4444070 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b8a46ec160 con 0x55b8a4423390 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8a47f9c90 con 0x55b8a441b1b0 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37278/0 (socket says 172.21.15.83:37278) 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/1952057986 learned_addr learned my addr 172.21.15.83:0/1952057986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:30.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a6be5c740 10 client.0 _ll_register_callbacks cb 0x55b8a47059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/1952057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 msgr2=0x55b8a4800290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a4800290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/1952057986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4444070 msgr2=0x55b8a48052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4444070 0x55b8a48052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.900+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/1952057986 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a4719c90 con 0x55b8a441b1b0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400bf50 tx=0x7f4a54011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8364cf389268fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a3effd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5401f070 con 0x55b8a441b1b0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a54015070 con 0x55b8a441b1b0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4a54011440 con 0x55b8a441b1b0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4a54011640 con 0x55b8a441b1b0 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a6be5c740 10 client.25942 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:30.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.901+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b8a480e230 con 0x55b8a441b1b0 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a54015070 con 0x55b8a441b1b0 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f4a540284f0 con 0x55b8a441b1b0 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 10 client.25942 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 10 client.25942 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 20 client.25942 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a44029f0 0x55b8a4811ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 4 client.25942 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8a4763df0 con 0x55b8a44029f0 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.902+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a44029f0 0x55b8a4811ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.903+0000 7f4a3effd640 10 client.25942 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:22:30.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.903+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a44029f0 0x55b8a4811ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=423d69fb12fd5e19 server_cookie=33f50071e40d2f8c in_seq=0 out_seq=0 2024-09-21T03:22:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.904+0000 7f4a3effd640 1 -- 172.21.15.83:0/1952057986 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8a4763df0 con 0x55b8a44029f0 2024-09-21T03:22:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.904+0000 7f4a3effd640 10 client.25942 handle_command_reply: tid=0 2024-09-21T03:22:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.904+0000 7f4a6be5c740 1 client.25942 shutdown 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 msgr2=0x55b8a47ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400bf50 tx=0x7f4a54011c30 comp rx=0 tx=0).stop 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 shutdown_connections 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a44029f0 0x55b8a4811ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4444070 0x55b8a48052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a4800290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/1952057986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a441b1b0 0x55b8a47ffcb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a3effd640 0 client.25942 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 >> 172.21.15.83:0/1952057986 conn(0x55b8a4438460 msgr2=0x55b8a47ee5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 shutdown_connections 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 1 -- 172.21.15.83:0/1952057986 wait complete. 2024-09-21T03:22:30.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.906+0000 7f4a6be5c740 20 client.25942 trim_cache size 0 max 16384 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.923+0000 7f4a6be5c740 1 Processor -- start 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.923+0000 7f4a6be5c740 1 -- start start 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a48055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b5a90 con 0x55b8a441b1b0 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b7690 con 0x55b8a4423390 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a6be5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b6160 con 0x55b8a4438460 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54718/0 (socket says 172.21.15.83:54718) 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 -- 172.21.15.83:0/304750541 learned_addr learned my addr 172.21.15.83:0/304750541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37294/0 (socket says 172.21.15.83:37294) 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a48055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 -- 172.21.15.83:0/304750541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 msgr2=0x55b8a48016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 -- 172.21.15.83:0/304750541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 msgr2=0x55b8a48055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a48055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 -- 172.21.15.83:0/304750541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a47ef570 con 0x55b8a441b1b0 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.924+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ece8756997a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.925+0000 7f4a457fa640 1 -- 172.21.15.83:0/304750541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50049070 con 0x55b8a441b1b0 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.925+0000 7f4a457fa640 1 -- 172.21.15.83:0/304750541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50005030 con 0x55b8a441b1b0 2024-09-21T03:22:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.925+0000 7f4a457fa640 1 -- 172.21.15.83:0/304750541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50035040 con 0x55b8a441b1b0 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- 172.21.15.83:0/304750541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 msgr2=0x55b8a4805090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a50000bc0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- 172.21.15.83:0/304750541 shutdown_connections 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a4423390 0x55b8a48055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a441b1b0 0x55b8a4805090 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/304750541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a48016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- 172.21.15.83:0/304750541 >> 172.21.15.83:0/304750541 conn(0x55b8a47e4540 msgr2=0x55b8a4800fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- 172.21.15.83:0/304750541 shutdown_connections 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- 172.21.15.83:0/304750541 wait complete. 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.926+0000 7f4a6be5c740 1 -- start start 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 Processor -- start 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 0x55b8a4806fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 0x55b8a4807570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 0x55b8a4807570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 0x55b8a4807570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54734/0 (socket says 172.21.15.83:54734) 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a5e7fc640 1 -- 172.21.15.83:0/403262538 learned_addr learned my addr 172.21.15.83:0/403262538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 0x55b8a4806fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a487a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b6fc0 con 0x55b8a4423390 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b6830 con 0x55b8a441b1b0 2024-09-21T03:22:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b3a70 con 0x55b8a4438460 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.927+0000 7f4a6be5c740 10 client.0 _ll_register_callbacks cb 0x7f4a54004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 msgr2=0x55b8a487a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a487a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/403262538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 msgr2=0x55b8a4807570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 0x55b8a4807570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a487ab40 con 0x55b8a441b1b0 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a5f7fe640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 0x55b8a4806fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400bf20 tx=0x7f4a5400b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ca62247eb960fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a3effd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.928+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5401f040 con 0x55b8a441b1b0 2024-09-21T03:22:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f4a540282c0 con 0x55b8a441b1b0 2024-09-21T03:22:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50005030 con 0x55b8a441b1b0 2024-09-21T03:22:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4a540284c0 con 0x55b8a441b1b0 2024-09-21T03:22:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a6be5c740 10 client.25952 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b8a487bd60 con 0x55b8a441b1b0 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.929+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a54015040 con 0x55b8a441b1b0 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f4a54041480 con 0x55b8a441b1b0 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 10 client.25952 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 10 client.25952 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 20 client.25952 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a4444070 0x55b8a487e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 4 client.25952 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4a50005030 con 0x55b8a4444070 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.930+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a4444070 0x55b8a487e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.931+0000 7f4a3effd640 10 client.25952 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.931+0000 7f4a5e7fc640 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a4444070 0x55b8a487e410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a7d18980bc13d89 server_cookie=71f44170ee1fe5 in_seq=0 out_seq=0 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.931+0000 7f4a3effd640 1 -- 172.21.15.83:0/403262538 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4a50005030 con 0x55b8a4444070 2024-09-21T03:22:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.931+0000 7f4a3effd640 10 client.25952 handle_command_reply: tid=0 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.931+0000 7f4a6be5c740 1 client.25952 shutdown 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 msgr2=0x55b8a4806fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 0x55b8a4806fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400bf20 tx=0x7f4a5400b3e0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 shutdown_connections 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55b8a4444070 0x55b8a487e410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8a4438460 0x55b8a487a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8a4423390 0x55b8a4807570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 --2- 172.21.15.83:0/403262538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a441b1b0 0x55b8a4806fb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 >> 172.21.15.83:0/403262538 conn(0x55b8a47e4540 msgr2=0x55b8a47eecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a3effd640 0 client.25952 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 shutdown_connections 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 1 -- 172.21.15.83:0/403262538 wait complete. 2024-09-21T03:22:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.932+0000 7f4a6be5c740 20 client.25952 trim_cache size 0 max 16384 2024-09-21T03:22:30.937 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:22:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 msgr2=0x7f4a480b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 0x7f4a480b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b8a47ea0e0 tx=0x7f4a580b1040 comp rx=0 tx=0).stop 2024-09-21T03:22:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 msgr2=0x7f4a601a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800ebc0 tx=0x7f4a5800c6a0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 shutdown_connections 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4a480b6700 0x7f4a480b8c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60116510 0x7f4a601aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6010af90 0x7f4a601a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 --2- 172.21.15.83:0/2251346757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a6010a640 0x7f4a601a4eb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 >> 172.21.15.83:0/2251346757 conn(0x7f4a6006e9f0 msgr2=0x7f4a60072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 shutdown_connections 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:30.933+0000 7f4a5ffff640 1 -- 172.21.15.83:0/2251346757 wait complete. 2024-09-21T03:22:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 310.0 s, 2550.4 tps, lat 39.433 ms stddev 45.754 2024-09-21T03:22:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:31 smithi142 ceph-mon[34512]: pgmap v501: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:22:31.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:31 smithi193 ceph-mon[35224]: pgmap v501: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:22:31.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:31 smithi083 ceph-mon[31219]: pgmap v501: 97 pgs: 97 active+clean; 3.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:22:31.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:22:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 -- 172.21.15.83:0/3414394324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1001083c0 msgr2=0x7fc1001087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 --2- 172.21.15.83:0/3414394324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1001083c0 0x7fc1001087d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009900 tx=0x7fc0f402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 -- 172.21.15.83:0/3414394324 shutdown_connections 2024-09-21T03:22:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 --2- 172.21.15.83:0/3414394324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100108d10 0x7fc10010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 --2- 172.21.15.83:0/3414394324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1001083c0 0x7fc1001087d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 --2- 172.21.15.83:0/3414394324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc100102a80 0x7fc100102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 -- 172.21.15.83:0/3414394324 >> 172.21.15.83:0/3414394324 conn(0x7fc100079740 msgr2=0x7fc100079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 -- 172.21.15.83:0/3414394324 shutdown_connections 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.769+0000 7fc1068f2640 1 -- 172.21.15.83:0/3414394324 wait complete. 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.771+0000 7fc1068f2640 1 Processor -- start 2024-09-21T03:22:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.771+0000 7fc1068f2640 1 -- start start 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 0x7fc100199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc100108d10 0x7fc10019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc100112a10 con 0x7fc100108d10 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc100112890 con 0x7fc100102a80 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc1068f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc100112b90 con 0x7fc1001083c0 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59878/0 (socket says 172.21.15.83:59878) 2024-09-21T03:22:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc0fffff640 1 -- 172.21.15.83:0/1389538412 learned_addr learned my addr 172.21.15.83:0/1389538412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.772+0000 7fc104e8f640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc100108d10 0x7fc10019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 0x7fc100199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0fffff640 1 -- 172.21.15.83:0/1389538412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 msgr2=0x7fc100199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0fffff640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 0x7fc100199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0fffff640 1 -- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc100108d10 msgr2=0x7fc10019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0fffff640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc100108d10 0x7fc10019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0fffff640 1 -- 172.21.15.83:0/1389538412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10019f350 con 0x7fc100102a80 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.773+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 0x7fc100199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc0fffff640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b4a0 tx=0x7fc0f000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc0fd7fa640 1 -- 172.21.15.83:0/1389538412 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0014070 con 0x7fc100102a80 2024-09-21T03:22:32.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc0fd7fa640 1 -- 172.21.15.83:0/1389538412 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f0010ac0 con 0x7fc100102a80 2024-09-21T03:22:32.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc0fd7fa640 1 -- 172.21.15.83:0/1389538412 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0017c30 con 0x7fc100102a80 2024-09-21T03:22:32.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10019f640 con 0x7fc100102a80 2024-09-21T03:22:32.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.774+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10006b820 con 0x7fc100102a80 2024-09-21T03:22:32.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.777+0000 7fc10ce66740 1 Processor -- start 2024-09-21T03:22:32.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.777+0000 7fc10ce66740 1 -- start start 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.777+0000 7fc0fd7fa640 1 -- 172.21.15.83:0/1389538412 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc0f0017070 con 0x7fc100102a80 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cc5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5c32ea0 con 0x55d6e58ec1b0 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5c222d0 con 0x55d6e5915070 2024-09-21T03:22:32.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc10ce66740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5c260e0 con 0x55d6e58f4390 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59890/0 (socket says 172.21.15.83:59890) 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 -- 172.21.15.83:0/2271018844 learned_addr learned my addr 172.21.15.83:0/2271018844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cc5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc0fffff640 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 -- 172.21.15.83:0/2271018844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 msgr2=0x55d6e5cc5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cc5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 -- 172.21.15.83:0/2271018844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 msgr2=0x55d6e5cc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.778+0000 7fc104e8f640 1 -- 172.21.15.83:0/2271018844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e5980650 con 0x55d6e5915070 2024-09-21T03:22:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0fd7fa640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 0x7fc0e80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0fd7fa640 1 -- 172.21.15.83:0/1389538412 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0f00e68d0 con 0x7fc100102a80 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0fffff640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 0x7fc0e80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc104e8f640 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8010370 tx=0x7fc0f8010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae834d3fa27e930d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0e67fc640 1 -- 172.21.15.83:0/2271018844 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8016070 con 0x55d6e5915070 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0e67fc640 1 -- 172.21.15.83:0/2271018844 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8004d20 con 0x55d6e5915070 2024-09-21T03:22:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.779+0000 7fc0e67fc640 1 -- 172.21.15.83:0/2271018844 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8005350 con 0x55d6e5915070 2024-09-21T03:22:32.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.781+0000 7fc10ce66740 1 -- 172.21.15.83:0/2271018844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 msgr2=0x55d6e5cc4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.781+0000 7fc10ce66740 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8010370 tx=0x7fc0f8010830 comp rx=0 tx=0).stop 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.781+0000 7fc0fffff640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 0x7fc0e80b8ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55d6e5cc3750 tx=0x7fc0f0010f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 -- 172.21.15.83:0/2271018844 shutdown_connections 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cc5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 --2- 172.21.15.83:0/2271018844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cc4ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 -- 172.21.15.83:0/2271018844 >> 172.21.15.83:0/2271018844 conn(0x55d6e5909460 msgr2=0x55d6e5cc29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 -- 172.21.15.83:0/2271018844 shutdown_connections 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.782+0000 7fc10ce66740 1 -- 172.21.15.83:0/2271018844 wait complete. 2024-09-21T03:22:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.783+0000 7fc10ce66740 1 -- start start 2024-09-21T03:22:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.783+0000 7fc10ce66740 1 Processor -- start 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.783+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cd06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5a83700 con 0x55d6e58ec1b0 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5bbd160 con 0x55d6e5915070 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5cd32b0 con 0x55d6e58f4390 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc10ce66740 10 client.0 _ll_register_callbacks cb 0x55d6e5bd69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54780/0 (socket says 172.21.15.83:54780) 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 -- 172.21.15.83:0/586738298 learned_addr learned my addr 172.21.15.83:0/586738298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cd06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 -- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 msgr2=0x55d6e5cd06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cd06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 -- 172.21.15.83:0/586738298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 msgr2=0x55d6e5cdc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cdc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 -- 172.21.15.83:0/586738298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e5beac90 con 0x55d6e58ec1b0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc104e8f640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8012ee0 tx=0x7fc0f8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2ae90f6dac16a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.784+0000 7fc0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8016070 con 0x55d6e58ec1b0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0f8012a20 con 0x55d6e58ec1b0 2024-09-21T03:22:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f8004d20 con 0x55d6e58ec1b0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0f8012c20 con 0x55d6e58ec1b0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc10ce66740 10 client.16425 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d6e5cdef80 con 0x55d6e58ec1b0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8003e20 con 0x55d6e58ec1b0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.785+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fc0f800f040 con 0x55d6e58ec1b0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 10 client.16425 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 10 client.16425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 20 client.16425 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58d39f0 0x55d6e5ce2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 4 client.16425 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc0f8004d20 con 0x55d6e58d39f0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58d39f0 0x55d6e5ce2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc0df7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.786+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58d39f0 0x55d6e5ce2390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fac6a752250f68b8 server_cookie=e573e423a141da42 in_seq=0 out_seq=0 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.787+0000 7fc0df7fe640 1 -- 172.21.15.83:0/586738298 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc0f8004d20 con 0x55d6e58d39f0 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.787+0000 7fc0df7fe640 10 client.16425 handle_command_reply: tid=0 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.787+0000 7fc10ce66740 1 client.16425 shutdown 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 msgr2=0x55d6e5cd01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8012ee0 tx=0x7fc0f8004ed0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 shutdown_connections 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e5915070 0x55d6e5cdc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58d39f0 0x55d6e5ce2390 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5cd06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 --2- 172.21.15.83:0/586738298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd01b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc0df7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 >> 172.21.15.83:0/586738298 conn(0x55d6e5909460 msgr2=0x55d6e5cc7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 shutdown_connections 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 1 -- 172.21.15.83:0/586738298 wait complete. 2024-09-21T03:22:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.788+0000 7fc10ce66740 20 client.16425 trim_cache size 0 max 16384 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.806+0000 7fc10ce66740 1 Processor -- start 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 -- start start 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5cb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ad480 con 0x55d6e5915070 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001af080 con 0x55d6e58f4390 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc10ce66740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001adb50 con 0x55d6e58ec1b0 2024-09-21T03:22:32.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59906/0 (socket says 172.21.15.83:59906) 2024-09-21T03:22:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5cb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/1238262390 learned_addr learned my addr 172.21.15.83:0/1238262390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc104e8f640 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/1238262390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 msgr2=0x55d6e5cd2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.807+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/1238262390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 msgr2=0x55d6e5cb8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5cb8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/1238262390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e5cb55d0 con 0x55d6e58f4390 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc104e8f640 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f402a9b0 tx=0x7fc0f4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ac8f4cef362e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.808+0000 7fc0e5ffb640 1 -- 172.21.15.83:0/1238262390 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4035040 con 0x55d6e58f4390 2024-09-21T03:22:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.809+0000 7fc0e5ffb640 1 -- 172.21.15.83:0/1238262390 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f40054f0 con 0x55d6e58f4390 2024-09-21T03:22:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.809+0000 7fc0e5ffb640 1 -- 172.21.15.83:0/1238262390 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40035f0 con 0x55d6e58f4390 2024-09-21T03:22:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.810+0000 7fc10ce66740 1 -- 172.21.15.83:0/1238262390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 msgr2=0x55d6e5cdc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.810+0000 7fc10ce66740 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f402a9b0 tx=0x7fc0f4009560 comp rx=0 tx=0).stop 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 -- 172.21.15.83:0/1238262390 shutdown_connections 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5cb8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58f4390 0x55d6e5cdc510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 --2- 172.21.15.83:0/1238262390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58ec1b0 0x55d6e5cd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 -- 172.21.15.83:0/1238262390 >> 172.21.15.83:0/1238262390 conn(0x55d6e5909460 msgr2=0x55d6e5cb8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 -- 172.21.15.83:0/1238262390 shutdown_connections 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 -- 172.21.15.83:0/1238262390 wait complete. 2024-09-21T03:22:32.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.811+0000 7fc10ce66740 1 -- start start 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 Processor -- start 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 0x55d6e5ce5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5ce30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5ce3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ae9b0 con 0x55d6e5915070 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ae220 con 0x55d6e58ec1b0 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ab460 con 0x55d6e58f4390 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc10ce66740 10 client.0 _ll_register_callbacks cb 0x7fc0f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 0x55d6e5ce5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc104e8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 0x55d6e5ce5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59918/0 (socket says 172.21.15.83:59918) 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc104e8f640 1 -- 172.21.15.83:0/4013012646 learned_addr learned my addr 172.21.15.83:0/4013012646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5ce30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc104e8f640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5ce3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/4013012646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 msgr2=0x55d6e5ce5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 0x55d6e5ce5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/4013012646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 msgr2=0x55d6e5ce3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5ce3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.812+0000 7fc0ff7fe640 1 -- 172.21.15.83:0/4013012646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e5ce3e10 con 0x55d6e58f4390 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0ff7fe640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5ce30e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031440 tx=0x7fc0f4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6537f4bae4a1c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:22:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4050070 con 0x55d6e58f4390 2024-09-21T03:22:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f40054f0 con 0x55d6e58f4390 2024-09-21T03:22:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fc0f40048d0 con 0x55d6e58f4390 2024-09-21T03:22:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0f405c050 con 0x55d6e58f4390 2024-09-21T03:22:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc10ce66740 10 client.25957 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d6e5d4caf0 con 0x55d6e58f4390 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.813+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f40054f0 con 0x55d6e58f4390 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fc0f4003a00 con 0x55d6e58f4390 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 10 client.25957 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 10 client.25957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 20 client.25957 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58a8da0 0x55d6e5d4f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 4 client.25957 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6e5c34df0 con 0x55d6e58a8da0 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.814+0000 7fc104e8f640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58a8da0 0x55d6e5d4f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc0df7fe640 10 client.25957 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:22:32.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc104e8f640 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58a8da0 0x55d6e5d4f290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d20302da3f73fe4 server_cookie=c3371f63a30255fe in_seq=0 out_seq=0 2024-09-21T03:22:32.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc0df7fe640 1 -- 172.21.15.83:0/4013012646 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6e5c34df0 con 0x55d6e58a8da0 2024-09-21T03:22:32.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc0df7fe640 10 client.25957 handle_command_reply: tid=0 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 client.25957 shutdown 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 msgr2=0x55d6e5ce30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5ce30e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031440 tx=0x7fc0f4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 shutdown_connections 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6e5915070 0x55d6e5ce3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d6e58a8da0 0x55d6e5d4f290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6e58f4390 0x55d6e5ce30e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 --2- 172.21.15.83:0/4013012646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d6e58ec1b0 0x55d6e5ce5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.815+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 >> 172.21.15.83:0/4013012646 conn(0x55d6e5909460 msgr2=0x55d6e5cb8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc0df7fe640 0 client.25957 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:22:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 shutdown_connections 2024-09-21T03:22:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc10ce66740 1 -- 172.21.15.83:0/4013012646 wait complete. 2024-09-21T03:22:32.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc10ce66740 20 client.25957 trim_cache size 0 max 16384 2024-09-21T03:22:32.824 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:22:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 msgr2=0x7fc0e80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 0x7fc0e80b8ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55d6e5cc3750 tx=0x7fc0f0010f90 comp rx=0 tx=0).stop 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 msgr2=0x7fc1001993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b4a0 tx=0x7fc0f000b960 comp rx=0 tx=0).stop 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 shutdown_connections 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0e80b67d0 0x7fc0e80b8ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc100108d10 0x7fc10019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1001083c0 0x7fc100199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 --2- 172.21.15.83:0/1389538412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100102a80 0x7fc1001993e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.816+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 >> 172.21.15.83:0/1389538412 conn(0x7fc100079740 msgr2=0x7fc1001052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.817+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 shutdown_connections 2024-09-21T03:22:32.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:32.817+0000 7fc1068f2640 1 -- 172.21.15.83:0/1389538412 wait complete. 2024-09-21T03:22:33.619 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:33 smithi083 ceph-mon[31219]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 780 op/s 2024-09-21T03:22:33.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:22:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:33 smithi142 ceph-mon[34512]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 780 op/s 2024-09-21T03:22:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:33 smithi193 ceph-mon[35224]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 780 op/s 2024-09-21T03:22:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 -- 172.21.15.83:0/119059492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428109ec0 msgr2=0x7fa4281106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 --2- 172.21.15.83:0/119059492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428109ec0 0x7fa4281106f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa420009ca0 tx=0x7fa42002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 -- 172.21.15.83:0/119059492 shutdown_connections 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 --2- 172.21.15.83:0/119059492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428109ec0 0x7fa4281106f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 --2- 172.21.15.83:0/119059492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109570 0x7fa428109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 --2- 172.21.15.83:0/119059492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428103c30 0x7fa428104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 -- 172.21.15.83:0/119059492 >> 172.21.15.83:0/119059492 conn(0x7fa42807b8d0 msgr2=0x7fa42807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 -- 172.21.15.83:0/119059492 shutdown_connections 2024-09-21T03:22:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.994+0000 7fa42fea5640 1 -- 172.21.15.83:0/119059492 wait complete. 2024-09-21T03:22:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 Processor -- start 2024-09-21T03:22:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 -- start start 2024-09-21T03:22:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428109570 0x7fa42819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109ec0 0x7fa42819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa428113b30 con 0x7fa428103c30 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281139b0 con 0x7fa428109570 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42fea5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa428113cb0 con 0x7fa428109ec0 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42dc41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42dc41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54816/0 (socket says 172.21.15.83:54816) 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.995+0000 7fa42dc41640 1 -- 172.21.15.83:0/4273852805 learned_addr learned my addr 172.21.15.83:0/4273852805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42d440640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428109570 0x7fa42819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42e442640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109ec0 0x7fa42819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 -- 172.21.15.83:0/4273852805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109ec0 msgr2=0x7fa42819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109ec0 0x7fa42819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 -- 172.21.15.83:0/4273852805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428109570 msgr2=0x7fa42819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428109570 0x7fa42819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 -- 172.21.15.83:0/4273852805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281a03d0 con 0x7fa428103c30 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42dc41640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa42400ea60 tx=0x7fa42400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa41effd640 1 -- 172.21.15.83:0/4273852805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42400cde0 con 0x7fa428103c30 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281a06c0 con 0x7fa428103c30 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa41effd640 1 -- 172.21.15.83:0/4273852805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4240035f0 con 0x7fa428103c30 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa41effd640 1 -- 172.21.15.83:0/4273852805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424010690 con 0x7fa428103c30 2024-09-21T03:22:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:34.996+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a6e20 con 0x7fa428103c30 2024-09-21T03:22:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.000+0000 7fa41effd640 1 -- 172.21.15.83:0/4273852805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa4240037a0 con 0x7fa428103c30 2024-09-21T03:22:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa41effd640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 0x7fa4140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 Processor -- start 2024-09-21T03:22:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa42d440640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 0x7fa4140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa41effd640 1 -- 172.21.15.83:0/4273852805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa4240e7ad0 con 0x7fa428103c30 2024-09-21T03:22:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 -- start start 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a90070 0x55e345e3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e3bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 0x55e345e3c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e345dadea0 con 0x55e345a6f390 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa42d440640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 0x7fa4140b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa428109090 tx=0x7fa418005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e345d9d2d0 con 0x55e345a671b0 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.001+0000 7fa436419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e345da10e0 con 0x55e345a90070 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42e442640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 0x55e345e3c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42dc41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a90070 0x55e345e3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42e442640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 0x55e345e3c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54828/0 (socket says 172.21.15.83:54828) 2024-09-21T03:22:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e3bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42e442640 1 -- 172.21.15.83:0/3122690253 learned_addr learned my addr 172.21.15.83:0/3122690253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 -- 172.21.15.83:0/3122690253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a90070 msgr2=0x55e345e3bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a90070 0x55e345e3bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 -- 172.21.15.83:0/3122690253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 msgr2=0x55e345e3c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 0x55e345e3c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.002+0000 7fa42d440640 1 -- 172.21.15.83:0/3122690253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e345afb650 con 0x55e345a671b0 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.003+0000 7fa42d440640 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e3bff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa42806a800 tx=0x7fa418014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292996252ddb0337 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.003+0000 7fa413fff640 1 -- 172.21.15.83:0/3122690253 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418004020 con 0x55e345a671b0 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.003+0000 7fa413fff640 1 -- 172.21.15.83:0/3122690253 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4180041c0 con 0x55e345a671b0 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.004+0000 7fa413fff640 1 -- 172.21.15.83:0/3122690253 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41801f680 con 0x55e345a671b0 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.004+0000 7fa436419740 1 -- 172.21.15.83:0/3122690253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 msgr2=0x55e345e3bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.004+0000 7fa436419740 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e3bff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa42806a800 tx=0x7fa418014000 comp rx=0 tx=0).stop 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 -- 172.21.15.83:0/3122690253 shutdown_connections 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a6f390 0x55e345e3c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e3bff0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 --2- 172.21.15.83:0/3122690253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a90070 0x55e345e3bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 -- 172.21.15.83:0/3122690253 >> 172.21.15.83:0/3122690253 conn(0x55e345a84460 msgr2=0x55e345e399a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 -- 172.21.15.83:0/3122690253 shutdown_connections 2024-09-21T03:22:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 -- 172.21.15.83:0/3122690253 wait complete. 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.005+0000 7fa436419740 1 -- start start 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 Processor -- start 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e47b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e345bfe700 con 0x55e345a90070 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e345d38160 con 0x55e345a6f390 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.006+0000 7fa436419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e345e4a330 con 0x55e345a671b0 2024-09-21T03:22:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42e442640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e47b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa436419740 10 client.0 _ll_register_callbacks cb 0x55e345d519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42e442640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e47b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54836/0 (socket says 172.21.15.83:54836) 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42e442640 1 -- 172.21.15.83:0/2624534645 learned_addr learned my addr 172.21.15.83:0/2624534645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42dc41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 -- 172.21.15.83:0/2624534645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 msgr2=0x55e345e47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 -- 172.21.15.83:0/2624534645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 msgr2=0x55e345e47b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e47b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.007+0000 7fa42d440640 1 -- 172.21.15.83:0/2624534645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e345d65c90 con 0x55e345a6f390 2024-09-21T03:22:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.008+0000 7fa4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.008+0000 7fa42d440640 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e475d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa418003db0 tx=0x7fa418011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840f6d6211725fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.008+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418012e30 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.008+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa418003f10 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.008+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4180041c0 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.009+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa418005030 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.009+0000 7fa436419740 10 client.25982 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.009+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e345e56010 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.009+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4180041c0 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fa41801f420 con 0x55e345a6f390 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 10 client.25982 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 10 client.25982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 20 client.25982 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a4e9f0 0x55e345e59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 4 client.25982 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e345dafdf0 con 0x55e345a4e9f0 2024-09-21T03:22:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.010+0000 7fa42dc41640 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a4e9f0 0x55e345e59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.011+0000 7fa4117fa640 10 client.25982 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:22:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.011+0000 7fa42dc41640 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a4e9f0 0x55e345e59480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45fb26b1bf675230 server_cookie=961f8e8e8b148639 in_seq=0 out_seq=0 2024-09-21T03:22:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.012+0000 7fa4117fa640 1 -- 172.21.15.83:0/2624534645 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e345dafdf0 con 0x55e345a4e9f0 2024-09-21T03:22:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.012+0000 7fa4117fa640 10 client.25982 handle_command_reply: tid=0 2024-09-21T03:22:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.013+0000 7fa436419740 1 client.25982 shutdown 2024-09-21T03:22:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.013+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 msgr2=0x55e345e475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.013+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e475d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa418003db0 tx=0x7fa418011b50 comp rx=0 tx=0).stop 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 shutdown_connections 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a4e9f0 0x55e345e59480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e475d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 --2- 172.21.15.83:0/2624534645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 >> 172.21.15.83:0/2624534645 conn(0x55e345a84460 msgr2=0x55e345e3ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa4117fa640 0 client.25982 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 shutdown_connections 2024-09-21T03:22:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 1 -- 172.21.15.83:0/2624534645 wait complete. 2024-09-21T03:22:35.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.014+0000 7fa436419740 20 client.25982 trim_cache size 0 max 16384 2024-09-21T03:22:35.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.034+0000 7fa436419740 1 Processor -- start 2024-09-21T03:22:35.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.034+0000 7fa436419740 1 -- start start 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e44280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281ad4a0 con 0x55e345a90070 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281af0a0 con 0x55e345a6f390 2024-09-21T03:22:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.035+0000 7fa436419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281adb70 con 0x55e345a671b0 2024-09-21T03:22:35.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.038+0000 7fa42d440640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e44280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.038+0000 7fa42d440640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e44280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59960/0 (socket says 172.21.15.83:59960) 2024-09-21T03:22:35.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.038+0000 7fa42d440640 1 -- 172.21.15.83:0/1102991029 learned_addr learned my addr 172.21.15.83:0/1102991029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:35.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.040+0000 7fa42dc41640 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.043+0000 7fa42dc41640 1 -- 172.21.15.83:0/1102991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 msgr2=0x55e345e44280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.043+0000 7fa42dc41640 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e44280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.043+0000 7fa42dc41640 1 -- 172.21.15.83:0/1102991029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 msgr2=0x55e345e447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.043+0000 7fa42dc41640 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.043+0000 7fa42dc41640 1 -- 172.21.15.83:0/1102991029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e345a4ec20 con 0x55e345a671b0 2024-09-21T03:22:35.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.047+0000 7fa42dc41640 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e49080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4240e52a0 tx=0x7fa424009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b2c713178df407 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.051+0000 7fa4137fe640 1 -- 172.21.15.83:0/1102991029 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240a9110 con 0x55e345a671b0 2024-09-21T03:22:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.051+0000 7fa4137fe640 1 -- 172.21.15.83:0/1102991029 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4240ad4b0 con 0x55e345a671b0 2024-09-21T03:22:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.052+0000 7fa436419740 1 -- 172.21.15.83:0/1102991029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 msgr2=0x55e345e49080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.052+0000 7fa436419740 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e49080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4240e52a0 tx=0x7fa424009490 comp rx=0 tx=0).stop 2024-09-21T03:22:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.060+0000 7fa436419740 1 -- 172.21.15.83:0/1102991029 shutdown_connections 2024-09-21T03:22:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.060+0000 7fa436419740 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.060+0000 7fa436419740 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a6f390 0x55e345e44280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.060+0000 7fa436419740 1 --2- 172.21.15.83:0/1102991029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a671b0 0x55e345e49080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.060+0000 7fa436419740 1 -- 172.21.15.83:0/1102991029 >> 172.21.15.83:0/1102991029 conn(0x55e345a84460 msgr2=0x55e345e3d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.064+0000 7fa436419740 1 -- 172.21.15.83:0/1102991029 shutdown_connections 2024-09-21T03:22:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.067+0000 7fa436419740 1 -- 172.21.15.83:0/1102991029 wait complete. 2024-09-21T03:22:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.068+0000 7fa436419740 1 -- start start 2024-09-21T03:22:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.068+0000 7fa436419740 1 Processor -- start 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281ae9d0 con 0x55e345a90070 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281ae240 con 0x55e345a671b0 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.076+0000 7fa436419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281ab4a0 con 0x55e345a6f390 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.077+0000 7fa436419740 10 client.0 _ll_register_callbacks cb 0x7fa424005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.077+0000 7fa42d440640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.077+0000 7fa42d440640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37406/0 (socket says 172.21.15.83:37406) 2024-09-21T03:22:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.077+0000 7fa42d440640 1 -- 172.21.15.83:0/2244825519 learned_addr learned my addr 172.21.15.83:0/2244825519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.081+0000 7fa42d440640 1 -- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 msgr2=0x55e345e5cd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:22:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.081+0000 7fa42d440640 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e5cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.081+0000 7fa42d440640 1 -- 172.21.15.83:0/2244825519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 msgr2=0x55e345e38580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.081+0000 7fa42d440640 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.081+0000 7fa42d440640 1 -- 172.21.15.83:0/2244825519 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e345e38d10 con 0x55e345a6f390 2024-09-21T03:22:35.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.083+0000 7fa42d440640 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa418009600 tx=0x7fa41801fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813b3ed72c4a7f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:35.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.083+0000 7fa4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:22:35.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.083+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418003900 con 0x55e345a6f390 2024-09-21T03:22:35.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.084+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e345dab8a0 con 0x55e345a6f390 2024-09-21T03:22:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.084+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa41801f630 con 0x55e345a6f390 2024-09-21T03:22:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.084+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa41804a050 con 0x55e345a6f390 2024-09-21T03:22:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.086+0000 7fa436419740 10 client.25975 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.086+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e345ed9450 con 0x55e345a6f390 2024-09-21T03:22:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.086+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e345dab8a0 con 0x55e345a6f390 2024-09-21T03:22:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fa418043060 con 0x55e345a6f390 2024-09-21T03:22:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 10 client.25975 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 10 client.25975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:22:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 20 client.25975 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a23da0 0x55e345edbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 4 client.25975 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:22:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.087+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e345da8cd0 con 0x55e345a23da0 2024-09-21T03:22:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.090+0000 7fa42dc41640 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a23da0 0x55e345edbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.091+0000 7fa42dc41640 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a23da0 0x55e345edbb60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5c24818d51017cc server_cookie=9057088242859cd3 in_seq=0 out_seq=0 2024-09-21T03:22:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.091+0000 7fa4117fa640 10 client.25975 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:22:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.091+0000 7fa4117fa640 1 -- 172.21.15.83:0/2244825519 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e345da8cd0 con 0x55e345a23da0 2024-09-21T03:22:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.091+0000 7fa4117fa640 10 client.25975 handle_command_reply: tid=0 2024-09-21T03:22:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.091+0000 7fa436419740 1 client.25975 shutdown 2024-09-21T03:22:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 msgr2=0x55e345e37fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa418009600 tx=0x7fa41801fa10 comp rx=0 tx=0).stop 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 shutdown_connections 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55e345a23da0 0x55e345edbb60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e345a90070 0x55e345e38580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e345a6f390 0x55e345e37fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 --2- 172.21.15.83:0/2244825519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e345a671b0 0x55e345e5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 >> 172.21.15.83:0/2244825519 conn(0x55e345a84460 msgr2=0x55e345e3d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.092+0000 7fa4117fa640 0 client.25975 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:22:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.093+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 shutdown_connections 2024-09-21T03:22:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.093+0000 7fa436419740 1 -- 172.21.15.83:0/2244825519 wait complete. 2024-09-21T03:22:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.093+0000 7fa436419740 20 client.25975 trim_cache size 0 max 16384 2024-09-21T03:22:35.095 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.094+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 msgr2=0x7fa4140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.094+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 0x7fa4140b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa428109090 tx=0x7fa418005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.094+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 msgr2=0x7fa42819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.094+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa42400ea60 tx=0x7fa42400ef20 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.094+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 shutdown_connections 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa4140b6680 0x7fa4140b8b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa428109ec0 0x7fa42819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa428109570 0x7fa42819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 --2- 172.21.15.83:0/4273852805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa428103c30 0x7fa42819a460 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 >> 172.21.15.83:0/4273852805 conn(0x7fa42807b8d0 msgr2=0x7fa428107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 shutdown_connections 2024-09-21T03:22:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:35.095+0000 7fa42fea5640 1 -- 172.21.15.83:0/4273852805 wait complete. 2024-09-21T03:22:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:35 smithi142 ceph-mon[34512]: pgmap v503: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-09-21T03:22:35.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:35 smithi193 ceph-mon[35224]: pgmap v503: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-09-21T03:22:35.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:35 smithi083 ceph-mon[31219]: pgmap v503: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-09-21T03:22:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 315.0 s, 2201.0 tps, lat 45.308 ms stddev 48.926 2024-09-21T03:22:36.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:22:36.379 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:36.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 -- 172.21.15.83:0/3560724606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 msgr2=0x7fe87810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 --2- 172.21.15.83:0/3560724606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe87810ecb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ca0 tx=0x7fe87002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 -- 172.21.15.83:0/3560724606 shutdown_connections 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 --2- 172.21.15.83:0/3560724606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe87810ecb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 --2- 172.21.15.83:0/3560724606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe878107b30 0x7fe878107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 --2- 172.21.15.83:0/3560724606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8781021f0 0x7fe878102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.919+0000 7fe87f954640 1 -- 172.21.15.83:0/3560724606 >> 172.21.15.83:0/3560724606 conn(0x7fe878077c60 msgr2=0x7fe8780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.920+0000 7fe87f954640 1 -- 172.21.15.83:0/3560724606 shutdown_connections 2024-09-21T03:22:36.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.920+0000 7fe87f954640 1 -- 172.21.15.83:0/3560724606 wait complete. 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 Processor -- start 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 -- start start 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8781021f0 0x7fe87819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 0x7fe87819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781115c0 con 0x7fe878108480 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe878111440 con 0x7fe878107b30 2024-09-21T03:22:36.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87f954640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe878111740 con 0x7fe8781021f0 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87def1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87def1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54872/0 (socket says 172.21.15.83:54872) 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87def1640 1 -- 172.21.15.83:0/421942611 learned_addr learned my addr 172.21.15.83:0/421942611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87ceef640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 0x7fe87819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.921+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8781021f0 0x7fe87819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 -- 172.21.15.83:0/421942611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8781021f0 msgr2=0x7fe87819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8781021f0 0x7fe87819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 -- 172.21.15.83:0/421942611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 msgr2=0x7fe87819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 0x7fe87819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 -- 172.21.15.83:0/421942611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781a2620 con 0x7fe878108480 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87ceef640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 0x7fe87819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87def1640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe870009430 tx=0x7fe8700317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe86e7fc640 1 -- 172.21.15.83:0/421942611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870003b70 con 0x7fe878108480 2024-09-21T03:22:36.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781a28b0 con 0x7fe878108480 2024-09-21T03:22:36.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a9070 con 0x7fe878108480 2024-09-21T03:22:36.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe86e7fc640 1 -- 172.21.15.83:0/421942611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe870003d10 con 0x7fe878108480 2024-09-21T03:22:36.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.922+0000 7fe86e7fc640 1 -- 172.21.15.83:0/421942611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87002f870 con 0x7fe878108480 2024-09-21T03:22:36.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.926+0000 7fe885ec8740 1 Processor -- start 2024-09-21T03:22:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.926+0000 7fe86e7fc640 1 -- 172.21.15.83:0/421942611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe87002fa10 con 0x7fe878108480 2024-09-21T03:22:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.926+0000 7fe885ec8740 1 -- start start 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19da7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f19d9f2ea0 con 0x55f19d6ac1b0 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f19d9e22d0 con 0x55f19d6b4390 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe885ec8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f19d9e60e0 con 0x55f19d6d5070 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54888/0 (socket says 172.21.15.83:54888) 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 -- 172.21.15.83:0/1799686408 learned_addr learned my addr 172.21.15.83:0/1799686408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87ceef640 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19da7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87ceef640 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 -- 172.21.15.83:0/1799686408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 msgr2=0x55f19da7c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da7c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 -- 172.21.15.83:0/1799686408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 msgr2=0x55f19da7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19da7d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe87d6f0640 1 -- 172.21.15.83:0/1799686408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19d740650 con 0x55f19d6ac1b0 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.927+0000 7fe86e7fc640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 0x7fe8640b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe87ceef640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 0x7fe8640b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe874009cf0 tx=0x7fe874011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d655941dbfb0012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe86e7fc640 1 -- 172.21.15.83:0/421942611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe870108750 con 0x7fe878108480 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe8637fe640 1 -- 172.21.15.83:0/1799686408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87401e070 con 0x55f19d6ac1b0 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe8637fe640 1 -- 172.21.15.83:0/1799686408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe874009de0 con 0x55f19d6ac1b0 2024-09-21T03:22:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.928+0000 7fe8637fe640 1 -- 172.21.15.83:0/1799686408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740196b0 con 0x55f19d6ac1b0 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.931+0000 7fe885ec8740 1 -- 172.21.15.83:0/1799686408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 msgr2=0x55f19da7cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.931+0000 7fe87ceef640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 0x7fe8640b8c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe87806aee0 tx=0x7fe86800a9b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.931+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe874009cf0 tx=0x7fe874011810 comp rx=0 tx=0).stop 2024-09-21T03:22:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 -- 172.21.15.83:0/1799686408 shutdown_connections 2024-09-21T03:22:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19da7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6ac1b0 0x55f19da7cbe0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1799686408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 -- 172.21.15.83:0/1799686408 >> 172.21.15.83:0/1799686408 conn(0x55f19d6c9460 msgr2=0x55f19da7a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 -- 172.21.15.83:0/1799686408 shutdown_connections 2024-09-21T03:22:36.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.932+0000 7fe885ec8740 1 -- 172.21.15.83:0/1799686408 wait complete. 2024-09-21T03:22:36.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 -- start start 2024-09-21T03:22:36.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 Processor -- start 2024-09-21T03:22:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f19d843700 con 0x55f19d6b4390 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f19d97d160 con 0x55f19d6ac1b0 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f19da8ae80 con 0x55f19d6d5070 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe885ec8740 10 client.0 _ll_register_callbacks cb 0x55f19d9969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.933+0000 7fe87ceef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87ceef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54892/0 (socket says 172.21.15.83:54892) 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87ceef640 1 -- 172.21.15.83:0/3459185569 learned_addr learned my addr 172.21.15.83:0/3459185569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 -- 172.21.15.83:0/3459185569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 msgr2=0x55f19da87d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da87d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 -- 172.21.15.83:0/3459185569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 msgr2=0x55f19da88280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 -- 172.21.15.83:0/3459185569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19d9aac90 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87ceef640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe87def1640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da88800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe870105f20 tx=0x7fe8700ce4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa68a308cdaf35f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700d7db0 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe8700d7040 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe874009de0 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.934+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe8700e9050 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 10 client.25984 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f19da96bb0 con 0x55f19d6d5070 2024-09-21T03:22:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe874009de0 con 0x55f19d6d5070 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fe8700c9f70 con 0x55f19d6d5070 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 10 client.25984 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 10 client.25984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 20 client.25984 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d6939f0 0x55f19da9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 4 client.25984 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19d9f4df0 con 0x55f19d6939f0 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.935+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d6939f0 0x55f19da9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.936+0000 7fe860ff9640 10 client.25984 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:22:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.936+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d6939f0 0x55f19da9a050 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da7fbc488648f712 server_cookie=c98519e7dcbe9443 in_seq=0 out_seq=0 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.936+0000 7fe860ff9640 1 -- 172.21.15.83:0/3459185569 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f19d9f4df0 con 0x55f19d6939f0 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.936+0000 7fe860ff9640 10 client.25984 handle_command_reply: tid=0 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 client.25984 shutdown 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 msgr2=0x55f19da88800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da88800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe870105f20 tx=0x7fe8700ce4b0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 shutdown_connections 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d6939f0 0x55f19da9a050 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da88800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da88280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 --2- 172.21.15.83:0/3459185569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe860ff9640 0 client.25984 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 >> 172.21.15.83:0/3459185569 conn(0x55f19d6c9460 msgr2=0x55f19da7f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 shutdown_connections 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.937+0000 7fe885ec8740 1 -- 172.21.15.83:0/3459185569 wait complete. 2024-09-21T03:22:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.938+0000 7fe885ec8740 20 client.25984 trim_cache size 0 max 16384 2024-09-21T03:22:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.976+0000 7fe885ec8740 1 Processor -- start 2024-09-21T03:22:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 -- start start 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00e500 con 0x55f19d6b4390 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c010100 con 0x55f19d6ac1b0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.979+0000 7fe885ec8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00ebd0 con 0x55f19d6d5070 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.980+0000 7fe87d6f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.980+0000 7fe87d6f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59996/0 (socket says 172.21.15.83:59996) 2024-09-21T03:22:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.980+0000 7fe87d6f0640 1 -- 172.21.15.83:0/2902993674 learned_addr learned my addr 172.21.15.83:0/2902993674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87ceef640 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 -- 172.21.15.83:0/2902993674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 msgr2=0x55f19da84f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da84f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 -- 172.21.15.83:0/2902993674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 msgr2=0x55f19da7dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da7dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 -- 172.21.15.83:0/2902993674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19da7b350 con 0x55f19d6ac1b0 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe87401b000 tx=0x7fe874009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dcdc02cc6d2af5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.981+0000 7fe87ceef640 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da7dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.982+0000 7fe862ffd640 1 -- 172.21.15.83:0/2902993674 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87401e040 con 0x55f19d6ac1b0 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.982+0000 7fe862ffd640 1 -- 172.21.15.83:0/2902993674 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe874031070 con 0x55f19d6ac1b0 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.983+0000 7fe885ec8740 1 -- 172.21.15.83:0/2902993674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 msgr2=0x55f19da89c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.983+0000 7fe885ec8740 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe87401b000 tx=0x7fe874009f70 comp rx=0 tx=0).stop 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 -- 172.21.15.83:0/2902993674 shutdown_connections 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6d5070 0x55f19da84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6b4390 0x55f19da7dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 --2- 172.21.15.83:0/2902993674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6ac1b0 0x55f19da89c20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 -- 172.21.15.83:0/2902993674 >> 172.21.15.83:0/2902993674 conn(0x55f19d6c9460 msgr2=0x55f19da7d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 -- 172.21.15.83:0/2902993674 shutdown_connections 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.985+0000 7fe885ec8740 1 -- 172.21.15.83:0/2902993674 wait complete. 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.986+0000 7fe885ec8740 1 -- start start 2024-09-21T03:22:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.986+0000 7fe885ec8740 1 Processor -- start 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6ac1b0 0x55f19dafe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19dafee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00fa30 con 0x55f19d6d5070 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00f2a0 con 0x55f19d6b4390 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00c4e0 con 0x55f19d6ac1b0 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe885ec8740 10 client.0 _ll_register_callbacks cb 0x7fe8700319f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.987+0000 7fe87def1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54910/0 (socket says 172.21.15.83:54910) 2024-09-21T03:22:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 -- 172.21.15.83:0/1706044221 learned_addr learned my addr 172.21.15.83:0/1706044221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 -- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6ac1b0 msgr2=0x55f19dafe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6ac1b0 0x55f19dafe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 -- 172.21.15.83:0/1706044221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 msgr2=0x55f19dafee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19dafee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.988+0000 7fe87def1640 1 -- 172.21.15.83:0/1706044221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19da90850 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe87def1640 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe8700ce320 tx=0x7fe870009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426af0f36c3749f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700ed020 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fe8700f0330 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f19d9f08a0 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.989+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe8700cd2f0 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 10 client.16458 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f19da8db80 con 0x55f19d6d5070 2024-09-21T03:22:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f19d9f08a0 con 0x55f19d6d5070 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fe8700d7020 con 0x55f19d6d5070 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 10 client.16458 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 20 client.16458 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d668da0 0x55f19db202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 4 client.16458 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:22:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.990+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f19d9edcd0 con 0x55f19d668da0 2024-09-21T03:22:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.993+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d668da0 0x55f19db202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.993+0000 7fe87d6f0640 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d668da0 0x55f19db202a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7dfdf476df8358e1 server_cookie=d0d990f73415b31d in_seq=0 out_seq=0 2024-09-21T03:22:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.993+0000 7fe860ff9640 10 client.16458 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe860ff9640 1 -- 172.21.15.83:0/1706044221 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f19d9edcd0 con 0x55f19d668da0 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe860ff9640 10 client.16458 handle_command_reply: tid=0 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 client.16458 shutdown 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 msgr2=0x55f19da90210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe8700ce320 tx=0x7fe870009460 comp rx=0 tx=0).stop 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 shutdown_connections 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55f19d668da0 0x55f19db202a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f19d6d5070 0x55f19da90210 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f19d6b4390 0x55f19dafee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe860ff9640 0 client.16458 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 --2- 172.21.15.83:0/1706044221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f19d6ac1b0 0x55f19dafe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 >> 172.21.15.83:0/1706044221 conn(0x55f19d6c9460 msgr2=0x55f19da89980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.994+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 shutdown_connections 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe885ec8740 1 -- 172.21.15.83:0/1706044221 wait complete. 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe885ec8740 20 client.16458 trim_cache size 0 max 16384 2024-09-21T03:22:36.997 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 msgr2=0x7fe8640b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 0x7fe8640b8c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe87806aee0 tx=0x7fe86800a9b0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 msgr2=0x7fe8781a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe870009430 tx=0x7fe8700317c0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 shutdown_connections 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8640b6700 0x7fe8640b8c10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe878108480 0x7fe8781a1ef0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe878107b30 0x7fe87819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 --2- 172.21.15.83:0/421942611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8781021f0 0x7fe87819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.995+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 >> 172.21.15.83:0/421942611 conn(0x7fe878077c60 msgr2=0x7fe878100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.996+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 shutdown_connections 2024-09-21T03:22:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:36.996+0000 7fe87f954640 1 -- 172.21.15.83:0/421942611 wait complete. 2024-09-21T03:22:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:37 smithi142 ceph-mon[34512]: pgmap v504: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-09-21T03:22:37.677 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:37 smithi083 ceph-mon[31219]: pgmap v504: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-09-21T03:22:37.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:37 smithi193 ceph-mon[35224]: pgmap v504: 97 pgs: 97 active+clean; 3.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-09-21T03:22:37.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:22:38.193 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:38.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.769+0000 7fda02104640 1 -- 172.21.15.83:0/877292384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc104840 msgr2=0x7fd9fc104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.769+0000 7fda02104640 1 --2- 172.21.15.83:0/877292384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc104840 0x7fd9fc104c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009900 tx=0x7fd9ec02a440 comp rx=0 tx=0).stop 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 -- 172.21.15.83:0/877292384 shutdown_connections 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 --2- 172.21.15.83:0/877292384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 --2- 172.21.15.83:0/877292384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc105270 0x7fd9fc10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 --2- 172.21.15.83:0/877292384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc104840 0x7fd9fc104c50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 -- 172.21.15.83:0/877292384 >> 172.21.15.83:0/877292384 conn(0x7fd9fc079730 msgr2=0x7fd9fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 -- 172.21.15.83:0/877292384 shutdown_connections 2024-09-21T03:22:38.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 -- 172.21.15.83:0/877292384 wait complete. 2024-09-21T03:22:38.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.770+0000 7fda02104640 1 Processor -- start 2024-09-21T03:22:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.771+0000 7fda02104640 1 -- start start 2024-09-21T03:22:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc104840 0x7fd9fc104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 0x7fd9fc102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc113b70 con 0x7fd9fc104840 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1139f0 con 0x7fd9fc10ddf0 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fda02104640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc113cf0 con 0x7fd9fc105270 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc104840 0x7fd9fc104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60038/0 (socket says 172.21.15.83:60038) 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2014204125 learned_addr learned my addr 172.21.15.83:0/2014204125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 0x7fd9fc102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 -- 172.21.15.83:0/2014204125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 msgr2=0x7fd9fc102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 -- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc104840 msgr2=0x7fd9fc104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc104840 0x7fd9fc104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.772+0000 7fd9faffd640 1 -- 172.21.15.83:0/2014204125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1034b0 con 0x7fd9fc105270 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 0x7fd9fc102750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc06ad30 tx=0x7fd9f000b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fd9f8ff9640 1 -- 172.21.15.83:0/2014204125 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0015020 con 0x7fd9fc105270 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fd9f8ff9640 1 -- 172.21.15.83:0/2014204125 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f0011ce0 con 0x7fd9fc105270 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc1ad060 con 0x7fd9fc105270 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fd9f8ff9640 1 -- 172.21.15.83:0/2014204125 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0010d80 con 0x7fd9fc105270 2024-09-21T03:22:38.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.773+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1ad5f0 con 0x7fd9fc105270 2024-09-21T03:22:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.777+0000 7fda08678740 1 Processor -- start 2024-09-21T03:22:38.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.777+0000 7fda08678740 1 -- start start 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.777+0000 7fd9f8ff9640 1 -- 172.21.15.83:0/2014204125 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd9f0018020 con 0x7fd9fc105270 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 0x55d0531c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d05313bea0 con 0x55d052e1e070 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d05312b2d0 con 0x55d052df51b0 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fda08678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d05312f0e0 con 0x55d052dfd390 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2024-09-21T03:22:38.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2341094478 learned_addr learned my addr 172.21.15.83:0/2341094478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 0x55d0531c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.778+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2341094478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 msgr2=0x55d0531c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2341094478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 msgr2=0x55d0531c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2341094478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d052e89650 con 0x55d052df51b0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 0x55d0531c5be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec006010 tx=0x7fd9ec005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d9975c355c89da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9f8ff9640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 0x7fd9e40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.779+0000 7fd9e1ffb640 1 -- 172.21.15.83:0/2341094478 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec035040 con 0x55d052df51b0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.780+0000 7fd9e1ffb640 1 -- 172.21.15.83:0/2341094478 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec0316a0 con 0x55d052df51b0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.780+0000 7fd9e1ffb640 1 -- 172.21.15.83:0/2341094478 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0047a0 con 0x55d052df51b0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.780+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 0x7fd9e40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.780+0000 7fd9f8ff9640 1 -- 172.21.15.83:0/2014204125 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd9f00e7140 con 0x7fd9fc105270 2024-09-21T03:22:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 0x7fd9e40b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40114a0 tx=0x7fd9f400b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 -- 172.21.15.83:0/2341094478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 msgr2=0x55d0531c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 0x55d0531c5be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec006010 tx=0x7fd9ec005f30 comp rx=0 tx=0).stop 2024-09-21T03:22:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 -- 172.21.15.83:0/2341094478 shutdown_connections 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052df51b0 0x55d0531c5be0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec006010 tx=0x7fd9ec005f30 comp rx=0 tx=0).stop 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 --2- 172.21.15.83:0/2341094478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 -- 172.21.15.83:0/2341094478 >> 172.21.15.83:0/2341094478 conn(0x55d052e12460 msgr2=0x55d0531c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 -- 172.21.15.83:0/2341094478 shutdown_connections 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.781+0000 7fda08678740 1 -- 172.21.15.83:0/2341094478 wait complete. 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.782+0000 7fda08678740 1 -- start start 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.782+0000 7fda08678740 1 Processor -- start 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d0531d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d052f8c700 con 0x55d052ddc9f0 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d0530c6160 con 0x55d052e1e070 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0531d3e20 con 0x55d052dfd390 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531dd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54946/0 (socket says 172.21.15.83:54946) 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2204487719 learned_addr learned my addr 172.21.15.83:0/2204487719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d0531d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.783+0000 7fda08678740 10 client.0 _ll_register_callbacks cb 0x55d0530df9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 msgr2=0x55d0531d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d0531d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2204487719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 msgr2=0x55d0531dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/2204487719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0530f3c90 con 0x55d052dfd390 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531d0de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55d0531c83a0 tx=0x7fd9ec0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d06625b682f0970 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec04e020 con 0x55d052dfd390 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ec0316a0 con 0x55d052dfd390 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd9ec0055e0 con 0x55d052dfd390 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.784+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd9ec035040 con 0x55d052dfd390 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.785+0000 7fda08678740 10 client.25996 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.785+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d0531dfb90 con 0x55d052dfd390 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.785+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec0316a0 con 0x55d052dfd390 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.785+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fd9ec003b40 con 0x55d052dfd390 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 10 client.25996 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 20 client.25996 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d052db1da0 0x55d0531e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 4 client.25996 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.786+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d05313ddf0 con 0x55d052db1da0 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.787+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d052db1da0 0x55d0531e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.788+0000 7fd9faffd640 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d052db1da0 0x55d0531e30d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82959cdc4ead795 server_cookie=ce50abc26b3991d7 in_seq=0 out_seq=0 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.788+0000 7fd9daffd640 10 client.25996 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.788+0000 7fd9daffd640 1 -- 172.21.15.83:0/2204487719 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d05313ddf0 con 0x55d052db1da0 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.788+0000 7fd9daffd640 10 client.25996 handle_command_reply: tid=0 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.788+0000 7fda08678740 1 client.25996 shutdown 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 msgr2=0x55d0531d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531d0de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55d0531c83a0 tx=0x7fd9ec0095c0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 shutdown_connections 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d052db1da0 0x55d0531e30d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d0531d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 --2- 172.21.15.83:0/2204487719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531d0de0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 >> 172.21.15.83:0/2204487719 conn(0x55d052e12460 msgr2=0x55d0531c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fd9daffd640 0 client.25996 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 shutdown_connections 2024-09-21T03:22:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 1 -- 172.21.15.83:0/2204487719 wait complete. 2024-09-21T03:22:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.789+0000 7fda08678740 20 client.25996 trim_cache size 0 max 16384 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.808+0000 7fda08678740 1 Processor -- start 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.808+0000 7fda08678740 1 -- start start 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b5d80 con 0x55d052e1e070 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b7980 con 0x55d052ddc9f0 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fda08678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b6450 con 0x55d052dfd390 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60050/0 (socket says 172.21.15.83:60050) 2024-09-21T03:22:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/1313217101 learned_addr learned my addr 172.21.15.83:0/1313217101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9faffd640 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/1313217101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 msgr2=0x55d0531dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/1313217101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 msgr2=0x55d0531cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.809+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.810+0000 7fd9fb7fe640 1 -- 172.21.15.83:0/1313217101 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0531d3520 con 0x55d052ddc9f0 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.810+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003ee0 tx=0x7fd9ec02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e54fcef2201c7a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.810+0000 7fd9e17fa640 1 -- 172.21.15.83:0/1313217101 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec04e070 con 0x55d052ddc9f0 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.811+0000 7fd9e17fa640 1 -- 172.21.15.83:0/1313217101 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec005dd0 con 0x55d052ddc9f0 2024-09-21T03:22:38.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.811+0000 7fd9e17fa640 1 -- 172.21.15.83:0/1313217101 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec004fa0 con 0x55d052ddc9f0 2024-09-21T03:22:38.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- 172.21.15.83:0/1313217101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 msgr2=0x55d0531d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003ee0 tx=0x7fd9ec02a440 comp rx=0 tx=0).stop 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- 172.21.15.83:0/1313217101 shutdown_connections 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052e1e070 0x55d0531cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 --2- 172.21.15.83:0/1313217101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052ddc9f0 0x55d0531d2a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- 172.21.15.83:0/1313217101 >> 172.21.15.83:0/1313217101 conn(0x55d052e12460 msgr2=0x55d0531d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- 172.21.15.83:0/1313217101 shutdown_connections 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- 172.21.15.83:0/1313217101 wait complete. 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.812+0000 7fda08678740 1 -- start start 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 Processor -- start 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d05324b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b72b0 con 0x55d052ddc9f0 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b6b20 con 0x55d052e1e070 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.813+0000 7fda08678740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b3d60 con 0x55d052dfd390 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fda08678740 10 client.0 _ll_register_callbacks cb 0x7fd9ec040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531e38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37520/0 (socket says 172.21.15.83:37520) 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d05324b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9faffd640 1 -- 172.21.15.83:0/2484808824 learned_addr learned my addr 172.21.15.83:0/2484808824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2484808824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 msgr2=0x55d0531e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2484808824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 msgr2=0x55d0531e3370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.814+0000 7fd9fbfff640 1 -- 172.21.15.83:0/2484808824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0531e41a0 con 0x55d052e1e070 2024-09-21T03:22:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.815+0000 7fd9fbfff640 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d05324b970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011250 tx=0x7fd9f4004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7197c5a3000346e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.815+0000 7fd9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.815+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f402c600 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fd9f402c7a0 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ec005dd0 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd9f402c9a0 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 10 client.26015 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d05324cff0 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec005dd0 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fd9f4024400 con 0x55d052e1e070 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 10 client.26015 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 20 client.26015 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d05324d780 0x55d05324fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 4 client.26015 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.816+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0531398a0 con 0x55d05324d780 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.817+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d05324d780 0x55d05324fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.817+0000 7fd9daffd640 10 client.26015 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.817+0000 7fd9fb7fe640 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d05324d780 0x55d05324fba0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece2d2f30b87185d server_cookie=cd4cb4db98bbfd06 in_seq=0 out_seq=0 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fd9daffd640 1 -- 172.21.15.83:0/2484808824 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0531398a0 con 0x55d05324d780 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fd9daffd640 10 client.26015 handle_command_reply: tid=0 2024-09-21T03:22:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 client.26015 shutdown 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 msgr2=0x55d05324b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d05324b970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011250 tx=0x7fd9f4004a60 comp rx=0 tx=0).stop 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 shutdown_connections 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d05324d780 0x55d05324fba0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d052e1e070 0x55d05324b970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d052dfd390 0x55d0531e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fd9daffd640 0 client.26015 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 --2- 172.21.15.83:0/2484808824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d052ddc9f0 0x55d0531e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 >> 172.21.15.83:0/2484808824 conn(0x55d052e12460 msgr2=0x55d0531c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 shutdown_connections 2024-09-21T03:22:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 1 -- 172.21.15.83:0/2484808824 wait complete. 2024-09-21T03:22:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.818+0000 7fda08678740 20 client.26015 trim_cache size 0 max 16384 2024-09-21T03:22:38.825 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 msgr2=0x7fd9e40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 0x7fd9e40b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40114a0 tx=0x7fd9f400b040 comp rx=0 tx=0).stop 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 msgr2=0x7fd9fc102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 0x7fd9fc102750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc06ad30 tx=0x7fd9f000b850 comp rx=0 tx=0).stop 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 shutdown_connections 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9e40b6700 0x7fd9e40b8c10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9fc10ddf0 0x7fd9fc102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9fc105270 0x7fd9fc102750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 --2- 172.21.15.83:0/2014204125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9fc104840 0x7fd9fc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 >> 172.21.15.83:0/2014204125 conn(0x7fd9fc079730 msgr2=0x7fd9fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 shutdown_connections 2024-09-21T03:22:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:38.819+0000 7fda02104640 1 -- 172.21.15.83:0/2014204125 wait complete. 2024-09-21T03:22:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:39 smithi142 ceph-mon[34512]: pgmap v505: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 757 op/s 2024-09-21T03:22:39.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:39 smithi083 ceph-mon[31219]: pgmap v505: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 757 op/s 2024-09-21T03:22:39.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:39 smithi193 ceph-mon[35224]: pgmap v505: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 757 op/s 2024-09-21T03:22:40.736 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:22:40.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:22:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 320.0 s, 1949.2 tps, lat 51.253 ms stddev 56.330 2024-09-21T03:22:41.123 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:41.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:41 smithi083 ceph-mon[31219]: pgmap v506: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-21T03:22:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:41 smithi142 ceph-mon[34512]: pgmap v506: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-21T03:22:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 -- 172.21.15.83:0/3507158691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e41083a0 msgr2=0x7f72e41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3507158691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e41083a0 0x7f72e41087b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009900 tx=0x7f72d802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 -- 172.21.15.83:0/3507158691 shutdown_connections 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3507158691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 0x7f72e410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3507158691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e41083a0 0x7f72e41087b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3507158691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4102a60 0x7f72e4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 -- 172.21.15.83:0/3507158691 >> 172.21.15.83:0/3507158691 conn(0x7f72e4079730 msgr2=0x7f72e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 -- 172.21.15.83:0/3507158691 shutdown_connections 2024-09-21T03:22:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.707+0000 7f72e9f53640 1 -- 172.21.15.83:0/3507158691 wait complete. 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 Processor -- start 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 -- start start 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 0x7f72e419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 0x7f72e419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72e4113ab0 con 0x7f72e4102a60 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72e4113930 con 0x7f72e41083a0 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e9f53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e4113c30 con 0x7f72e4108cf0 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 0x7f72e419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:48990/0 (socket says 172.21.15.83:48990) 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.708+0000 7f72e2ffd640 1 -- 172.21.15.83:0/3462776217 learned_addr learned my addr 172.21.15.83:0/3462776217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e3fff640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 0x7f72e419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e2ffd640 1 -- 172.21.15.83:0/3462776217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 msgr2=0x7f72e419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 0x7f72e419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e2ffd640 1 -- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 msgr2=0x7f72e419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 0x7f72e419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e2ffd640 1 -- 172.21.15.83:0/3462776217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e41a0230 con 0x7f72e41083a0 2024-09-21T03:22:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.709+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 0x7f72e419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009850 tx=0x7f72d80305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e0ff9640 1 -- 172.21.15.83:0/3462776217 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d803f070 con 0x7f72e41083a0 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e41a04c0 con 0x7f72e41083a0 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e0ff9640 1 -- 172.21.15.83:0/3462776217 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004460 con 0x7f72e41083a0 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e0ff9640 1 -- 172.21.15.83:0/3462776217 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8035030 con 0x7f72e41083a0 2024-09-21T03:22:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.710+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e410fc90 con 0x7f72e41083a0 2024-09-21T03:22:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 Processor -- start 2024-09-21T03:22:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 -- start start 2024-09-21T03:22:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752dfa490 0x5617531ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752e02670 0x5617531caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561753140e60 con 0x561752e02670 2024-09-21T03:22:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561753130290 con 0x561752dfa490 2024-09-21T03:22:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.714+0000 7f72f04c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617531340a0 con 0x561752e23350 2024-09-21T03:22:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:58064/0 (socket says 172.21.15.83:58064) 2024-09-21T03:22:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 -- 172.21.15.83:0/1692613416 learned_addr learned my addr 172.21.15.83:0/1692613416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 -- 172.21.15.83:0/1692613416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752dfa490 msgr2=0x5617531ca970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752dfa490 0x5617531ca970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 -- 172.21.15.83:0/1692613416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752e02670 msgr2=0x5617531caeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752e02670 0x5617531caeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.715+0000 7f72e3fff640 1 -- 172.21.15.83:0/1692613416 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561752e8e930 con 0x561752e23350 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.716+0000 7f72e3fff640 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00a430 tx=0x7f72dc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca220077d43e759 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.716+0000 7f72e0ff9640 1 -- 172.21.15.83:0/3462776217 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f72d8042070 con 0x7f72e41083a0 2024-09-21T03:22:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.717+0000 7f72c9ffb640 1 -- 172.21.15.83:0/1692613416 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc013790 con 0x561752e23350 2024-09-21T03:22:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.717+0000 7f72c9ffb640 1 -- 172.21.15.83:0/1692613416 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc019070 con 0x561752e23350 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.717+0000 7f72c9ffb640 1 -- 172.21.15.83:0/1692613416 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc014de0 con 0x561752e23350 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.717+0000 7f72e0ff9640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 0x7f72cc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 -- 172.21.15.83:0/1692613416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 msgr2=0x5617531ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00a430 tx=0x7f72dc009650 comp rx=0 tx=0).stop 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 0x7f72cc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 -- 172.21.15.83:0/1692613416 shutdown_connections 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752e02670 0x5617531caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752dfa490 0x5617531ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 --2- 172.21.15.83:0/1692613416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e23350 0x5617531ca430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 -- 172.21.15.83:0/1692613416 >> 172.21.15.83:0/1692613416 conn(0x561752e17740 msgr2=0x5617531c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72e0ff9640 1 -- 172.21.15.83:0/3462776217 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f72d8107ff0 con 0x7f72e41083a0 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 -- 172.21.15.83:0/1692613416 shutdown_connections 2024-09-21T03:22:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72f04c7740 1 -- 172.21.15.83:0/1692613416 wait complete. 2024-09-21T03:22:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.718+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 0x7f72cc0b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5617531c90b0 tx=0x7f72d4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 -- start start 2024-09-21T03:22:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 Processor -- start 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531dee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e23350 0x5617531ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561752f91890 con 0x561752dfa490 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5617530cad40 con 0x561752e23350 2024-09-21T03:22:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617531e1340 con 0x561752e02670 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72f04c7740 10 client.0 _ll_register_callbacks cb 0x56175312ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41276/0 (socket says 172.21.15.83:41276) 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.719+0000 7f72e37fe640 1 -- 172.21.15.83:0/2607551904 learned_addr learned my addr 172.21.15.83:0/2607551904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e3fff640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e23350 0x5617531ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 -- 172.21.15.83:0/2607551904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 msgr2=0x5617531dee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531dee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 -- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e23350 msgr2=0x5617531ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e23350 0x5617531ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 -- 172.21.15.83:0/2607551904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617531ead40 con 0x561752dfa490 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72e37fe640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f72d400fa30 tx=0x7f72d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece295046be80cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.720+0000 7f72c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d402b060 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f72d401cc50 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72dc019070 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f72d401ce50 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72f04c7740 10 client.16479 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.721+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5617531ed6d0 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc019070 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f72d401a430 con 0x561752dfa490 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 10 client.16479 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 20 client.16479 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752de1cd0 0x5617531f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 4 client.16479 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.722+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561753142db0 con 0x561752de1cd0 2024-09-21T03:22:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.723+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752de1cd0 0x5617531f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.723+0000 7f72c2ffd640 10 client.16479 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.723+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752de1cd0 0x5617531f0c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1ac2b9fd6236aa server_cookie=8499534649f137ca in_seq=0 out_seq=0 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72c2ffd640 1 -- 172.21.15.83:0/2607551904 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561753142db0 con 0x561752de1cd0 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72c2ffd640 10 client.16479 handle_command_reply: tid=0 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 client.16479 shutdown 2024-09-21T03:22:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 msgr2=0x5617531de8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f72d400fa30 tx=0x7f72d4004730 comp rx=0 tx=0).stop 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 shutdown_connections 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752de1cd0 0x5617531f0c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.724+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e23350 0x5617531ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72c2ffd640 0 client.16479 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531dee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2607551904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531de8b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 >> 172.21.15.83:0/2607551904 conn(0x561752e17740 msgr2=0x5617531c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 shutdown_connections 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 1 -- 172.21.15.83:0/2607551904 wait complete. 2024-09-21T03:22:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.725+0000 7f72f04c7740 20 client.16479 trim_cache size 0 max 16384 2024-09-21T03:22:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 Processor -- start 2024-09-21T03:22:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 -- start start 2024-09-21T03:22:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 0x5617531db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 0x5617531dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e02670 0x5617531f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41ad2e0 con 0x561752de1cd0 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41aeee0 con 0x561752e02670 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.750+0000 7f72f04c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41ad9b0 con 0x561752dfa490 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 0x5617531db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 0x5617531db8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41286/0 (socket says 172.21.15.83:41286) 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e37fe640 1 -- 172.21.15.83:0/2256966034 learned_addr learned my addr 172.21.15.83:0/2256966034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 0x5617531dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e3fff640 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e02670 0x5617531f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 -- 172.21.15.83:0/2256966034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e02670 msgr2=0x5617531f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e02670 0x5617531f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 -- 172.21.15.83:0/2256966034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 msgr2=0x5617531db8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 0x5617531db8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.751+0000 7f72e2ffd640 1 -- 172.21.15.83:0/2256966034 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617531c8860 con 0x561752dfa490 2024-09-21T03:22:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.752+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 0x5617531dbde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72d8105760 tx=0x7f72d8038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421ebbc756fa66f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.752+0000 7f72c97fa640 1 -- 172.21.15.83:0/2256966034 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8030e40 con 0x561752dfa490 2024-09-21T03:22:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.752+0000 7f72c97fa640 1 -- 172.21.15.83:0/2256966034 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80b91d0 con 0x561752dfa490 2024-09-21T03:22:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.752+0000 7f72c97fa640 1 -- 172.21.15.83:0/2256966034 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d81059e0 con 0x561752dfa490 2024-09-21T03:22:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 -- 172.21.15.83:0/2256966034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 msgr2=0x5617531dbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 0x5617531dbde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72d8105760 tx=0x7f72d8038000 comp rx=0 tx=0).stop 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 -- 172.21.15.83:0/2256966034 shutdown_connections 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752e02670 0x5617531f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752dfa490 0x5617531dbde0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 --2- 172.21.15.83:0/2256966034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752de1cd0 0x5617531db8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 -- 172.21.15.83:0/2256966034 >> 172.21.15.83:0/2256966034 conn(0x561752e17740 msgr2=0x5617531dcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 -- 172.21.15.83:0/2256966034 shutdown_connections 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.753+0000 7f72f04c7740 1 -- 172.21.15.83:0/2256966034 wait complete. 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.754+0000 7f72f04c7740 1 -- start start 2024-09-21T03:22:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.754+0000 7f72f04c7740 1 Processor -- start 2024-09-21T03:22:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41ae810 con 0x561752dfa490 2024-09-21T03:22:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41ae080 con 0x561752de1cd0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e41ab2e0 con 0x561752e02670 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72f04c7740 10 client.0 _ll_register_callbacks cb 0x7f72d4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49018/0 (socket says 172.21.15.83:49018) 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 -- 172.21.15.83:0/3428228189 learned_addr learned my addr 172.21.15.83:0/3428228189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e3fff640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 -- 172.21.15.83:0/3428228189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 msgr2=0x5617531e2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531e2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 -- 172.21.15.83:0/3428228189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 msgr2=0x5617531e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e37fe640 1 -- 172.21.15.83:0/3428228189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175325b610 con 0x561752de1cd0 2024-09-21T03:22:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e3fff640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531e2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.755+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.756+0000 7f72e37fe640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f72d401aca0 tx=0x7f72d401acd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4910ddbac087ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.756+0000 7f72c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.756+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d401c7e0 con 0x561752de1cd0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.756+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7f72d401a5a0 con 0x561752de1cd0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.757+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56175313e860 con 0x561752de1cd0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.757+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f72d401ca50 con 0x561752de1cd0 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 10 client.26030 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56175325c8a0 con 0x561752de1cd0 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d401cbf0 con 0x561752de1cd0 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f72d4039020 con 0x561752de1cd0 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 10 client.26030 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 20 client.26030 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752e23350 0x56175325f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 4 client.26030 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:22:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.758+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56175313e860 con 0x561752e23350 2024-09-21T03:22:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.759+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752e23350 0x56175325f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.759+0000 7f72c2ffd640 10 client.26030 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.759+0000 7f72e2ffd640 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752e23350 0x56175325f0c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2771eb1cbdd8 server_cookie=a7a8c004510c4652 in_seq=0 out_seq=0 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72c2ffd640 1 -- 172.21.15.83:0/3428228189 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56175313e860 con 0x561752e23350 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72c2ffd640 10 client.26030 handle_command_reply: tid=0 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 client.26030 shutdown 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 msgr2=0x5617531e15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f72d401aca0 tx=0x7f72d401acd0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 shutdown_connections 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561752e23350 0x56175325f0c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561752e02670 0x5617531e2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561752dfa490 0x5617531e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 --2- 172.21.15.83:0/3428228189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561752de1cd0 0x5617531e15f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.761+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 >> 172.21.15.83:0/3428228189 conn(0x561752e17740 msgr2=0x5617531dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.762+0000 7f72c2ffd640 0 client.26030 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.762+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 shutdown_connections 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.762+0000 7f72f04c7740 1 -- 172.21.15.83:0/3428228189 wait complete. 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.762+0000 7f72f04c7740 20 client.26030 trim_cache size 0 max 16384 2024-09-21T03:22:41.765 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0d6230b2-cfc0-4376-939d-c59a2997a3f3", 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 msgr2=0x7f72cc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 0x7f72cc0b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5617531c90b0 tx=0x7f72d4005f90 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 msgr2=0x7f72e419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009850 tx=0x7f72d80305d0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 shutdown_connections 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f72cc0b6700 0x7f72cc0b8c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e4108cf0 0x7f72e419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e41083a0 0x7f72e419a860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 --2- 172.21.15.83:0/3462776217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e4102a60 0x7f72e419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 >> 172.21.15.83:0/3462776217 conn(0x7f72e4079730 msgr2=0x7f72e41041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 shutdown_connections 2024-09-21T03:22:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:41.763+0000 7f72e9f53640 1 -- 172.21.15.83:0/3462776217 wait complete. 2024-09-21T03:22:41.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:41 smithi193 ceph-mon[35224]: pgmap v506: 97 pgs: 97 active+clean; 3.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-21T03:22:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:22:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:42 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:22:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:42 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:42 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:42.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:22:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.598 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:43 smithi083 ceph-mon[31219]: pgmap v507: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 734 op/s 2024-09-21T03:22:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:22:43.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:43 smithi142 ceph-mon[34512]: pgmap v507: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 734 op/s 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.728+0000 7fa9f3afb640 1 -- 172.21.15.83:0/629388958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec11cc30 msgr2=0x7fa9ec11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.728+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/629388958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec11f140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ca0 tx=0x7fa9e402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.728+0000 7fa9f3afb640 1 -- 172.21.15.83:0/629388958 shutdown_connections 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/629388958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec11f140 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/629388958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/629388958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec111f40 0x7fa9ec10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 -- 172.21.15.83:0/629388958 >> 172.21.15.83:0/629388958 conn(0x7fa9ec070ec0 msgr2=0x7fa9ec0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 -- 172.21.15.83:0/629388958 shutdown_connections 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.729+0000 7fa9f3afb640 1 -- 172.21.15.83:0/629388958 wait complete. 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 Processor -- start 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 -- start start 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec111f40 0x7fa9ec076f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec122900 con 0x7fa9ec10a9e0 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec122780 con 0x7fa9ec11cc30 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f3afb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec122a80 con 0x7fa9ec111f40 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.730+0000 7fa9f2098640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.731+0000 7fa9f2098640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49050/0 (socket says 172.21.15.83:49050) 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.731+0000 7fa9f2098640 1 -- 172.21.15.83:0/1754734080 learned_addr learned my addr 172.21.15.83:0/1754734080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.731+0000 7fa9f1096640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec111f40 0x7fa9ec076f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f2098640 1 -- 172.21.15.83:0/1754734080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec111f40 msgr2=0x7fa9ec076f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec111f40 0x7fa9ec076f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f2098640 1 -- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 msgr2=0x7fa9ec0769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec0769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.733+0000 7fa9f2098640 1 -- 172.21.15.83:0/1754734080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec07c900 con 0x7fa9ec11cc30 2024-09-21T03:22:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.734+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec0769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:22:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.734+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40098d0 tx=0x7fa9e4030cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.734+0000 7fa9e2ffd640 1 -- 172.21.15.83:0/1754734080 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e403b810 con 0x7fa9ec11cc30 2024-09-21T03:22:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.734+0000 7fa9e2ffd640 1 -- 172.21.15.83:0/1754734080 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e4030ea0 con 0x7fa9ec11cc30 2024-09-21T03:22:43.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.735+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec07cb90 con 0x7fa9ec11cc30 2024-09-21T03:22:43.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.735+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec0833d0 con 0x7fa9ec11cc30 2024-09-21T03:22:43.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.736+0000 7fa9e2ffd640 1 -- 172.21.15.83:0/1754734080 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4044dc0 con 0x7fa9ec11cc30 2024-09-21T03:22:43.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.741+0000 7fa9fa06f740 1 Processor -- start 2024-09-21T03:22:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.745+0000 7fa9e2ffd640 1 -- 172.21.15.83:0/1754734080 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa9e4043030 con 0x7fa9ec11cc30 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.745+0000 7fa9fa06f740 1 -- start start 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdba9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbaa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595bdb1fe50 con 0x5595bd7d91b0 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595bdb0f280 con 0x5595bd802070 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5595bdb13090 con 0x5595bd7e1390 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9f1096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9f1096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49056/0 (socket says 172.21.15.83:49056) 2024-09-21T03:22:43.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.746+0000 7fa9f1096640 1 -- 172.21.15.83:0/2063994745 learned_addr learned my addr 172.21.15.83:0/2063994745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:43.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.747+0000 7fa9f1897640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdba9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbaa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 -- 172.21.15.83:0/2063994745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 msgr2=0x5595bdbaa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbaa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 -- 172.21.15.83:0/2063994745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 msgr2=0x5595bdba9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdba9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 -- 172.21.15.83:0/2063994745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595bd86d650 con 0x5595bd802070 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1897640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdba9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9e2ffd640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 0x7fa9d80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.748+0000 7fa9f1096640 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00df80 tx=0x7fa9dc00d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a8f38f1578ef48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.749+0000 7fa9d7fff640 1 -- 172.21.15.83:0/2063994745 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004220 con 0x5595bd802070 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.749+0000 7fa9d7fff640 1 -- 172.21.15.83:0/2063994745 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9dc0043c0 con 0x5595bd802070 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.749+0000 7fa9d7fff640 1 -- 172.21.15.83:0/2063994745 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc003a30 con 0x5595bd802070 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.749+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 0x7fa9d80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.749+0000 7fa9e2ffd640 1 -- 172.21.15.83:0/1754734080 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9e41086e0 con 0x7fa9ec11cc30 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.750+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2063994745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 msgr2=0x5595bdba9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.750+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00df80 tx=0x7fa9dc00d490 comp rx=0 tx=0).stop 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.750+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 0x7fa9d80b8c40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5595bdba82d0 tx=0x7fa9e80094c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2063994745 shutdown_connections 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbaa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdba9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2063994745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdba9650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2063994745 >> 172.21.15.83:0/2063994745 conn(0x5595bd7f6460 msgr2=0x5595bdba7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2063994745 shutdown_connections 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2063994745 wait complete. 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.751+0000 7fa9fa06f740 1 -- start start 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.752+0000 7fa9fa06f740 1 Processor -- start 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.752+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbb4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.752+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbc1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9f1897640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbb4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9f1096640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9f1897640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbb4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49070/0 (socket says 172.21.15.83:49070) 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595bd970700 con 0x5595bd802070 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595bdaaa110 con 0x5595bd7d91b0 2024-09-21T03:22:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9f2098640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbc1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9f1897640 1 -- 172.21.15.83:0/3754933636 learned_addr learned my addr 172.21.15.83:0/3754933636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5595bdbb7e30 con 0x5595bd7e1390 2024-09-21T03:22:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.753+0000 7fa9fa06f740 10 client.0 _ll_register_callbacks cb 0x5595bdac3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 -- 172.21.15.83:0/3754933636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 msgr2=0x5595bdbb5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 -- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 msgr2=0x5595bdbb4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbb4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595bdad7c40 con 0x5595bd802070 2024-09-21T03:22:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9f2098640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbc1120 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4105e50 tx=0x7fa9e4106410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c64fb89a22451f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40b9020 con 0x5595bd802070 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9e40cd7b0 con 0x5595bd802070 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.754+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9dc0043c0 con 0x5595bd802070 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa9e40ce060 con 0x5595bd802070 2024-09-21T03:22:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 10 client.16494 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5595bdbc3b80 con 0x5595bd802070 2024-09-21T03:22:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9dc0043c0 con 0x5595bd802070 2024-09-21T03:22:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fa9e4079ce0 con 0x5595bd802070 2024-09-21T03:22:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 10 client.16494 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 20 client.16494 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.755+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd7c09f0 0x5595bdbc7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.756+0000 7fa9fa06f740 4 client.16494 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.756+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595bdb21da0 con 0x5595bd7c09f0 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.756+0000 7fa9f1897640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd7c09f0 0x5595bdbc7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.756+0000 7fa9d57fa640 10 client.16494 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9f1897640 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd7c09f0 0x5595bdbc7250 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9861b8bc0f638b server_cookie=c3774c720c9dc7f1 in_seq=0 out_seq=0 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9d57fa640 1 -- 172.21.15.83:0/3754933636 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595bdb21da0 con 0x5595bd7c09f0 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9d57fa640 10 client.16494 handle_command_reply: tid=0 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9fa06f740 1 client.16494 shutdown 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 msgr2=0x5595bdbc1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.757+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbc1120 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4105e50 tx=0x7fa9e4106410 comp rx=0 tx=0).stop 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 shutdown_connections 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd7c09f0 0x5595bdbc7250 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbc1120 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/3754933636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbb4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 >> 172.21.15.83:0/3754933636 conn(0x5595bd7f6460 msgr2=0x5595bdbac830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9d57fa640 0 client.16494 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 shutdown_connections 2024-09-21T03:22:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 1 -- 172.21.15.83:0/3754933636 wait complete. 2024-09-21T03:22:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.758+0000 7fa9fa06f740 20 client.16494 trim_cache size 0 max 16384 2024-09-21T03:22:43.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 Processor -- start 2024-09-21T03:22:43.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 -- start start 2024-09-21T03:22:43.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdbb52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbc1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdbac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1178a0 con 0x5595bd7d91b0 2024-09-21T03:22:43.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1194a0 con 0x5595bd802070 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.782+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec117f70 con 0x5595bd7e1390 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f2098640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdbac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdbb52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f2098640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdbac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:49082/0 (socket says 172.21.15.83:49082) 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f2098640 1 -- 172.21.15.83:0/2992466941 learned_addr learned my addr 172.21.15.83:0/2992466941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1096640 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbc1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 -- 172.21.15.83:0/2992466941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 msgr2=0x5595bdbc1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbc1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 -- 172.21.15.83:0/2992466941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 msgr2=0x5595bdbac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdbac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 -- 172.21.15.83:0/2992466941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595bdbb60a0 con 0x5595bd7d91b0 2024-09-21T03:22:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9f1897640 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdbb52a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5595bdbc7890 tx=0x7fa9e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6376fb15a2e242fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9d77fe640 1 -- 172.21.15.83:0/2992466941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e802d070 con 0x5595bd7d91b0 2024-09-21T03:22:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9d77fe640 1 -- 172.21.15.83:0/2992466941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e801fd20 con 0x5595bd7d91b0 2024-09-21T03:22:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.783+0000 7fa9d77fe640 1 -- 172.21.15.83:0/2992466941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e801ebe0 con 0x5595bd7d91b0 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2992466941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 msgr2=0x5595bdbb52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdbb52a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5595bdbc7890 tx=0x7fa9e8014000 comp rx=0 tx=0).stop 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2992466941 shutdown_connections 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd802070 0x5595bdbac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbc1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/2992466941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd7d91b0 0x5595bdbb52a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2992466941 >> 172.21.15.83:0/2992466941 conn(0x5595bd7f6460 msgr2=0x5595bdbb4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.784+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2992466941 shutdown_connections 2024-09-21T03:22:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.785+0000 7fa9fa06f740 1 -- 172.21.15.83:0/2992466941 wait complete. 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.785+0000 7fa9fa06f740 1 -- start start 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.785+0000 7fa9fa06f740 1 Processor -- start 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbb9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec118dd0 con 0x5595bd802070 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec118640 con 0x5595bd7d91b0 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec115880 con 0x5595bd7e1390 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.786+0000 7fa9fa06f740 10 client.0 _ll_register_callbacks cb 0x7fa9e4009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f1096640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbb9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f1096640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:58142/0 (socket says 172.21.15.83:58142) 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f1096640 1 -- 172.21.15.83:0/1719344011 learned_addr learned my addr 172.21.15.83:0/1719344011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:22:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 -- 172.21.15.83:0/1719344011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 msgr2=0x5595bdbb8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 -- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 msgr2=0x5595bdbbda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbbda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.787+0000 7fa9f2098640 1 -- 172.21.15.83:0/1719344011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595bdc30b50 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.788+0000 7fa9f2098640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbb9670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4031e00 tx=0x7fa9e40ca0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af34bc41a196b057 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.788+0000 7fa9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.788+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40b9040 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.788+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e801fd20 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.788+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8397+0+0 (secure 0 0 0) 0x7fa9e41080d0 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa9e40cdea0 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 10 client.16500 fetch_fsmap learned FSMap version 41 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5595bdc31cb0 con 0x5595bd802070 2024-09-21T03:22:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e801fd20 con 0x5595bd802070 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fa9e4079ce0 con 0x5595bd802070 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 10 client.16500 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 20 client.16500 populate_metadata read hostname 'smithi083' 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd795da0 0x5595bdc342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 4 client.16500 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.789+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595bdb1d850 con 0x5595bd795da0 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd795da0 0x5595bdc342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9d57fa640 10 client.16500 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9f1897640 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd795da0 0x5595bdc342e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedee51edbb07ca6 server_cookie=312831f2c72e7ef7 in_seq=0 out_seq=0 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9d57fa640 1 -- 172.21.15.83:0/1719344011 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x5595bdb1d850 con 0x5595bd795da0 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9d57fa640 10 client.16500 handle_command_reply: tid=0 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.790+0000 7fa9fa06f740 1 client.16500 shutdown 2024-09-21T03:22:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 msgr2=0x5595bdbb9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbb9670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4031e00 tx=0x7fa9e40ca0b0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 shutdown_connections 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5595bd795da0 0x5595bdc342e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595bd802070 0x5595bdbb9670 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595bd7e1390 0x5595bdbb8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 --2- 172.21.15.83:0/1719344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595bd7d91b0 0x5595bdbbda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9d57fa640 0 client.16500 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 >> 172.21.15.83:0/1719344011 conn(0x5595bd7f6460 msgr2=0x5595bdbac750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 shutdown_connections 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 1 -- 172.21.15.83:0/1719344011 wait complete. 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.791+0000 7fa9fa06f740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T03:22:43.796 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0d6230b2-cfc0-4376-939d-c59a2997a3f3", 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:22:43.797 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:22:43.800 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:22:43.802 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 msgr2=0x7fa9d80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 0x7fa9d80b8c40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5595bdba82d0 tx=0x7fa9e80094c0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 msgr2=0x7fa9ec07c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40098d0 tx=0x7fa9e4030cc0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 shutdown_connections 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d80b6730 0x7fa9d80b8c40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9ec11cc30 0x7fa9ec07c230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9ec111f40 0x7fa9ec076f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 --2- 172.21.15.83:0/1754734080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec10a9e0 0x7fa9ec0769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:43.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.792+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 >> 172.21.15.83:0/1754734080 conn(0x7fa9ec070ec0 msgr2=0x7fa9ec11d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.793+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 shutdown_connections 2024-09-21T03:22:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:22:43.793+0000 7fa9f3afb640 1 -- 172.21.15.83:0/1754734080 wait complete. 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:22:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:43 smithi193 ceph-mon[35224]: pgmap v507: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 734 op/s 2024-09-21T03:22:44.619 INFO:tasks.cephfs.filesystem:scrub status for tag:0d6230b2-cfc0-4376-939d-c59a2997a3f3 - {'path': '/', 'tag': '0d6230b2-cfc0-4376-939d-c59a2997a3f3', 'options': 'recursive,force'} 2024-09-21T03:22:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:45 smithi142 ceph-mon[34512]: pgmap v508: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 617 op/s 2024-09-21T03:22:45.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:45 smithi193 ceph-mon[35224]: pgmap v508: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 617 op/s 2024-09-21T03:22:45.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:45 smithi083 ceph-mon[31219]: pgmap v508: 97 pgs: 97 active+clean; 4.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 617 op/s 2024-09-21T03:22:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 325.0 s, 2807.6 tps, lat 35.827 ms stddev 40.734 2024-09-21T03:22:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:47 smithi142 ceph-mon[34512]: pgmap v509: 97 pgs: 97 active+clean; 4.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-09-21T03:22:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:47 smithi193 ceph-mon[35224]: pgmap v509: 97 pgs: 97 active+clean; 4.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-09-21T03:22:47.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:47 smithi083 ceph-mon[31219]: pgmap v509: 97 pgs: 97 active+clean; 4.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-09-21T03:22:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:48 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:48 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:48.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:48 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:48.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:48 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:48.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:48 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:48.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:48 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:49 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:22:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:49 smithi142 ceph-mon[34512]: pgmap v510: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.9 MiB/s wr, 766 op/s 2024-09-21T03:22:49.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:49.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:49.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:49 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:22:49.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:49 smithi083 ceph-mon[31219]: pgmap v510: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.9 MiB/s wr, 766 op/s 2024-09-21T03:22:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:49 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:22:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:49 smithi193 ceph-mon[35224]: pgmap v510: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.9 MiB/s wr, 766 op/s 2024-09-21T03:22:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 330.0 s, 2333.8 tps, lat 42.343 ms stddev 47.872 2024-09-21T03:22:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:51 smithi142 ceph-mon[34512]: pgmap v511: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T03:22:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:51 smithi193 ceph-mon[35224]: pgmap v511: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T03:22:51.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:51 smithi083 ceph-mon[31219]: pgmap v511: 97 pgs: 97 active+clean; 4.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-09-21T03:22:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:53 smithi142 ceph-mon[34512]: pgmap v512: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 690 op/s 2024-09-21T03:22:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:53 smithi193 ceph-mon[35224]: pgmap v512: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 690 op/s 2024-09-21T03:22:53.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:53 smithi083 ceph-mon[31219]: pgmap v512: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 690 op/s 2024-09-21T03:22:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 335.0 s, 1765.8 tps, lat 57.192 ms stddev 59.063 2024-09-21T03:22:56.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:56 smithi142 ceph-mon[34512]: pgmap v513: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-09-21T03:22:56.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:56 smithi193 ceph-mon[35224]: pgmap v513: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-09-21T03:22:56.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:56 smithi083 ceph-mon[31219]: pgmap v513: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 649 op/s 2024-09-21T03:22:58.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:58 smithi142 ceph-mon[34512]: pgmap v514: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 639 op/s 2024-09-21T03:22:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:58 smithi193 ceph-mon[35224]: pgmap v514: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 639 op/s 2024-09-21T03:22:58.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:58 smithi083 ceph-mon[31219]: pgmap v514: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 639 op/s 2024-09-21T03:22:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:22:59 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:22:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:22:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:22:59 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:22:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:59.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:59.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:22:59 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:23:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 340.0 s, 2132.4 tps, lat 41.479 ms stddev 46.643 2024-09-21T03:23:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:01 smithi083 ceph-mon[31219]: pgmap v515: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 709 op/s 2024-09-21T03:23:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:01 smithi142 ceph-mon[34512]: pgmap v515: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 709 op/s 2024-09-21T03:23:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:01 smithi193 ceph-mon[35224]: pgmap v515: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 709 op/s 2024-09-21T03:23:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:03 smithi083 ceph-mon[31219]: pgmap v516: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-21T03:23:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:03 smithi142 ceph-mon[34512]: pgmap v516: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-21T03:23:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:03 smithi193 ceph-mon[35224]: pgmap v516: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-21T03:23:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:04 smithi083 ceph-mon[31219]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:23:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:04 smithi142 ceph-mon[34512]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:23:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:04 smithi193 ceph-mon[35224]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:23:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:05 smithi083 ceph-mon[31219]: pgmap v518: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-09-21T03:23:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:05 smithi142 ceph-mon[34512]: pgmap v518: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-09-21T03:23:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:05 smithi193 ceph-mon[35224]: pgmap v518: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-09-21T03:23:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 345.0 s, 1440.0 tps, lat 77.691 ms stddev 161.758 2024-09-21T03:23:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:07 smithi083 ceph-mon[31219]: pgmap v519: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-09-21T03:23:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:07 smithi142 ceph-mon[34512]: pgmap v519: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-09-21T03:23:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:07 smithi193 ceph-mon[35224]: pgmap v519: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.50k op/s 2024-09-21T03:23:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:08 smithi142 ceph-mon[34512]: pgmap v520: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 1.83k op/s 2024-09-21T03:23:08.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:08 smithi193 ceph-mon[35224]: pgmap v520: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 1.83k op/s 2024-09-21T03:23:08.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:08 smithi083 ceph-mon[31219]: pgmap v520: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 4.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 MiB/s wr, 1.83k op/s 2024-09-21T03:23:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:10 smithi142 ceph-mon[34512]: pgmap v521: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2024-09-21T03:23:10.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:10 smithi193 ceph-mon[35224]: pgmap v521: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2024-09-21T03:23:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:10 smithi083 ceph-mon[31219]: pgmap v521: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2024-09-21T03:23:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 350.0 s, 2378.8 tps, lat 41.740 ms stddev 44.207 2024-09-21T03:23:12.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:12 smithi142 ceph-mon[34512]: pgmap v522: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-21T03:23:12.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:12 smithi193 ceph-mon[35224]: pgmap v522: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-21T03:23:12.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:12 smithi083 ceph-mon[31219]: pgmap v522: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 1.93k op/s 2024-09-21T03:23:14.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:23:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:14 smithi083 ceph-mon[31219]: pgmap v523: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-21T03:23:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:14 smithi083 ceph-mon[31219]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:23:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:14 smithi142 ceph-mon[34512]: pgmap v523: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-21T03:23:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:14 smithi142 ceph-mon[34512]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:23:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:14 smithi193 ceph-mon[35224]: pgmap v523: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 24 MiB/s wr, 1.40k op/s 2024-09-21T03:23:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:14 smithi193 ceph-mon[35224]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 -- 172.21.15.83:0/1094598184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 msgr2=0x7f9bf810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 --2- 172.21.15.83:0/1094598184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 0x7f9bf810f550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009be0 tx=0x7f9bf002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 -- 172.21.15.83:0/1094598184 shutdown_connections 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 --2- 172.21.15.83:0/1094598184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 0x7f9bf810f550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 --2- 172.21.15.83:0/1094598184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf81083d0 0x7f9bf81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 --2- 172.21.15.83:0/1094598184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf8102ae0 0x7f9bf8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 -- 172.21.15.83:0/1094598184 >> 172.21.15.83:0/1094598184 conn(0x7f9bf8079790 msgr2=0x7f9bf8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.595+0000 7f9bfd185640 1 -- 172.21.15.83:0/1094598184 shutdown_connections 2024-09-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.596+0000 7f9bfd185640 1 -- 172.21.15.83:0/1094598184 wait complete. 2024-09-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.597+0000 7f9bfd185640 1 Processor -- start 2024-09-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.597+0000 7f9bfd185640 1 -- start start 2024-09-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.598+0000 7f9bfd185640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf8102ae0 0x7f9bf8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.598+0000 7f9bfd185640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bfd185640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 0x7f9bf8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bfd185640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81129e0 con 0x7f9bf8102ae0 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bfd185640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8112860 con 0x7f9bf81083d0 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bfd185640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8112b60 con 0x7f9bf8108d20 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44648/0 (socket says 172.21.15.83:44648) 2024-09-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 -- 172.21.15.83:0/2493955304 learned_addr learned my addr 172.21.15.83:0/2493955304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 -- 172.21.15.83:0/2493955304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 msgr2=0x7f9bf8199ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:23:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 0x7f9bf8199ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 -- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf8102ae0 msgr2=0x7f9bf8199040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:23:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf8102ae0 0x7f9bf8199040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.599+0000 7f9bf659c640 1 -- 172.21.15.83:0/2493955304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf819e320 con 0x7f9bf81083d0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0058c0 tx=0x7f9bec00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bdffff640 1 -- 172.21.15.83:0/2493955304 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00cd30 con 0x7f9bf81083d0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bdffff640 1 -- 172.21.15.83:0/2493955304 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0108e0 con 0x7f9bf81083d0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bdffff640 1 -- 172.21.15.83:0/2493955304 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0188e0 con 0x7f9bf81083d0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf819e610 con 0x7f9bf81083d0 2024-09-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.600+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf819eb00 con 0x7f9bf81083d0 2024-09-21T03:23:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.604+0000 7f9bdffff640 1 -- 172.21.15.83:0/2493955304 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9bec01f020 con 0x7f9bf81083d0 2024-09-21T03:23:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9bdffff640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 0x7f9be00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9bdffff640 1 -- 172.21.15.83:0/2493955304 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9bec0e77d0 con 0x7f9bf81083d0 2024-09-21T03:23:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 Processor -- start 2024-09-21T03:23:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 -- start start 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 0x7f9be00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c4a828e50 con 0x564c4a4e21b0 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c4a818280 con 0x564c4a4ea390 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.606+0000 7f9c036f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564c4a81c090 con 0x564c4a50b070 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58280/0 (socket says 172.21.15.83:58280) 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 -- 172.21.15.83:0/1692195142 learned_addr learned my addr 172.21.15.83:0/1692195142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf759e640 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 -- 172.21.15.83:0/1692195142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 msgr2=0x564c4a8b2650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b2650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 -- 172.21.15.83:0/1692195142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 msgr2=0x564c4a8b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 -- 172.21.15.83:0/1692195142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4a576650 con 0x564c4a4e21b0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf659c640 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009230 tx=0x7f9bec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f746c9d2b3c5ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bdd7fa640 1 -- 172.21.15.83:0/1692195142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0e5390 con 0x564c4a4e21b0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 0x7f9be00b8c90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806c3e0 tx=0x7f9be800b370 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bdd7fa640 1 -- 172.21.15.83:0/1692195142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0b20f0 con 0x564c4a4e21b0 2024-09-21T03:23:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.607+0000 7f9bdd7fa640 1 -- 172.21.15.83:0/1692195142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0acad0 con 0x564c4a4e21b0 2024-09-21T03:23:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 -- 172.21.15.83:0/1692195142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 msgr2=0x564c4a8b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009230 tx=0x7f9bec00cf60 comp rx=0 tx=0).stop 2024-09-21T03:23:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 -- 172.21.15.83:0/1692195142 shutdown_connections 2024-09-21T03:23:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4e21b0 0x564c4a8b2b90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 --2- 172.21.15.83:0/1692195142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 -- 172.21.15.83:0/1692195142 >> 172.21.15.83:0/1692195142 conn(0x564c4a4ff460 msgr2=0x564c4a8b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 -- 172.21.15.83:0/1692195142 shutdown_connections 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.609+0000 7f9c036f9740 1 -- 172.21.15.83:0/1692195142 wait complete. 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 -- start start 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 Processor -- start 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c4a679700 con 0x564c4a50b070 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c4a7b3110 con 0x564c4a4ea390 2024-09-21T03:23:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564c4a8c0e30 con 0x564c4a4e21b0 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.610+0000 7f9c036f9740 10 client.0 _ll_register_callbacks cb 0x564c4a7cc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44668/0 (socket says 172.21.15.83:44668) 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58296/0 (socket says 172.21.15.83:58296) 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf659c640 1 -- 172.21.15.83:0/2049017385 learned_addr learned my addr 172.21.15.83:0/2049017385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 -- 172.21.15.83:0/2049017385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 msgr2=0x564c4a8bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 -- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 msgr2=0x564c4a8be2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8be2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 -- 172.21.15.83:0/2049017385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4a7e0c40 con 0x564c4a50b070 2024-09-21T03:23:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bf759e640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0034040 tx=0x7f9bf0030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b6099fad8f766d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.611+0000 7f9bdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0041030 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9bf0003970 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec0b20f0 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0036040 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9c036f9740 10 client.16509 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.612+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564c4a8ccba0 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0b20f0 con 0x564c4a50b070 2024-09-21T03:23:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9bf0004130 con 0x564c4a50b070 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 10 client.16509 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 20 client.16509 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a4c99f0 0x564c4a8d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 4 client.16509 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4a82ada0 con 0x564c4a4c99f0 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.613+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a4c99f0 0x564c4a8d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.614+0000 7f9bdaff5640 10 client.16509 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.614+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a4c99f0 0x564c4a8d00e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372fc873e7640970 server_cookie=c732f71da932e3d8 in_seq=0 out_seq=0 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.615+0000 7f9bdaff5640 1 -- 172.21.15.83:0/2049017385 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c4a82ada0 con 0x564c4a4c99f0 2024-09-21T03:23:15.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.615+0000 7f9bdaff5640 10 client.16509 handle_command_reply: tid=0 2024-09-21T03:23:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.615+0000 7f9c036f9740 1 client.16509 shutdown 2024-09-21T03:23:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 msgr2=0x564c4a8ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0034040 tx=0x7f9bf0030640 comp rx=0 tx=0).stop 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 shutdown_connections 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a4c99f0 0x564c4a8d00e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a50b070 0x564c4a8ca120 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4ea390 0x564c4a8be2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2049017385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 >> 172.21.15.83:0/2049017385 conn(0x564c4a4ff460 msgr2=0x564c4a8b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9bdaff5640 0 client.16509 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 shutdown_connections 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 1 -- 172.21.15.83:0/2049017385 wait complete. 2024-09-21T03:23:15.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.616+0000 7f9c036f9740 20 client.16509 trim_cache size 0 max 16384 2024-09-21T03:23:15.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.641+0000 7f9c036f9740 1 Processor -- start 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.641+0000 7f9c036f9740 1 -- start start 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.641+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4e21b0 0x564c4a8bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.641+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.641+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9c036f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b5d20 con 0x564c4a4ea390 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9c036f9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b78f0 con 0x564c4a4e21b0 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9c036f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b63f0 con 0x564c4a50b070 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58312/0 (socket says 172.21.15.83:58312) 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 -- 172.21.15.83:0/2334126155 learned_addr learned my addr 172.21.15.83:0/2334126155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4e21b0 0x564c4a8bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf759e640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 -- 172.21.15.83:0/2334126155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 msgr2=0x564c4a8b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 -- 172.21.15.83:0/2334126155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4e21b0 msgr2=0x564c4a8bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4e21b0 0x564c4a8bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf659c640 1 -- 172.21.15.83:0/2334126155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4a8bc270 con 0x564c4a4ea390 2024-09-21T03:23:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.642+0000 7f9bf759e640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:23:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.643+0000 7f9bf659c640 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ace50 tx=0x7f9bec0b6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec16af2dda5b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.644+0000 7f9bdcff9640 1 -- 172.21.15.83:0/2334126155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0e5390 con 0x564c4a4ea390 2024-09-21T03:23:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.644+0000 7f9bdcff9640 1 -- 172.21.15.83:0/2334126155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0b20f0 con 0x564c4a4ea390 2024-09-21T03:23:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.644+0000 7f9bdcff9640 1 -- 172.21.15.83:0/2334126155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec010cf0 con 0x564c4a4ea390 2024-09-21T03:23:15.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 -- 172.21.15.83:0/2334126155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 msgr2=0x564c4a8afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ace50 tx=0x7f9bec0b6490 comp rx=0 tx=0).stop 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 -- 172.21.15.83:0/2334126155 shutdown_connections 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a50b070 0x564c4a8b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8afb70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 --2- 172.21.15.83:0/2334126155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a4e21b0 0x564c4a8bfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 -- 172.21.15.83:0/2334126155 >> 172.21.15.83:0/2334126155 conn(0x564c4a4ff460 msgr2=0x564c4a8b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 -- 172.21.15.83:0/2334126155 shutdown_connections 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.645+0000 7f9c036f9740 1 -- 172.21.15.83:0/2334126155 wait complete. 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.646+0000 7f9c036f9740 1 -- start start 2024-09-21T03:23:15.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.646+0000 7f9c036f9740 1 Processor -- start 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.646+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.646+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.646+0000 7f9c036f9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a50b070 0x564c4a933b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9c036f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b7220 con 0x564c4a4ea390 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9c036f9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b6ac0 con 0x564c4a50b070 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9c036f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81b3d20 con 0x564c4a4e21b0 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9c036f9740 10 client.0 _ll_register_callbacks cb 0x7f9bec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a50b070 0x564c4a933b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58324/0 (socket says 172.21.15.83:58324) 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 -- 172.21.15.83:0/374167399 learned_addr learned my addr 172.21.15.83:0/374167399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 -- 172.21.15.83:0/374167399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 msgr2=0x564c4a8c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 -- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a50b070 msgr2=0x564c4a933b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a50b070 0x564c4a933b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.647+0000 7f9bf659c640 1 -- 172.21.15.83:0/374167399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4a9341e0 con 0x564c4a4ea390 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.648+0000 7f9bf659c640 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0e53c0 tx=0x7f9bec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb495de8b1bbec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.648+0000 7f9bdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.648+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0cf300 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.649+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9bec010430 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.649+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec0b20f0 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.649+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9bec010630 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.649+0000 7f9c036f9740 10 client.16515 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.649+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564c4a93b550 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0ac2e0 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9bec0b3280 con 0x564c4a4ea390 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 10 client.16515 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 20 client.16515 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a49eda0 0x564c4a93d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 4 client.16515 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9bec0b20f0 con 0x564c4a49eda0 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.650+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a49eda0 0x564c4a93d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.651+0000 7f9bdaff5640 10 client.16515 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.651+0000 7f9bf6d9d640 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a49eda0 0x564c4a93d730 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc968cae8d84c493 server_cookie=c86c3b6ed1b61783 in_seq=0 out_seq=0 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.651+0000 7f9bdaff5640 1 -- 172.21.15.83:0/374167399 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x7f9bec0b20f0 con 0x564c4a49eda0 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.651+0000 7f9bdaff5640 10 client.16515 handle_command_reply: tid=0 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.651+0000 7f9c036f9740 1 client.16515 shutdown 2024-09-21T03:23:15.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 msgr2=0x564c4a8c30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0e53c0 tx=0x7f9bec00cf60 comp rx=0 tx=0).stop 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 shutdown_connections 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x564c4a49eda0 0x564c4a93d730 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c4a50b070 0x564c4a933b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4a4ea390 0x564c4a8c30c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 --2- 172.21.15.83:0/374167399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c4a4e21b0 0x564c4a8c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 >> 172.21.15.83:0/374167399 conn(0x564c4a4ff460 msgr2=0x564c4a8b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9bdaff5640 0 client.16515 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 shutdown_connections 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 1 -- 172.21.15.83:0/374167399 wait complete. 2024-09-21T03:23:15.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.652+0000 7f9c036f9740 20 client.16515 trim_cache size 0 max 16384 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.658 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:23:15.659 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.660 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.661 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:15.662 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:15.663 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:15.663 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:15.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:23:15.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:23:15.663 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 msgr2=0x7f9be00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 0x7f9be00b8c90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806c3e0 tx=0x7f9be800b370 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 msgr2=0x7f9bf8199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0058c0 tx=0x7f9bec00ee50 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 shutdown_connections 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9be00b6780 0x7f9be00b8c90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8108d20 0x7f9bf8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bf81083d0 0x7f9bf8199580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 --2- 172.21.15.83:0/2493955304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf8102ae0 0x7f9bf8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 >> 172.21.15.83:0/2493955304 conn(0x7f9bf8079790 msgr2=0x7f9bf8106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:15.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 shutdown_connections 2024-09-21T03:23:15.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:15.653+0000 7f9bfd185640 1 -- 172.21.15.83:0/2493955304 wait complete. 2024-09-21T03:23:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 355.0 s, 2331.0 tps, lat 42.680 ms stddev 46.567 2024-09-21T03:23:16.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:15 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:16.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:15 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:16.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:15 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:16.481 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:23:16.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:23:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:16 smithi142 ceph-mon[34512]: pgmap v525: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-09-21T03:23:17.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:16 smithi083 ceph-mon[31219]: pgmap v525: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-09-21T03:23:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:16 smithi193 ceph-mon[35224]: pgmap v525: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-09-21T03:23:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.439+0000 7f880ae3e640 1 -- 172.21.15.83:0/3175524703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 msgr2=0x7f8804103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.439+0000 7f880ae3e640 1 --2- 172.21.15.83:0/3175524703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f8804103fa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-09-21T03:23:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 -- 172.21.15.83:0/3175524703 shutdown_connections 2024-09-21T03:23:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 --2- 172.21.15.83:0/3175524703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8804109e20 0x7f8804110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 --2- 172.21.15.83:0/3175524703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88041094d0 0x7f88041098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 --2- 172.21.15.83:0/3175524703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f8804103fa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 -- 172.21.15.83:0/3175524703 >> 172.21.15.83:0/3175524703 conn(0x7f88040ff5e0 msgr2=0x7f8804101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 -- 172.21.15.83:0/3175524703 shutdown_connections 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.440+0000 7f880ae3e640 1 -- 172.21.15.83:0/3175524703 wait complete. 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 Processor -- start 2024-09-21T03:23:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 -- start start 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f880419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88041094d0 0x7f880419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8804113b40 con 0x7f88041094d0 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88041139c0 con 0x7f8804103b90 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f880ae3e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8804113cc0 con 0x7f8804109e20 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f88093db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.441+0000 7f88093db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:42860/0 (socket says 172.21.15.83:42860) 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 -- 172.21.15.83:0/2920364289 learned_addr learned my addr 172.21.15.83:0/2920364289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f8808bda640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f880419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f8803fff640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88041094d0 0x7f880419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 -- 172.21.15.83:0/2920364289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 msgr2=0x7f880419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f880419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 -- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88041094d0 msgr2=0x7f880419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88041094d0 0x7f880419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 -- 172.21.15.83:0/2920364289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88041a26a0 con 0x7f8804109e20 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.442+0000 7f88093db640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00b970 tx=0x7f87fc00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.443+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc019070 con 0x7f8804109e20 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.443+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87fc014440 con 0x7f8804109e20 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.443+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88041a2990 con 0x7f8804109e20 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.443+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0075d0 con 0x7f8804109e20 2024-09-21T03:23:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.443+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88041a90f0 con 0x7f8804109e20 2024-09-21T03:23:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.444+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8804103fa0 con 0x7f8804109e20 2024-09-21T03:23:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.446+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f87fc01c070 con 0x7f8804109e20 2024-09-21T03:23:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.447+0000 7f8801ffb640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 0x7f87f40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.447+0000 7f8808bda640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 0x7f87f40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.447+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f87fc0e6b40 con 0x7f8804109e20 2024-09-21T03:23:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.448+0000 7f8808bda640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 0x7f87f40b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87f8031440 tx=0x7f87f8009560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:17.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.451+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87fc0b14c0 con 0x7f8804109e20 2024-09-21T03:23:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.783+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88041a9460 con 0x7f8804109e20 2024-09-21T03:23:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.783+0000 7f8801ffb640 1 -- 172.21.15.83:0/2920364289 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11987 (secure 0 0 0) 0x7f87fc0b6400 con 0x7f8804109e20 2024-09-21T03:23:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:23:17.786 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":42,"btime":"2024-09-21T03:23:14:903524+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:23:14.045223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:23:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 42 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 msgr2=0x7f87f40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 0x7f87f40b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87f8031440 tx=0x7f87f8009560 comp rx=0 tx=0).stop 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 msgr2=0x7f88041a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00b970 tx=0x7f87fc00be30 comp rx=0 tx=0).stop 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 shutdown_connections 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f87f40b6700 0x7f87f40b8c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8804109e20 0x7f88041a1f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88041094d0 0x7f880419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 --2- 172.21.15.83:0/2920364289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8804103b90 0x7f880419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.790+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 >> 172.21.15.83:0/2920364289 conn(0x7f88040ff5e0 msgr2=0x7f8804108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.792+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 shutdown_connections 2024-09-21T03:23:17.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:17.793+0000 7f880ae3e640 1 -- 172.21.15.83:0/2920364289 wait complete. 2024-09-21T03:23:18.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:17 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2920364289' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:23:18.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:17 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2920364289' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:23:18.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:17 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2920364289' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:23:18.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:23:19.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:18 smithi083 ceph-mon[31219]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-09-21T03:23:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:18 smithi193 ceph-mon[35224]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-09-21T03:23:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:18 smithi142 ceph-mon[34512]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-09-21T03:23:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 -- 172.21.15.83:0/624258061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c1049b0 msgr2=0x7fc65c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 --2- 172.21.15.83:0/624258061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c1049b0 0x7fc65c104dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009900 tx=0x7fc64c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 -- 172.21.15.83:0/624258061 shutdown_connections 2024-09-21T03:23:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 --2- 172.21.15.83:0/624258061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 --2- 172.21.15.83:0/624258061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c105300 0x7fc65c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 --2- 172.21.15.83:0/624258061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c1049b0 0x7fc65c104dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 -- 172.21.15.83:0/624258061 >> 172.21.15.83:0/624258061 conn(0x7fc65c079730 msgr2=0x7fc65c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 -- 172.21.15.83:0/624258061 shutdown_connections 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.812+0000 7fc662395640 1 -- 172.21.15.83:0/624258061 wait complete. 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 Processor -- start 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 -- start start 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 0x7fc65c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c113a70 con 0x7fc65c1049b0 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1138f0 con 0x7fc65c105300 2024-09-21T03:23:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc662395640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c113bf0 con 0x7fc65c10de10 2024-09-21T03:23:19.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.813+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 0x7fc65c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43274/0 (socket says 172.21.15.83:43274) 2024-09-21T03:23:19.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 -- 172.21.15.83:0/3776776246 learned_addr learned my addr 172.21.15.83:0/3776776246 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:19.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc660932640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 -- 172.21.15.83:0/3776776246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 msgr2=0x7fc65c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 -- 172.21.15.83:0/3776776246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 msgr2=0x7fc65c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 0x7fc65c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 -- 172.21.15.83:0/3776776246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c1a04f0 con 0x7fc65c1049b0 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc660932640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 0x7fc65c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc65bfff640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc64c032fd0 tx=0x7fc64c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.814+0000 7fc6597fa640 1 -- 172.21.15.83:0/3776776246 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c03f030 con 0x7fc65c1049b0 2024-09-21T03:23:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.815+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1a0780 con 0x7fc65c1049b0 2024-09-21T03:23:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.815+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a6f40 con 0x7fc65c1049b0 2024-09-21T03:23:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.817+0000 7fc6597fa640 1 -- 172.21.15.83:0/3776776246 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc64c004490 con 0x7fc65c1049b0 2024-09-21T03:23:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.817+0000 7fc6597fa640 1 -- 172.21.15.83:0/3776776246 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c035070 con 0x7fc65c1049b0 2024-09-21T03:23:19.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.819+0000 7fc6597fa640 1 -- 172.21.15.83:0/3776776246 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc64c042030 con 0x7fc65c1049b0 2024-09-21T03:23:19.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.820+0000 7fc668909740 1 Processor -- start 2024-09-21T03:23:19.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.820+0000 7fc668909740 1 -- start start 2024-09-21T03:23:19.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc6597fa640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 0x7fc6440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 0x55be80f41be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f42120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be80eb7ea0 con 0x55be80b711b0 2024-09-21T03:23:19.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be80ea72d0 con 0x55be80b9a070 2024-09-21T03:23:19.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc668909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be80eab0e0 con 0x55be80b79390 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc660932640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 0x7fc6440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc6597fa640 1 -- 172.21.15.83:0/3776776246 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc64c107870 con 0x7fc65c1049b0 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36822/0 (socket says 172.21.15.83:36822) 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 -- 172.21.15.83:0/2594077344 learned_addr learned my addr 172.21.15.83:0/2594077344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 0x55be80f41be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc660932640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f42120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 -- 172.21.15.83:0/2594077344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 msgr2=0x55be80f42120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f42120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 -- 172.21.15.83:0/2594077344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 msgr2=0x55be80f41be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 0x55be80f41be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc65bfff640 1 -- 172.21.15.83:0/2594077344 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be80c05650 con 0x55be80b9a070 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.821+0000 7fc660932640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f42120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0c9d50 tx=0x7fc64c03b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd766e9b7ed8eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 0x55be80f41be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc6427fc640 1 -- 172.21.15.83:0/2594077344 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0cd290 con 0x55be80b9a070 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc660932640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 0x7fc6440b8c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc654005fd0 tx=0x7fc654009ac0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc6427fc640 1 -- 172.21.15.83:0/2594077344 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc64c0cde10 con 0x55be80b9a070 2024-09-21T03:23:19.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.822+0000 7fc6427fc640 1 -- 172.21.15.83:0/2594077344 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0c92d0 con 0x55be80b9a070 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 -- 172.21.15.83:0/2594077344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 msgr2=0x55be80f416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0c9d50 tx=0x7fc64c03b460 comp rx=0 tx=0).stop 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 -- 172.21.15.83:0/2594077344 shutdown_connections 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f42120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b711b0 0x55be80f41be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 --2- 172.21.15.83:0/2594077344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b9a070 0x55be80f416a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.823+0000 7fc668909740 1 -- 172.21.15.83:0/2594077344 >> 172.21.15.83:0/2594077344 conn(0x55be80b8e460 msgr2=0x55be80f3f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.824+0000 7fc668909740 1 -- 172.21.15.83:0/2594077344 shutdown_connections 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.824+0000 7fc668909740 1 -- 172.21.15.83:0/2594077344 wait complete. 2024-09-21T03:23:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.824+0000 7fc668909740 1 -- start start 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.825+0000 7fc668909740 1 Processor -- start 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.825+0000 7fc668909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.825+0000 7fc668909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f4d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.825+0000 7fc668909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b9a070 0x55be80f58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.825+0000 7fc668909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be80d08700 con 0x55be80b9a070 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc668909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be80e42160 con 0x55be80b711b0 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36828/0 (socket says 172.21.15.83:36828) 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 -- 172.21.15.83:0/2099973183 learned_addr learned my addr 172.21.15.83:0/2099973183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be80ea72d0 con 0x55be80b79390 2024-09-21T03:23:19.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc660932640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b9a070 0x55be80f58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f4d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc668909740 10 client.0 _ll_register_callbacks cb 0x55be80e5b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 -- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 msgr2=0x55be80f4d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f4d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 -- 172.21.15.83:0/2099973183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b9a070 msgr2=0x55be80f58d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b9a070 0x55be80f58d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 -- 172.21.15.83:0/2099973183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be80e6fc90 con 0x55be80b711b0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc65bfff640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0cd710 tx=0x7fc64c105540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31731ffeb4a73116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.826+0000 7fc63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0ccbe0 con 0x55be80b711b0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc64c0c90d0 con 0x55be80b711b0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc64c0cde10 con 0x55be80b711b0 2024-09-21T03:23:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc64c0b3030 con 0x55be80b711b0 2024-09-21T03:23:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc668909740 10 client.26081 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55be80f5b660 con 0x55be80b711b0 2024-09-21T03:23:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc64c0cde10 con 0x55be80b711b0 2024-09-21T03:23:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fc64c031a90 con 0x55be80b711b0 2024-09-21T03:23:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc668909740 10 client.26081 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.827+0000 7fc668909740 10 client.26081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:23:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.828+0000 7fc668909740 20 client.26081 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.828+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b589f0 0x55be80f5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.828+0000 7fc668909740 4 client.26081 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.828+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be80eb9df0 con 0x55be80b589f0 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.828+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b589f0 0x55be80f5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.829+0000 7fc63b7fe640 10 client.26081 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.829+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b589f0 0x55be80f5eb40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6711949bca23401 server_cookie=1bdca83af7df3049 in_seq=0 out_seq=0 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.829+0000 7fc63b7fe640 1 -- 172.21.15.83:0/2099973183 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be80eb9df0 con 0x55be80b589f0 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.829+0000 7fc63b7fe640 10 client.26081 handle_command_reply: tid=0 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.829+0000 7fc668909740 1 client.26081 shutdown 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 msgr2=0x55be80f4ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0cd710 tx=0x7fc64c105540 comp rx=0 tx=0).stop 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 shutdown_connections 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b589f0 0x55be80f5eb40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc63b7fe640 0 client.26081 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b9a070 0x55be80f58d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b79390 0x55be80f4d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 --2- 172.21.15.83:0/2099973183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f4ce30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 >> 172.21.15.83:0/2099973183 conn(0x55be80b8e460 msgr2=0x55be80f44830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 shutdown_connections 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 1 -- 172.21.15.83:0/2099973183 wait complete. 2024-09-21T03:23:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.830+0000 7fc668909740 20 client.26081 trim_cache size 0 max 16384 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.848+0000 7fc668909740 1 Processor -- start 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.848+0000 7fc668909740 1 -- start start 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b589f0 0x55be80f44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f58cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800e630 con 0x55be80b8e460 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc648010200 con 0x55be80b711b0 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc668909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800ed00 con 0x55be80b589f0 2024-09-21T03:23:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43298/0 (socket says 172.21.15.83:43298) 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f58cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc65bfff640 1 -- 172.21.15.83:0/3191397083 learned_addr learned my addr 172.21.15.83:0/3191397083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b589f0 0x55be80f44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 -- 172.21.15.83:0/3191397083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b589f0 msgr2=0x55be80f44eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b589f0 0x55be80f44eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 -- 172.21.15.83:0/3191397083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 msgr2=0x55be80f4bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc660932640 1 -- 172.21.15.83:0/3191397083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be80f43990 con 0x55be80b711b0 2024-09-21T03:23:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.849+0000 7fc65bfff640 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.850+0000 7fc660932640 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f58cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc654012df0 tx=0x7fc654014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690533ed85153f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.850+0000 7fc641ffb640 1 -- 172.21.15.83:0/3191397083 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65402d070 con 0x55be80b711b0 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.850+0000 7fc641ffb640 1 -- 172.21.15.83:0/3191397083 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc654020b90 con 0x55be80b711b0 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.850+0000 7fc641ffb640 1 -- 172.21.15.83:0/3191397083 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65401d5e0 con 0x55be80b711b0 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- 172.21.15.83:0/3191397083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 msgr2=0x55be80f58cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f58cb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc654012df0 tx=0x7fc654014040 comp rx=0 tx=0).stop 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- 172.21.15.83:0/3191397083 shutdown_connections 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b711b0 0x55be80f58cb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b589f0 0x55be80f44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 --2- 172.21.15.83:0/3191397083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b8e460 0x55be80f4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- 172.21.15.83:0/3191397083 >> 172.21.15.83:0/3191397083 conn(0x55be80f3a780 msgr2=0x55be80f447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- 172.21.15.83:0/3191397083 shutdown_connections 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- 172.21.15.83:0/3191397083 wait complete. 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.851+0000 7fc668909740 1 -- start start 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 Processor -- start 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 0x55be80f514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 0x55be80f519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 0x55be80fc8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800fb30 con 0x55be80b589f0 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 0x55be80f514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 0x55be80f514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43308/0 (socket says 172.21.15.83:43308) 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc65bfff640 1 -- 172.21.15.83:0/1805702298 learned_addr learned my addr 172.21.15.83:0/1805702298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:19.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800f3d0 con 0x55be80b8e460 2024-09-21T03:23:19.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800c630 con 0x55be80b711b0 2024-09-21T03:23:19.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc668909740 10 client.0 _ll_register_callbacks cb 0x7fc64c0c9640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:19.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.852+0000 7fc660932640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 0x55be80fc8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 0x55be80f519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 -- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 msgr2=0x55be80f519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 0x55be80f519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 -- 172.21.15.83:0/1805702298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 msgr2=0x55be80f514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:19.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 0x55be80f514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be80fc8af0 con 0x55be80b8e460 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc65b7fe640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 0x55be80f519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc660932640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 0x55be80fc8500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc654028c00 tx=0x7fc654028a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef8627fce37d0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.855+0000 7fc63b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65402d040 con 0x55be80b8e460 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc654020b90 con 0x55be80b8e460 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc654020990 con 0x55be80b8e460 2024-09-21T03:23:19.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc65400fe60 con 0x55be80b8e460 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc668909740 10 client.26087 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55be80fc9d50 con 0x55be80b8e460 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc654020b90 con 0x55be80b8e460 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.856+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fc654030030 con 0x55be80b8e460 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 10 client.26087 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 10 client.26087 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 20 client.26087 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b79390 0x55be80fcc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 4 client.26087 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:23:19.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be80eb58a0 con 0x55be80b79390 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc65bfff640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b79390 0x55be80fcc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc63b7fe640 10 client.26087 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.857+0000 7fc65bfff640 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b79390 0x55be80fcc480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea1f14a81491b2d9 server_cookie=46a88db22d3c5a87 in_seq=0 out_seq=0 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc63b7fe640 1 -- 172.21.15.83:0/1805702298 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be80eb58a0 con 0x55be80b79390 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc63b7fe640 10 client.26087 handle_command_reply: tid=0 2024-09-21T03:23:19.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 client.26087 shutdown 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 msgr2=0x55be80fc8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 0x55be80fc8500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc654028c00 tx=0x7fc654028a80 comp rx=0 tx=0).stop 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 shutdown_connections 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55be80b79390 0x55be80fcc480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be80b8e460 0x55be80fc8500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc63b7fe640 0 client.26087 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be80b711b0 0x55be80f519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 --2- 172.21.15.83:0/1805702298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be80b589f0 0x55be80f514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 >> 172.21.15.83:0/1805702298 conn(0x55be80f3a780 msgr2=0x55be80f3edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 shutdown_connections 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 1 -- 172.21.15.83:0/1805702298 wait complete. 2024-09-21T03:23:19.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.858+0000 7fc668909740 20 client.26087 trim_cache size 0 max 16384 2024-09-21T03:23:19.862 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:23:19.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 msgr2=0x7fc6440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 0x7fc6440b8c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc654005fd0 tx=0x7fc654009ac0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 msgr2=0x7fc65c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:19.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc64c032fd0 tx=0x7fc64c031a20 comp rx=0 tx=0).stop 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 shutdown_connections 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc6440b6700 0x7fc6440b8c10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc65c10de10 0x7fc65c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc65c105300 0x7fc65c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 --2- 172.21.15.83:0/3776776246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc65c1049b0 0x7fc65c19a580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 >> 172.21.15.83:0/3776776246 conn(0x7fc65c079730 msgr2=0x7fc65c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 shutdown_connections 2024-09-21T03:23:19.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:19.859+0000 7fc662395640 1 -- 172.21.15.83:0/3776776246 wait complete. 2024-09-21T03:23:20.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:23:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 360.0 s, 2251.4 tps, lat 44.886 ms stddev 51.299 2024-09-21T03:23:21.085 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:21.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:20 smithi083 ceph-mon[31219]: pgmap v527: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-09-21T03:23:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:20 smithi193 ceph-mon[35224]: pgmap v527: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-09-21T03:23:21.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:20 smithi142 ceph-mon[34512]: pgmap v527: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 -- 172.21.15.83:0/126778870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0103080 msgr2=0x7f46a0103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 --2- 172.21.15.83:0/126778870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0103080 0x7f46a0103490 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002a410 comp rx=0 tx=0).stop 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 -- 172.21.15.83:0/126778870 shutdown_connections 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 --2- 172.21.15.83:0/126778870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0105ac0 0x7f46a010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 --2- 172.21.15.83:0/126778870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a0105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 --2- 172.21.15.83:0/126778870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0103080 0x7f46a0103490 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 -- 172.21.15.83:0/126778870 >> 172.21.15.83:0/126778870 conn(0x7f46a0077c60 msgr2=0x7f46a00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 -- 172.21.15.83:0/126778870 shutdown_connections 2024-09-21T03:23:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.701+0000 7f46a501d640 1 -- 172.21.15.83:0/126778870 wait complete. 2024-09-21T03:23:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.702+0000 7f46a501d640 1 Processor -- start 2024-09-21T03:23:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 -- start start 2024-09-21T03:23:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0103080 0x7f46a010c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0105ac0 0x7f46a010d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111900 con 0x7f46a0105ac0 2024-09-21T03:23:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111780 con 0x7f46a0103080 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f46a501d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111a80 con 0x7f46a0105170 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f469e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f469f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0105ac0 0x7f46a010d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f469e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:34276/0 (socket says 172.21.15.83:34276) 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f469e59c640 1 -- 172.21.15.83:0/2964043477 learned_addr learned my addr 172.21.15.83:0/2964043477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.703+0000 7f469ed9d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0103080 0x7f46a010c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 -- 172.21.15.83:0/2964043477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0103080 msgr2=0x7f46a010c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0103080 0x7f46a010c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 -- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0105ac0 msgr2=0x7f46a010d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0105ac0 0x7f46a010d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 -- 172.21.15.83:0/2964043477 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a0073ee0 con 0x7f46a0105170 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f469e59c640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f469400e970 tx=0x7f469400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f4687fff640 1 -- 172.21.15.83:0/2964043477 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400ccc0 con 0x7f46a0105170 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a00741d0 con 0x7f46a0105170 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f4687fff640 1 -- 172.21.15.83:0/2964043477 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940035f0 con 0x7f46a0105170 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f4687fff640 1 -- 172.21.15.83:0/2964043477 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400ce60 con 0x7f46a0105170 2024-09-21T03:23:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.704+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a00746e0 con 0x7f46a0105170 2024-09-21T03:23:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.708+0000 7f4687fff640 1 -- 172.21.15.83:0/2964043477 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4694003aa0 con 0x7f46a0105170 2024-09-21T03:23:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.709+0000 7f4687fff640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 0x7f46880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.709+0000 7f4687fff640 1 -- 172.21.15.83:0/2964043477 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f46940e7990 con 0x7f46a0105170 2024-09-21T03:23:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.709+0000 7f469ed9d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 0x7f46880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.709+0000 7f46ab591740 1 Processor -- start 2024-09-21T03:23:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.709+0000 7f46ab591740 1 -- start start 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 0x55c7d679c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6712ea0 con 0x55c7d63d4390 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7d67022d0 con 0x55c7d63cc1b0 2024-09-21T03:23:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f46ab591740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c7d67060e0 con 0x55c7d63f5070 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36890/0 (socket says 172.21.15.83:36890) 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 -- 172.21.15.83:0/3913994239 learned_addr learned my addr 172.21.15.83:0/3913994239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469f59e640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43338/0 (socket says 172.21.15.83:43338) 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469ed9d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 0x7f46880b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ab0 tx=0x7f4690005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469ed9d640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 0x55c7d679c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 -- 172.21.15.83:0/3913994239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 msgr2=0x55c7d679c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 0x55c7d679c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 -- 172.21.15.83:0/3913994239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 msgr2=0x55c7d679d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469e59c640 1 -- 172.21.15.83:0/3913994239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d6460650 con 0x55c7d63cc1b0 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.710+0000 7f469ed9d640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 0x55c7d679c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.711+0000 7f469e59c640 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f46940a9da0 tx=0x7f46940ad000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494a154a985098cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.711+0000 7f46857fa640 1 -- 172.21.15.83:0/3913994239 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940b22b0 con 0x55c7d63cc1b0 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.711+0000 7f46857fa640 1 -- 172.21.15.83:0/3913994239 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940b71f0 con 0x55c7d63cc1b0 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.711+0000 7f46857fa640 1 -- 172.21.15.83:0/3913994239 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a9120 con 0x55c7d63cc1b0 2024-09-21T03:23:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 -- 172.21.15.83:0/3913994239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 msgr2=0x55c7d679cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f46940a9da0 tx=0x7f46940ad000 comp rx=0 tx=0).stop 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 -- 172.21.15.83:0/3913994239 shutdown_connections 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63cc1b0 0x55c7d679cbe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 --2- 172.21.15.83:0/3913994239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63f5070 0x55c7d679c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.712+0000 7f46ab591740 1 -- 172.21.15.83:0/3913994239 >> 172.21.15.83:0/3913994239 conn(0x55c7d63e9460 msgr2=0x55c7d679a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.713+0000 7f46ab591740 1 -- 172.21.15.83:0/3913994239 shutdown_connections 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.713+0000 7f46ab591740 1 -- 172.21.15.83:0/3913994239 wait complete. 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.713+0000 7f46ab591740 1 -- start start 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.713+0000 7f46ab591740 1 Processor -- start 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d67a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7d6563700 con 0x55c7d63d4390 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7d669d160 con 0x55c7d63f5070 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c7d67aaed0 con 0x55c7d63b39f0 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f46ab591740 10 client.0 _ll_register_callbacks cb 0x55c7d66b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36894/0 (socket says 172.21.15.83:36894) 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 -- 172.21.15.83:0/4164859286 learned_addr learned my addr 172.21.15.83:0/4164859286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469ed9d640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d67a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469f59e640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 -- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 msgr2=0x55c7d67b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 -- 172.21.15.83:0/4164859286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 msgr2=0x55c7d67a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.714+0000 7f469e59c640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d67a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.715+0000 7f469e59c640 1 -- 172.21.15.83:0/4164859286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d66cac90 con 0x55c7d63f5070 2024-09-21T03:23:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.717+0000 7f469e59c640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f46940e5580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4703cf337a5513a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.717+0000 7f4682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.717+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940aeb00 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.717+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46940b71f0 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.717+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f46940a9120 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f46940b22b0 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 10 client.26099 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c7d67b6d00 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940b71f0 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f46940b6730 con 0x55c7d63f5070 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 10 client.26099 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 10 client.26099 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 20 client.26099 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d6388da0 0x55c7d67ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.718+0000 7f46ab591740 4 client.26099 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.719+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7d6714df0 con 0x55c7d6388da0 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.719+0000 7f469ed9d640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d6388da0 0x55c7d67ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.719+0000 7f4682ff5640 10 client.26099 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.719+0000 7f469ed9d640 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d6388da0 0x55c7d67ba060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9e59ffc5d41d413 server_cookie=fa08cd8847bc1591 in_seq=0 out_seq=0 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.720+0000 7f4682ff5640 1 -- 172.21.15.83:0/4164859286 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7d6714df0 con 0x55c7d6388da0 2024-09-21T03:23:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.720+0000 7f4682ff5640 10 client.26099 handle_command_reply: tid=0 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.720+0000 7f46ab591740 1 client.26099 shutdown 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.720+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 msgr2=0x55c7d67a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.720+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f46940e5580 comp rx=0 tx=0).stop 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.721+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 shutdown_connections 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.721+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d6388da0 0x55c7d67ba060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63f5070 0x55c7d67a83a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f4682ff5640 0 client.26099 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:23:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 --2- 172.21.15.83:0/4164859286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d67a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 >> 172.21.15.83:0/4164859286 conn(0x55c7d63e9460 msgr2=0x55c7d679f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 shutdown_connections 2024-09-21T03:23:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 1 -- 172.21.15.83:0/4164859286 wait complete. 2024-09-21T03:23:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.722+0000 7f46ab591740 20 client.26099 trim_cache size 0 max 16384 2024-09-21T03:23:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.740+0000 7f46ab591740 1 Processor -- start 2024-09-21T03:23:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.740+0000 7f46ab591740 1 -- start start 2024-09-21T03:23:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.740+0000 7f46ab591740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 0x55c7d679a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.740+0000 7f46ab591740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 0x55c7d67b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.740+0000 7f46ab591740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f46ab591740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01be410 con 0x55c7d63d4390 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f46ab591740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01bffe0 con 0x55c7d63b39f0 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 0x55c7d679a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f46ab591740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01beae0 con 0x55c7d6388da0 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 0x55c7d679a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:34308/0 (socket says 172.21.15.83:34308) 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469ed9d640 1 -- 172.21.15.83:0/2922566776 learned_addr learned my addr 172.21.15.83:0/2922566776 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 0x55c7d67b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 -- 172.21.15.83:0/2922566776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 msgr2=0x55c7d679a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 0x55c7d679a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 -- 172.21.15.83:0/2922566776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 msgr2=0x55c7d679f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 -- 172.21.15.83:0/2922566776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d679e990 con 0x55c7d63b39f0 2024-09-21T03:23:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.741+0000 7f469e59c640 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 0x55c7d67b4130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46940ad4d0 tx=0x7f46940e51b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46de415037e070c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.742+0000 7f4684ff9640 1 -- 172.21.15.83:0/2922566776 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a9e20 con 0x55c7d63b39f0 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.742+0000 7f4684ff9640 1 -- 172.21.15.83:0/2922566776 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940b71f0 con 0x55c7d63b39f0 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.742+0000 7f4684ff9640 1 -- 172.21.15.83:0/2922566776 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940ac4a0 con 0x55c7d63b39f0 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 -- 172.21.15.83:0/2922566776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 msgr2=0x55c7d67b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 0x55c7d67b4130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46940ad4d0 tx=0x7f46940e51b0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 -- 172.21.15.83:0/2922566776 shutdown_connections 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d63d4390 0x55c7d679f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63b39f0 0x55c7d67b4130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 --2- 172.21.15.83:0/2922566776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d6388da0 0x55c7d679a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.743+0000 7f46ab591740 1 -- 172.21.15.83:0/2922566776 >> 172.21.15.83:0/2922566776 conn(0x55c7d63e9460 msgr2=0x55c7d6799eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.744+0000 7f46ab591740 1 -- 172.21.15.83:0/2922566776 shutdown_connections 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.744+0000 7f46ab591740 1 -- 172.21.15.83:0/2922566776 wait complete. 2024-09-21T03:23:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.744+0000 7f46ab591740 1 -- start start 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.744+0000 7f46ab591740 1 Processor -- start 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 0x55c7d67bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01bf910 con 0x55c7d6388da0 2024-09-21T03:23:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01bf1b0 con 0x55c7d63d4390 2024-09-21T03:23:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01bc410 con 0x55c7d63b39f0 2024-09-21T03:23:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f46ab591740 10 client.0 _ll_register_callbacks cb 0x7f4698009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43370/0 (socket says 172.21.15.83:43370) 2024-09-21T03:23:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469ed9d640 1 -- 172.21.15.83:0/3873648777 learned_addr learned my addr 172.21.15.83:0/3873648777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 0x55c7d67bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469e59c640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 -- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 msgr2=0x55c7d67bb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67bb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 -- 172.21.15.83:0/3873648777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 msgr2=0x55c7d67bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469f59e640 1 -- 172.21.15.83:0/3873648777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d68239e0 con 0x55c7d63d4390 2024-09-21T03:23:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.745+0000 7f469ed9d640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f469f59e640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 0x55c7d67bbcf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f469800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d088509608b02bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f4682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698004b20 con 0x55c7d63d4390 2024-09-21T03:23:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f4698004cc0 con 0x55c7d63d4390 2024-09-21T03:23:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46940b71f0 con 0x55c7d63d4390 2024-09-21T03:23:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4698005020 con 0x55c7d63d4390 2024-09-21T03:23:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f46ab591740 10 client.26105 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.746+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c7d6824cd0 con 0x55c7d63d4390 2024-09-21T03:23:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.747+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940b71f0 con 0x55c7d63d4390 2024-09-21T03:23:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.747+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f469801b070 con 0x55c7d63d4390 2024-09-21T03:23:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.747+0000 7f46ab591740 10 client.26105 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.747+0000 7f46ab591740 10 client.26105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:23:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.748+0000 7f46ab591740 20 client.26105 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.748+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d63f5070 0x55c7d6827540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.748+0000 7f46ab591740 4 client.26105 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:23:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.748+0000 7f469ed9d640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d63f5070 0x55c7d6827540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.748+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7d67108a0 con 0x55c7d63f5070 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.749+0000 7f469ed9d640 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d63f5070 0x55c7d6827540 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b0b82dcd54ca0ee server_cookie=98ccaf20e8caead1 in_seq=0 out_seq=0 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.749+0000 7f4682ff5640 10 client.26105 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.749+0000 7f4682ff5640 1 -- 172.21.15.83:0/3873648777 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7d67108a0 con 0x55c7d63f5070 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.749+0000 7f4682ff5640 10 client.26105 handle_command_reply: tid=0 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.749+0000 7f46ab591740 1 client.26105 shutdown 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.750+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 msgr2=0x55c7d67bbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.750+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 0x55c7d67bbcf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f469800f790 comp rx=0 tx=0).stop 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.750+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 shutdown_connections 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.750+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7d63d4390 0x55c7d67bbcf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.750+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c7d63f5070 0x55c7d6827540 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7d63b39f0 0x55c7d67bb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f4682ff5640 0 client.26105 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 1 --2- 172.21.15.83:0/3873648777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7d6388da0 0x55c7d67bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 >> 172.21.15.83:0/3873648777 conn(0x55c7d63e9460 msgr2=0x55c7d679f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 shutdown_connections 2024-09-21T03:23:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 1 -- 172.21.15.83:0/3873648777 wait complete. 2024-09-21T03:23:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.751+0000 7f46ab591740 20 client.26105 trim_cache size 0 max 16384 2024-09-21T03:23:21.767 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:23:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 msgr2=0x7f46880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 0x7f46880b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ab0 tx=0x7f4690005a20 comp rx=0 tx=0).stop 2024-09-21T03:23:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 msgr2=0x7f46a010ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f469400e970 tx=0x7f469400ed70 comp rx=0 tx=0).stop 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 shutdown_connections 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f46880b6680 0x7f46880b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a0105ac0 0x7f46a010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a0105170 0x7f46a010ce30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 --2- 172.21.15.83:0/2964043477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46a0103080 0x7f46a010c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 >> 172.21.15.83:0/2964043477 conn(0x7f46a0077c60 msgr2=0x7f46a0106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 shutdown_connections 2024-09-21T03:23:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:21.752+0000 7f46a501d640 1 -- 172.21.15.83:0/2964043477 wait complete. 2024-09-21T03:23:22.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:23:23.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:22 smithi193 ceph-mon[35224]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-09-21T03:23:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:23.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:22 smithi083 ceph-mon[31219]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-09-21T03:23:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:22 smithi142 ceph-mon[34512]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 679 op/s 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.926+0000 7f3f93499640 1 -- 172.21.15.83:0/2654009636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c11cba0 msgr2=0x7f3f8c11efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.926+0000 7f3f93499640 1 --2- 172.21.15.83:0/2654009636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c11cba0 0x7f3f8c11efc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f8801a910 tx=0x7f3f8803b450 comp rx=0 tx=0).stop 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.928+0000 7f3f93499640 1 -- 172.21.15.83:0/2654009636 shutdown_connections 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.928+0000 7f3f93499640 1 --2- 172.21.15.83:0/2654009636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c11cba0 0x7f3f8c11efc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.928+0000 7f3f93499640 1 --2- 172.21.15.83:0/2654009636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c10dbf0 0x7f3f8c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.928+0000 7f3f93499640 1 --2- 172.21.15.83:0/2654009636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.928+0000 7f3f93499640 1 -- 172.21.15.83:0/2654009636 >> 172.21.15.83:0/2654009636 conn(0x7f3f8c06e9f0 msgr2=0x7f3f8c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.930+0000 7f3f93499640 1 -- 172.21.15.83:0/2654009636 shutdown_connections 2024-09-21T03:23:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.932+0000 7f3f93499640 1 -- 172.21.15.83:0/2654009636 wait complete. 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 Processor -- start 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 -- start start 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c10dbf0 0x7f3f8c11c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c11cba0 0x7f3f8c118c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114c00 con 0x7f3f8c10dbf0 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114a80 con 0x7f3f8c11cba0 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.933+0000 7f3f93499640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c114d80 con 0x7f3f8c111cd0 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.936+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.936+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:34334/0 (socket says 172.21.15.83:34334) 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.936+0000 7f3f91c96640 1 -- 172.21.15.83:0/3832019270 learned_addr learned my addr 172.21.15.83:0/3832019270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.936+0000 7f3f92497640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c10dbf0 0x7f3f8c11c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 -- 172.21.15.83:0/3832019270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c11cba0 msgr2=0x7f3f8c118c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c11cba0 0x7f3f8c118c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 -- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c10dbf0 msgr2=0x7f3f8c11c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c10dbf0 0x7f3f8c11c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 -- 172.21.15.83:0/3832019270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c119430 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b450 tx=0x7f3f7c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f837fe640 1 -- 172.21.15.83:0/3832019270 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c015020 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c07b680 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c07bc40 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f837fe640 1 -- 172.21.15.83:0/3832019270 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c011d40 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.937+0000 7f3f837fe640 1 -- 172.21.15.83:0/3832019270 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c010e40 con 0x7f3f8c111cd0 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.940+0000 7f3f9ac6f740 1 Processor -- start 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.940+0000 7f3f9ac6f740 1 -- start start 2024-09-21T03:23:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565123b07ea0 con 0x5651237c9390 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565123af72d0 con 0x5651237ea070 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565123afb0e0 con 0x5651237c11b0 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92c98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92c98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b91860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36932/0 (socket says 172.21.15.83:36932) 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92c98640 1 -- 172.21.15.83:0/977301936 learned_addr learned my addr 172.21.15.83:0/977301936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:34346/0 (socket says 172.21.15.83:34346) 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 -- 172.21.15.83:0/977301936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 msgr2=0x565123b91860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b91860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 -- 172.21.15.83:0/977301936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 msgr2=0x565123b922e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b922e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f92497640 1 -- 172.21.15.83:0/977301936 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123855650 con 0x5651237c11b0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.941+0000 7f3f837fe640 1 -- 172.21.15.83:0/3832019270 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3f7c018020 con 0x7f3f8c111cd0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f92497640 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c110950 tx=0x7f3f84010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92761da74753aab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f80ff9640 1 -- 172.21.15.83:0/977301936 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84016070 con 0x5651237c11b0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f80ff9640 1 -- 172.21.15.83:0/977301936 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f84012900 con 0x5651237c11b0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f837fe640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 0x7f3f780b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f80ff9640 1 -- 172.21.15.83:0/977301936 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840119a0 con 0x5651237c11b0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f92c98640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 0x7f3f780b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.942+0000 7f3f837fe640 1 -- 172.21.15.83:0/3832019270 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f3f7c0e6f20 con 0x7f3f8c111cd0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/977301936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 msgr2=0x565123b91da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f92c98640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 0x7f3f780b8c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f8801a6d0 tx=0x7f3f8803b950 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c110950 tx=0x7f3f84010800 comp rx=0 tx=0).stop 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/977301936 shutdown_connections 2024-09-21T03:23:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b91da0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/977301936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b91860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/977301936 >> 172.21.15.83:0/977301936 conn(0x5651237de460 msgr2=0x565123b8f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/977301936 shutdown_connections 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.943+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/977301936 wait complete. 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 -- start start 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 Processor -- start 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b9ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565123958700 con 0x5651237c9390 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565123a92160 con 0x5651237ea070 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565123ba0090 con 0x5651237c11b0 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f9ac6f740 10 client.0 _ll_register_callbacks cb 0x565123aab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:24.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43386/0 (socket says 172.21.15.83:43386) 2024-09-21T03:23:24.626 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:23:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.944+0000 7f3f91c96640 1 -- 172.21.15.83:0/3076617893 learned_addr learned my addr 172.21.15.83:0/3076617893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f92497640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b9ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f92c98640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 -- 172.21.15.83:0/3076617893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 msgr2=0x565123b9ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b9ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 -- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 msgr2=0x565123b9d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 -- 172.21.15.83:0/3076617893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565123abfc90 con 0x5651237c9390 2024-09-21T03:23:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009fd0 tx=0x7f3f7c0107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758fccc9915e0f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.945+0000 7f3f75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c092d30 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f84012900 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f3f7c0d41b0 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c0ad460 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f9ac6f740 10 client.16560 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x565123babda0 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.947+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f84012900 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f3f7c0b4ce0 con 0x5651237c9390 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 10 client.16560 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 20 client.16560 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5651237a89f0 0x565123baf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 4 client.16560 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565123b09df0 con 0x5651237a89f0 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.948+0000 7f3f92497640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5651237a89f0 0x565123baf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.949+0000 7f3f92497640 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5651237a89f0 0x565123baf790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d6f1445797c1e7e server_cookie=d85aa74c6dd3f007 in_seq=0 out_seq=0 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.949+0000 7f3f75ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f75ffb640 1 -- 172.21.15.83:0/3076617893 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565123b09df0 con 0x5651237a89f0 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f75ffb640 10 client.16560 handle_command_reply: tid=0 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 client.16560 shutdown 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 msgr2=0x565123b9d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009fd0 tx=0x7f3f7c0107b0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 shutdown_connections 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5651237a89f0 0x565123baf790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9d3d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3076617893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b9ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 >> 172.21.15.83:0/3076617893 conn(0x5651237de460 msgr2=0x565123b949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f75ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 shutdown_connections 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3076617893 wait complete. 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.950+0000 7f3f9ac6f740 20 client.16560 trim_cache size 0 max 16384 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.963+0000 7f3f9ac6f740 1 Processor -- start 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.963+0000 7f3f9ac6f740 1 -- start start 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b95320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7000e6b0 con 0x5651237c9390 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f70010280 con 0x5651237ea070 2024-09-21T03:23:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7000ed80 con 0x5651237c11b0 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43400/0 (socket says 172.21.15.83:43400) 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 -- 172.21.15.83:0/3721148405 learned_addr learned my addr 172.21.15.83:0/3721148405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f92497640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f92c98640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 -- 172.21.15.83:0/3721148405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 msgr2=0x565123b95320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b95320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 -- 172.21.15.83:0/3721148405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 msgr2=0x565123b9a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 -- 172.21.15.83:0/3721148405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123b9e150 con 0x5651237c9390 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f92497640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b95320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f91c96640 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0a9fd0 tx=0x7f3f7c0ac930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834377d677d5b9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f77fff640 1 -- 172.21.15.83:0/3721148405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0c6070 con 0x5651237c9390 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f77fff640 1 -- 172.21.15.83:0/3721148405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0b1030 con 0x5651237c9390 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.964+0000 7f3f77fff640 1 -- 172.21.15.83:0/3721148405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0a87d0 con 0x5651237c9390 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3721148405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 msgr2=0x565123b9eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0a9fd0 tx=0x7f3f7c0ac930 comp rx=0 tx=0).stop 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3721148405 shutdown_connections 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237ea070 0x565123b9a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237c9390 0x565123b9eac0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/3721148405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123b95320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3721148405 >> 172.21.15.83:0/3721148405 conn(0x5651237de460 msgr2=0x565123b9e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3721148405 shutdown_connections 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.965+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/3721148405 wait complete. 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 -- start start 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 Processor -- start 2024-09-21T03:23:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123ba5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237c9390 0x565123ba2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7000fbb0 con 0x5651237ea070 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7000f450 con 0x5651237c9390 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7000c6b0 con 0x5651237c11b0 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f9ac6f740 10 client.0 _ll_register_callbacks cb 0x7f3f7c004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f92c98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f92c98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43406/0 (socket says 172.21.15.83:43406) 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f92c98640 1 -- 172.21.15.83:0/919756785 learned_addr learned my addr 172.21.15.83:0/919756785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f91c96640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237c9390 0x565123ba2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.966+0000 7f3f92497640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123ba5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 -- 172.21.15.83:0/919756785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 msgr2=0x565123ba5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123ba5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 -- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237c9390 msgr2=0x565123ba2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237c9390 0x565123ba2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 -- 172.21.15.83:0/919756785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565123ba3960 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92497640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123ba5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f92c98640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3f8803b9c0 tx=0x7f3f8801a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf89b2c434873ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88046050 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f3f88005020 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c0b1030 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3f88069020 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f9ac6f740 10 client.16566 fetch_fsmap learned FSMap version 42 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x565123c19eb0 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.967+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0b1030 con 0x5651237ea070 2024-09-21T03:23:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f3f88003cc0 con 0x5651237ea070 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 10 client.16566 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 20 client.16566 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56512377dda0 0x565123c1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 4 client.16566 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565123b058a0 con 0x56512377dda0 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f92497640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56512377dda0 0x565123c1c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f92497640 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56512377dda0 0x565123c1c6d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75818f1d43b137ce server_cookie=1bff8956db259685 in_seq=0 out_seq=0 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.968+0000 7f3f75ffb640 10 client.16566 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f75ffb640 1 -- 172.21.15.83:0/919756785 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565123b058a0 con 0x56512377dda0 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f75ffb640 10 client.16566 handle_command_reply: tid=0 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 client.16566 shutdown 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 msgr2=0x565123ba33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3f8803b9c0 tx=0x7f3f8801a270 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 shutdown_connections 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56512377dda0 0x565123c1c6d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651237ea070 0x565123ba33a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651237c9390 0x565123ba2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f75ffb640 0 client.16566 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 --2- 172.21.15.83:0/919756785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651237c11b0 0x565123ba5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 >> 172.21.15.83:0/919756785 conn(0x5651237de460 msgr2=0x565123b950f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 shutdown_connections 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 1 -- 172.21.15.83:0/919756785 wait complete. 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f9ac6f740 20 client.16566 trim_cache size 0 max 16384 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 msgr2=0x7f3f780b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 0x7f3f780b8c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f8801a6d0 tx=0x7f3f8803b950 comp rx=0 tx=0).stop 2024-09-21T03:23:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 msgr2=0x7f3f8c1186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.969+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b450 tx=0x7f3f7c00b910 comp rx=0 tx=0).stop 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 shutdown_connections 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f780b6730 0x7f3f780b8c40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c11cba0 0x7f3f8c118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f8c111cd0 0x7f3f8c1186d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 --2- 172.21.15.83:0/3832019270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f8c10dbf0 0x7f3f8c11c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 >> 172.21.15.83:0/3832019270 conn(0x7f3f8c06e9f0 msgr2=0x7f3f8c10a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 shutdown_connections 2024-09-21T03:23:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:23.970+0000 7f3f93499640 1 -- 172.21.15.83:0/3832019270 wait complete. 2024-09-21T03:23:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:24 smithi193 ceph-mon[35224]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-09-21T03:23:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:24 smithi083 ceph-mon[31219]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-09-21T03:23:25.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:24 smithi142 ceph-mon[34512]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-09-21T03:23:25.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:23:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 365.0 s, 2447.2 tps, lat 40.890 ms stddev 45.513 2024-09-21T03:23:26.107 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:26.231 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:25 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:25 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:26.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:25 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.661+0000 7fcddfea8640 1 -- 172.21.15.83:0/3891640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 msgr2=0x7fcdd810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.661+0000 7fcddfea8640 1 --2- 172.21.15.83:0/3891640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd810f520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ca0 tx=0x7fcdd002a8d0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.662+0000 7fcddfea8640 1 -- 172.21.15.83:0/3891640062 shutdown_connections 2024-09-21T03:23:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.662+0000 7fcddfea8640 1 --2- 172.21.15.83:0/3891640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd810f520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.662+0000 7fcddfea8640 1 --2- 172.21.15.83:0/3891640062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.662+0000 7fcddfea8640 1 --2- 172.21.15.83:0/3891640062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 0x7fcdd8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.662+0000 7fcddfea8640 1 -- 172.21.15.83:0/3891640062 >> 172.21.15.83:0/3891640062 conn(0x7fcdd8079730 msgr2=0x7fcdd8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.664+0000 7fcddfea8640 1 -- 172.21.15.83:0/3891640062 shutdown_connections 2024-09-21T03:23:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.664+0000 7fcddfea8640 1 -- 172.21.15.83:0/3891640062 wait complete. 2024-09-21T03:23:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 Processor -- start 2024-09-21T03:23:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 -- start start 2024-09-21T03:23:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 0x7fcdd81141b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd8112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd806b1f0 con 0x7fcdd8108cf0 2024-09-21T03:23:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd806b070 con 0x7fcdd81083a0 2024-09-21T03:23:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.665+0000 7fcddfea8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd806b370 con 0x7fcdd8102a60 2024-09-21T03:23:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.666+0000 7fcddd443640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.666+0000 7fcddd443640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36974/0 (socket says 172.21.15.83:36974) 2024-09-21T03:23:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.666+0000 7fcddd443640 1 -- 172.21.15.83:0/1867809361 learned_addr learned my addr 172.21.15.83:0/1867809361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.668+0000 7fcdddc44640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 0x7fcdd81141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.668+0000 7fcdde445640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd8112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.670+0000 7fcdde445640 1 -- 172.21.15.83:0/1867809361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 msgr2=0x7fcdd81141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.670+0000 7fcdde445640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 0x7fcdd81141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.670+0000 7fcdde445640 1 -- 172.21.15.83:0/1867809361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 msgr2=0x7fcdd81127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.670+0000 7fcdde445640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.670+0000 7fcdde445640 1 -- 172.21.15.83:0/1867809361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd8113500 con 0x7fcdd8108cf0 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.671+0000 7fcddd443640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.673+0000 7fcdde445640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd8112ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcdd002adf0 tx=0x7fcdd0031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.673+0000 7fcdceffd640 1 -- 172.21.15.83:0/1867809361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0003db0 con 0x7fcdd8108cf0 2024-09-21T03:23:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.674+0000 7fcdceffd640 1 -- 172.21.15.83:0/1867809361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd00044f0 con 0x7fcdd8108cf0 2024-09-21T03:23:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.674+0000 7fcdceffd640 1 -- 172.21.15.83:0/1867809361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd002fd00 con 0x7fcdd8108cf0 2024-09-21T03:23:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.674+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd819e5e0 con 0x7fcdd8108cf0 2024-09-21T03:23:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.675+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd819ead0 con 0x7fcdd8108cf0 2024-09-21T03:23:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.684+0000 7fcde641c740 1 Processor -- start 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.685+0000 7fcde641c740 1 -- start start 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.685+0000 7fcde641c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f1226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.685+0000 7fcde641c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.685+0000 7fcde641c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed5a390 0x55d34f123120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.685+0000 7fcde641c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d34f098ea0 con 0x55d34ed7b070 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcde641c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34f0882d0 con 0x55d34ed5a390 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcde641c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d34f08c0e0 con 0x55d34ed521b0 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcdceffd640 1 -- 172.21.15.83:0/1867809361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcdd0004040 con 0x7fcdd8108cf0 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:34404/0 (socket says 172.21.15.83:34404) 2024-09-21T03:23:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.686+0000 7fcdddc44640 1 -- 172.21.15.83:0/1112624135 learned_addr learned my addr 172.21.15.83:0/1112624135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:26.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 -- 172.21.15.83:0/1112624135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed5a390 msgr2=0x55d34f123120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed5a390 0x55d34f123120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 -- 172.21.15.83:0/1112624135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 msgr2=0x55d34f1226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f1226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 -- 172.21.15.83:0/1112624135 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34ede6650 con 0x55d34ed521b0 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.687+0000 7fcdddc44640 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400cbb0 tx=0x7fcdd400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=705033d781b7fe3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.688+0000 7fcdceffd640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 0x7fcdc40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.688+0000 7fcdceffd640 1 -- 172.21.15.83:0/1867809361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fcdd0108d50 con 0x7fcdd8108cf0 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.688+0000 7fcdc3fff640 1 -- 172.21.15.83:0/1112624135 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4005220 con 0x55d34ed521b0 2024-09-21T03:23:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.688+0000 7fcdc3fff640 1 -- 172.21.15.83:0/1112624135 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4004c50 con 0x55d34ed521b0 2024-09-21T03:23:26.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.689+0000 7fcdc3fff640 1 -- 172.21.15.83:0/1112624135 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4010720 con 0x55d34ed521b0 2024-09-21T03:23:26.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.689+0000 7fcde641c740 1 -- 172.21.15.83:0/1112624135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 msgr2=0x55d34f122be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.689+0000 7fcde641c740 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400cbb0 tx=0x7fcdd400e720 comp rx=0 tx=0).stop 2024-09-21T03:23:26.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcddd443640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 0x7fcdc40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcddd443640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 0x7fcdc40b8d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8006e70 tx=0x7fcdc8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcde641c740 1 -- 172.21.15.83:0/1112624135 shutdown_connections 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcde641c740 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed5a390 0x55d34f123120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcde641c740 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed521b0 0x55d34f122be0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcde641c740 1 --2- 172.21.15.83:0/1112624135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f1226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.690+0000 7fcde641c740 1 -- 172.21.15.83:0/1112624135 >> 172.21.15.83:0/1112624135 conn(0x55d34ed6f460 msgr2=0x55d34f120590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.691+0000 7fcde641c740 1 -- 172.21.15.83:0/1112624135 shutdown_connections 2024-09-21T03:23:26.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.691+0000 7fcde641c740 1 -- 172.21.15.83:0/1112624135 wait complete. 2024-09-21T03:23:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.691+0000 7fcde641c740 1 -- start start 2024-09-21T03:23:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.691+0000 7fcde641c740 1 Processor -- start 2024-09-21T03:23:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f12dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d34eee9700 con 0x55d34ed7b070 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34f023160 con 0x55d34ed521b0 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d34f130ed0 con 0x55d34ed5a390 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcde641c740 10 client.0 _ll_register_callbacks cb 0x55d34f03c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcdde445640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcdde445640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43458/0 (socket says 172.21.15.83:43458) 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.692+0000 7fcdde445640 1 -- 172.21.15.83:0/2642733186 learned_addr learned my addr 172.21.15.83:0/2642733186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcddd443640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f12dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 -- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 msgr2=0x55d34f12e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 -- 172.21.15.83:0/2642733186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 msgr2=0x55d34f12dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f12dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 -- 172.21.15.83:0/2642733186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34f050c90 con 0x55d34ed7b070 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f12dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdde445640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce890 tx=0x7fcdd00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdea6473d746e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.693+0000 7fcdc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00046a0 con 0x55d34ed7b070 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fcdd0106de0 con 0x55d34ed7b070 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd4004c50 con 0x55d34ed7b070 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcdd00ca300 con 0x55d34ed7b070 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcde641c740 10 client.16575 fetch_fsmap learned FSMap version 43 2024-09-21T03:23:26.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.694+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d34f13cbc0 con 0x55d34ed7b070 2024-09-21T03:23:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.695+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4004c50 con 0x55d34ed7b070 2024-09-21T03:23:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fcdd00f0ba0 con 0x55d34ed7b070 2024-09-21T03:23:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 10 client.16575 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T03:23:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 10 client.16575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:23:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 20 client.16575 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed399f0 0x55d34f140110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 4 client.16575 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.696+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34f09adf0 con 0x55d34ed399f0 2024-09-21T03:23:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.697+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed399f0 0x55d34f140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.698+0000 7fcdc17fa640 10 client.16575 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:23:26.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.698+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed399f0 0x55d34f140110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc85bcfaee60e6b7 server_cookie=f4a077abd3a368c8 in_seq=0 out_seq=0 2024-09-21T03:23:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.699+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2642733186 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d34f09adf0 con 0x55d34ed399f0 2024-09-21T03:23:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.699+0000 7fcdc17fa640 10 client.16575 handle_command_reply: tid=0 2024-09-21T03:23:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.699+0000 7fcde641c740 1 client.16575 shutdown 2024-09-21T03:23:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.699+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 msgr2=0x55d34f12e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.699+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce890 tx=0x7fcdd00095c0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 shutdown_connections 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed399f0 0x55d34f140110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed7b070 0x55d34f12e760 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 --2- 172.21.15.83:0/2642733186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f12dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 >> 172.21.15.83:0/2642733186 conn(0x55d34ed6f460 msgr2=0x55d34f125880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcdc17fa640 0 client.16575 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 shutdown_connections 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 1 -- 172.21.15.83:0/2642733186 wait complete. 2024-09-21T03:23:26.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.700+0000 7fcde641c740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T03:23:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.722+0000 7fcde641c740 1 Processor -- start 2024-09-21T03:23:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.722+0000 7fcde641c740 1 -- start start 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.722+0000 7fcde641c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 0x55d34f1241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.722+0000 7fcde641c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.722+0000 7fcde641c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 0x55d34f125eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcde641c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81be3c0 con 0x55d34ed521b0 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcde641c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81bff60 con 0x55d34ed7b070 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcde641c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81bea90 con 0x55d34ed5a390 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 0x55d34f1241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 0x55d34f1241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43472/0 (socket says 172.21.15.83:43472) 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdddc44640 1 -- 172.21.15.83:0/2133116335 learned_addr learned my addr 172.21.15.83:0/2133116335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 0x55d34f125eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcddd443640 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 -- 172.21.15.83:0/2133116335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 msgr2=0x55d34f12e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 -- 172.21.15.83:0/2133116335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 msgr2=0x55d34f1241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 0x55d34f1241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 -- 172.21.15.83:0/2133116335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34f130d60 con 0x55d34ed7b070 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.723+0000 7fcdde445640 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 0x55d34f125eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00094e0 tx=0x7fcdd002a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78684455291d44f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.724+0000 7fcdc37fe640 1 -- 172.21.15.83:0/2133116335 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00046a0 con 0x55d34ed7b070 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.724+0000 7fcdc37fe640 1 -- 172.21.15.83:0/2133116335 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd0036030 con 0x55d34ed7b070 2024-09-21T03:23:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.724+0000 7fcdc37fe640 1 -- 172.21.15.83:0/2133116335 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00ce4e0 con 0x55d34ed7b070 2024-09-21T03:23:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 -- 172.21.15.83:0/2133116335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 msgr2=0x55d34f125eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 0x55d34f125eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00094e0 tx=0x7fcdd002a8d0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 -- 172.21.15.83:0/2133116335 shutdown_connections 2024-09-21T03:23:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed7b070 0x55d34f125eb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed5a390 0x55d34f12e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 --2- 172.21.15.83:0/2133116335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed521b0 0x55d34f1241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.725+0000 7fcde641c740 1 -- 172.21.15.83:0/2133116335 >> 172.21.15.83:0/2133116335 conn(0x55d34ed6f460 msgr2=0x55d34f1257a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.726+0000 7fcde641c740 1 -- 172.21.15.83:0/2133116335 shutdown_connections 2024-09-21T03:23:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.726+0000 7fcde641c740 1 -- 172.21.15.83:0/2133116335 wait complete. 2024-09-21T03:23:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.726+0000 7fcde641c740 1 -- start start 2024-09-21T03:23:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.727+0000 7fcde641c740 1 Processor -- start 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.727+0000 7fcde641c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f132370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 0x55d34f1328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed7b070 0x55d34f132df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81bf890 con 0x55d34ed5a390 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f132370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81bf160 con 0x55d34ed521b0 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcdddc44640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f132370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:37030/0 (socket says 172.21.15.83:37030) 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8113aa0 con 0x55d34ed7b070 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcdddc44640 1 -- 172.21.15.83:0/2835424607 learned_addr learned my addr 172.21.15.83:0/2835424607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 0x55d34f1328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcde641c740 10 client.0 _ll_register_callbacks cb 0x7fcdd0009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcdde445640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed7b070 0x55d34f132df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 -- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed7b070 msgr2=0x55d34f132df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed7b070 0x55d34f132df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 -- 172.21.15.83:0/2835424607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 msgr2=0x55d34f132370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f132370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 -- 172.21.15.83:0/2835424607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34f1a9cc0 con 0x55d34ed5a390 2024-09-21T03:23:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcddd443640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 0x55d34f1328b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80047b0 tx=0x7fcdc8006e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845ef5d55b4161e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.728+0000 7fcdc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8026020 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fcdc800f040 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd0036030 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcdc8031050 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcde641c740 10 client.16581 fetch_fsmap learned FSMap version 43 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.729+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d34f1aafd0 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd0036030 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fcdc8013070 con 0x55d34ed5a390 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 10 client.16581 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T03:23:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 20 client.16581 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed0eda0 0x55d34f1ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 4 client.16581 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d34f0968a0 con 0x55d34ed0eda0 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.730+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed0eda0 0x55d34f1ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.731+0000 7fcdc17fa640 10 client.16581 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.731+0000 7fcdddc44640 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed0eda0 0x55d34f1ad8a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2b6acba0334d674 server_cookie=c52a98f74a99f721 in_seq=0 out_seq=0 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.732+0000 7fcdc17fa640 1 -- 172.21.15.83:0/2835424607 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d34f0968a0 con 0x55d34ed0eda0 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.732+0000 7fcdc17fa640 10 client.16581 handle_command_reply: tid=0 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.732+0000 7fcde641c740 1 client.16581 shutdown 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.732+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 msgr2=0x55d34f1328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.732+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 0x55d34f1328b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80047b0 tx=0x7fcdc8006e00 comp rx=0 tx=0).stop 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 shutdown_connections 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d34ed0eda0 0x55d34f1ad8a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d34ed7b070 0x55d34f132df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d34ed5a390 0x55d34f1328b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcdc17fa640 0 client.16581 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 --2- 172.21.15.83:0/2835424607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34ed521b0 0x55d34f132370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 >> 172.21.15.83:0/2835424607 conn(0x55d34ed6f460 msgr2=0x55d34f12e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 shutdown_connections 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 1 -- 172.21.15.83:0/2835424607 wait complete. 2024-09-21T03:23:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.733+0000 7fcde641c740 20 client.16581 trim_cache size 0 max 16384 2024-09-21T03:23:26.742 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 msgr2=0x7fcdc40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 0x7fcdc40b8d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8006e70 tx=0x7fcdc8008040 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 msgr2=0x7fcdd8112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd8112ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcdd002adf0 tx=0x7fcdd0031a00 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 shutdown_connections 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcdc40b6850 0x7fcdc40b8d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdd8108cf0 0x7fcdd8112ce0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdd81083a0 0x7fcdd81127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 --2- 172.21.15.83:0/1867809361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdd8102a60 0x7fcdd81141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.734+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 >> 172.21.15.83:0/1867809361 conn(0x7fcdd8079730 msgr2=0x7fcdd8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.735+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 shutdown_connections 2024-09-21T03:23:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:26.735+0000 7fcddfea8640 1 -- 172.21.15.83:0/1867809361 wait complete. 2024-09-21T03:23:27.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:26 smithi083 ceph-mon[31219]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-21T03:23:27.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:26 smithi193 ceph-mon[35224]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-21T03:23:27.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:26 smithi142 ceph-mon[34512]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-09-21T03:23:28.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:23:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:29.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:28 smithi083 ceph-mon[31219]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3020190361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810f150 msgr2=0x7f1cb810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3020190361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810f150 0x7f1cb810f560 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800d910 tx=0x7f1ca802e570 comp rx=0 tx=0).stop 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3020190361 shutdown_connections 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3020190361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb810ad10 0x7f1cb8116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3020190361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb810a3c0 0x7f1cb810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3020190361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810f150 0x7f1cb810f560 secure :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800d910 tx=0x7f1ca802e570 comp rx=0 tx=0).stop 2024-09-21T03:23:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.269+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3020190361 >> 172.21.15.83:0/3020190361 conn(0x7f1cb80712f0 msgr2=0x7f1cb8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.273+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3020190361 shutdown_connections 2024-09-21T03:23:29.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.274+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3020190361 wait complete. 2024-09-21T03:23:29.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.275+0000 7f1cbcff4640 1 Processor -- start 2024-09-21T03:23:29.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.275+0000 7f1cbcff4640 1 -- start start 2024-09-21T03:23:29.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb810a3c0 0x7f1cb81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810ad10 0x7f1cb81a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb811a170 con 0x7f1cb810ad10 2024-09-21T03:23:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8119ff0 con 0x7f1cb810a3c0 2024-09-21T03:23:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.276+0000 7f1cbcff4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb811a2f0 con 0x7f1cb81a63d0 2024-09-21T03:23:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.277+0000 7f1cb6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.277+0000 7f1cb6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53776/0 (socket says 172.21.15.83:53776) 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.277+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/3500593218 learned_addr learned my addr 172.21.15.83:0/3500593218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.278+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/3500593218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb810a3c0 msgr2=0x7f1cb81a0a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.278+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb810a3c0 0x7f1cb81a0a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.278+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810ad10 msgr2=0x7f1cb81a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.278+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810ad10 0x7f1cb81a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.278+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/3500593218 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb81a6d20 con 0x7f1cb81a63d0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b970 tx=0x7f1cb000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1ca77fe640 1 -- 172.21.15.83:0/3500593218 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0019070 con 0x7f1cb81a63d0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1ca77fe640 1 -- 172.21.15.83:0/3500593218 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb0014440 con 0x7f1cb81a63d0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb81a7010 con 0x7f1cb81a63d0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81adaa0 con 0x7f1cb81a63d0 2024-09-21T03:23:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.279+0000 7f1ca77fe640 1 -- 172.21.15.83:0/3500593218 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00075d0 con 0x7f1cb81a63d0 2024-09-21T03:23:29.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.283+0000 7f1ca77fe640 1 -- 172.21.15.83:0/3500593218 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1cb001c070 con 0x7f1cb81a63d0 2024-09-21T03:23:29.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.285+0000 7f1ca77fe640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 0x7f1ca00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.285+0000 7f1cb659c640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 0x7f1ca00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.285+0000 7f1ca77fe640 1 -- 172.21.15.83:0/3500593218 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f1cb00e7ad0 con 0x7f1cb81a63d0 2024-09-21T03:23:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.286+0000 7f1cb659c640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 0x7f1ca00b8d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009230 tx=0x7f1ca8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.286+0000 7f1cc3568740 1 Processor -- start 2024-09-21T03:23:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.286+0000 7f1cc3568740 1 -- start start 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ce070 0x560d6ad756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad75be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d6acebea0 con 0x560d6a9a51b0 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d6acdb2d0 con 0x560d6a9ce070 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cc3568740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d6acdf0e0 con 0x560d6a9ad390 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cb6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cb6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53778/0 (socket says 172.21.15.83:53778) 2024-09-21T03:23:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/4110346183 learned_addr learned my addr 172.21.15.83:0/4110346183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.287+0000 7f1cb659c640 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ce070 0x560d6ad756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.288+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/4110346183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ce070 msgr2=0x560d6ad756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.288+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ce070 0x560d6ad756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.288+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/4110346183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 msgr2=0x560d6ad75be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.288+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad75be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.288+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/4110346183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6aa39650 con 0x560d6a9ad390 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.289+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00a9f40 tx=0x7f1cb0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9852084b81f7701c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.290+0000 7f1ca4ff9640 1 -- 172.21.15.83:0/4110346183 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00ae070 con 0x560d6a9ad390 2024-09-21T03:23:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.290+0000 7f1ca4ff9640 1 -- 172.21.15.83:0/4110346183 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00b2450 con 0x560d6a9ad390 2024-09-21T03:23:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.290+0000 7f1ca4ff9640 1 -- 172.21.15.83:0/4110346183 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00acbd0 con 0x560d6a9ad390 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 -- 172.21.15.83:0/4110346183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 msgr2=0x560d6ad76120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00a9f40 tx=0x7f1cb0004a40 comp rx=0 tx=0).stop 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 -- 172.21.15.83:0/4110346183 shutdown_connections 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad76120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad75be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4110346183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ce070 0x560d6ad756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 -- 172.21.15.83:0/4110346183 >> 172.21.15.83:0/4110346183 conn(0x560d6a9c2460 msgr2=0x560d6ad73590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 -- 172.21.15.83:0/4110346183 shutdown_connections 2024-09-21T03:23:29.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.291+0000 7f1cc3568740 1 -- 172.21.15.83:0/4110346183 wait complete. 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 -- start start 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 Processor -- start 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 0x560d6ad80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 0x560d6ad81720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d6ab3c700 con 0x560d6a9ce070 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d6ac76160 con 0x560d6a9a51b0 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.292+0000 7f1cc3568740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d6ad83e80 con 0x560d6a9ad390 2024-09-21T03:23:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cc3568740 10 client.0 _ll_register_callbacks cb 0x560d6ac8f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53792/0 (socket says 172.21.15.83:53792) 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 -- 172.21.15.83:0/4019179163 learned_addr learned my addr 172.21.15.83:0/4019179163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 0x560d6ad80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 0x560d6ad81720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 -- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 msgr2=0x560d6ad80ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 0x560d6ad80ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 -- 172.21.15.83:0/4019179163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 msgr2=0x560d6ad81720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 0x560d6ad81720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 -- 172.21.15.83:0/4019179163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6aca3c90 con 0x560d6a9ad390 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 0x560d6ad81720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb659c640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 0x560d6ad80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e6a0 tx=0x7f1cac00eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaecb25707b0ceaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.293+0000 7f1c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:23:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00cc40 con 0x560d6a9ad390 2024-09-21T03:23:29.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00b2450 con 0x560d6a9ad390 2024-09-21T03:23:29.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f1cac00cde0 con 0x560d6a9ad390 2024-09-21T03:23:29.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1cac017440 con 0x560d6a9ad390 2024-09-21T03:23:29.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1cc3568740 10 client.26116 fetch_fsmap learned FSMap version 43 2024-09-21T03:23:29.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560d6ad8fb90 con 0x560d6a9ad390 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.294+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00b2450 con 0x560d6a9ad390 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f1cac015070 con 0x560d6a9ad390 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 10 client.26116 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 20 client.26116 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a98c9f0 0x560d6ad930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 4 client.26116 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.295+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d6aceddf0 con 0x560d6a98c9f0 2024-09-21T03:23:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.296+0000 7f1cb659c640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a98c9f0 0x560d6ad930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.296+0000 7f1c9dffb640 10 client.26116 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.296+0000 7f1cb659c640 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a98c9f0 0x560d6ad930f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6767c80002b9f8 server_cookie=1f589981662a05bd in_seq=0 out_seq=0 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1c9dffb640 1 -- 172.21.15.83:0/4019179163 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d6aceddf0 con 0x560d6a98c9f0 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1c9dffb640 10 client.26116 handle_command_reply: tid=0 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 client.26116 shutdown 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 msgr2=0x560d6ad811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e6a0 tx=0x7f1cac00eb60 comp rx=0 tx=0).stop 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 shutdown_connections 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a98c9f0 0x560d6ad930f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1c9dffb640 0 client.26116 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9ce070 0x560d6ad81720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad811e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 --2- 172.21.15.83:0/4019179163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9a51b0 0x560d6ad80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 >> 172.21.15.83:0/4019179163 conn(0x560d6a9c2460 msgr2=0x560d6ad78830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.297+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 shutdown_connections 2024-09-21T03:23:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.298+0000 7f1cc3568740 1 -- 172.21.15.83:0/4019179163 wait complete. 2024-09-21T03:23:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.298+0000 7f1cc3568740 20 client.26116 trim_cache size 0 max 16384 2024-09-21T03:23:29.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:28 smithi193 ceph-mon[35224]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-09-21T03:23:29.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.317+0000 7f1cc3568740 1 Processor -- start 2024-09-21T03:23:29.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.317+0000 7f1cc3568740 1 -- start start 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 0x560d6ad7de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9a51b0 0x560d6ad7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 0x560d6ad766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800e6b0 con 0x560d6a98c9f0 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98010250 con 0x560d6a9ad390 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cc3568740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800ed80 con 0x560d6a9a51b0 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.318+0000 7f1cb659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 0x560d6ad7de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 0x560d6ad7de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45542/0 (socket says 172.21.15.83:45542) 2024-09-21T03:23:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb659c640 1 -- 172.21.15.83:0/276900230 learned_addr learned my addr 172.21.15.83:0/276900230 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 0x560d6ad766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/276900230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9a51b0 msgr2=0x560d6ad7e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:29.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9a51b0 0x560d6ad7e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/276900230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 msgr2=0x560d6ad7de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 0x560d6ad7de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.319+0000 7f1cb6d9d640 1 -- 172.21.15.83:0/276900230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6ad837d0 con 0x560d6a9ad390 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.320+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 0x560d6ad766d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009940 tx=0x7f1cb0014d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925dddb6d7991d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.320+0000 7f1c9ffff640 1 -- 172.21.15.83:0/276900230 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b2450 con 0x560d6a9ad390 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.320+0000 7f1c9ffff640 1 -- 172.21.15.83:0/276900230 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00ae070 con 0x560d6a9ad390 2024-09-21T03:23:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.320+0000 7f1c9ffff640 1 -- 172.21.15.83:0/276900230 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00ace30 con 0x560d6a9ad390 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.321+0000 7f1cc3568740 1 -- 172.21.15.83:0/276900230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 msgr2=0x560d6ad766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.321+0000 7f1cc3568740 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 0x560d6ad766d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009940 tx=0x7f1cb0014d70 comp rx=0 tx=0).stop 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 -- 172.21.15.83:0/276900230 shutdown_connections 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a9ad390 0x560d6ad766d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9a51b0 0x560d6ad7e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 --2- 172.21.15.83:0/276900230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a98c9f0 0x560d6ad7de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 -- 172.21.15.83:0/276900230 >> 172.21.15.83:0/276900230 conn(0x560d6a9c2460 msgr2=0x560d6ad7f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 -- 172.21.15.83:0/276900230 shutdown_connections 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 -- 172.21.15.83:0/276900230 wait complete. 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 -- start start 2024-09-21T03:23:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.322+0000 7f1cc3568740 1 Processor -- start 2024-09-21T03:23:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 0x560d6ad89920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800fb80 con 0x560d6a9a51b0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800f450 con 0x560d6a98c9f0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800c6b0 con 0x560d6a9ad390 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cc3568740 10 client.0 _ll_register_callbacks cb 0x7f1cac0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45550/0 (socket says 172.21.15.83:45550) 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 0x560d6ad89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb5d9b640 1 -- 172.21.15.83:0/1513075979 learned_addr learned my addr 172.21.15.83:0/1513075979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb6d9d640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 -- 172.21.15.83:0/1513075979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 msgr2=0x560d6ad85310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad85310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 -- 172.21.15.83:0/1513075979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 msgr2=0x560d6ad84dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb659c640 1 -- 172.21.15.83:0/1513075979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6ad85aa0 con 0x560d6a98c9f0 2024-09-21T03:23:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.323+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.325+0000 7f1cb659c640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 0x560d6ad89920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1cb806d980 tx=0x7f1ca8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0904e2e41e5ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.325+0000 7f1c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.325+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca805d060 con 0x560d6a98c9f0 2024-09-21T03:23:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.326+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f1ca800d4d0 con 0x560d6a98c9f0 2024-09-21T03:23:29.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.326+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00ae070 con 0x560d6a98c9f0 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ca80046c0 con 0x560d6a98c9f0 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 10 client.26144 fetch_fsmap learned FSMap version 43 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560d6adfda90 con 0x560d6a98c9f0 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb00ae070 con 0x560d6a98c9f0 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f1ca80035f0 con 0x560d6a98c9f0 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 10 client.26144 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 20 client.26144 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a9ce070 0x560d6ae001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 4 client.26144 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:23:29.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.327+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d6ace98a0 con 0x560d6a9ce070 2024-09-21T03:23:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.328+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a9ce070 0x560d6ae001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.328+0000 7f1c9dffb640 10 client.26144 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.328+0000 7f1cb5d9b640 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a9ce070 0x560d6ae001e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05bc353cf7de8e8 server_cookie=2cdae551fa9ff103 in_seq=0 out_seq=0 2024-09-21T03:23:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.329+0000 7f1c9dffb640 1 -- 172.21.15.83:0/1513075979 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d6ace98a0 con 0x560d6a9ce070 2024-09-21T03:23:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.329+0000 7f1c9dffb640 10 client.26144 handle_command_reply: tid=0 2024-09-21T03:23:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 client.26144 shutdown 2024-09-21T03:23:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 msgr2=0x560d6ad89920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 0x560d6ad89920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1cb806d980 tx=0x7f1ca8009260 comp rx=0 tx=0).stop 2024-09-21T03:23:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 shutdown_connections 2024-09-21T03:23:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d6a9ce070 0x560d6ae001e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d6a9ad390 0x560d6ad85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d6a9a51b0 0x560d6ad84dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 --2- 172.21.15.83:0/1513075979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d6a98c9f0 0x560d6ad89920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 >> 172.21.15.83:0/1513075979 conn(0x560d6a9c2460 msgr2=0x560d6ad8c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1c9dffb640 0 client.26144 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 shutdown_connections 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 1 -- 172.21.15.83:0/1513075979 wait complete. 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.330+0000 7f1cc3568740 20 client.26144 trim_cache size 0 max 16384 2024-09-21T03:23:29.333 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:23:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.331+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 msgr2=0x7f1ca00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 0x7f1ca00b8d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009230 tx=0x7f1ca8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 msgr2=0x7f1cb81a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b970 tx=0x7f1cb000be30 comp rx=0 tx=0).stop 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 shutdown_connections 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f1ca00b6800 0x7f1ca00b8d10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cb81a63d0 0x7f1cb81a67e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810ad10 0x7f1cb81a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 --2- 172.21.15.83:0/3500593218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cb810a3c0 0x7f1cb81a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 >> 172.21.15.83:0/3500593218 conn(0x7f1cb80712f0 msgr2=0x7f1cb810e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 shutdown_connections 2024-09-21T03:23:29.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:29.332+0000 7f1cbcff4640 1 -- 172.21.15.83:0/3500593218 wait complete. 2024-09-21T03:23:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:28 smithi142 ceph-mon[34512]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-09-21T03:23:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 370.0 s, 2290.2 tps, lat 43.610 ms stddev 51.509 2024-09-21T03:23:31.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:30 smithi193 ceph-mon[35224]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:23:31.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:30 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:30 smithi083 ceph-mon[31219]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:23:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:30 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:31.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:23:31.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:23:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:30 smithi142 ceph-mon[34512]: pgmap v532: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:23:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:30 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:23:31.791 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:32.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 -- 172.21.15.83:0/2171397834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1404116640 msgr2=0x7f1404118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 --2- 172.21.15.83:0/2171397834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1404116640 0x7f1404118b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009ca0 tx=0x7f13fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 -- 172.21.15.83:0/2171397834 shutdown_connections 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 --2- 172.21.15.83:0/2171397834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1404116640 0x7f1404118b50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 --2- 172.21.15.83:0/2171397834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410aee0 0x7f1404115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 --2- 172.21.15.83:0/2171397834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140410a590 0x7f140410a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 -- 172.21.15.83:0/2171397834 >> 172.21.15.83:0/2171397834 conn(0x7f1404070ec0 msgr2=0x7f14040712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 -- 172.21.15.83:0/2171397834 shutdown_connections 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.328+0000 7f1408a5e640 1 -- 172.21.15.83:0/2171397834 wait complete. 2024-09-21T03:23:32.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 Processor -- start 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 -- start start 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 0x7f14041a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140410aee0 0x7f14041a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 0x7f14041a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f140411c310 con 0x7f140410aee0 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f140411c190 con 0x7f140410a590 2024-09-21T03:23:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1408a5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f140411c490 con 0x7f1404116640 2024-09-21T03:23:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.330+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140410aee0 0x7f14041a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.331+0000 7f140259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 0x7f14041a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.331+0000 7f140259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 0x7f14041a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36244/0 (socket says 172.21.15.83:36244) 2024-09-21T03:23:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.331+0000 7f140259c640 1 -- 172.21.15.83:0/3043229269 learned_addr learned my addr 172.21.15.83:0/3043229269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.331+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 0x7f14041a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 -- 172.21.15.83:0/3043229269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 msgr2=0x7f14041a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 0x7f14041a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 -- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140410aee0 msgr2=0x7f14041a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140410aee0 0x7f14041a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 -- 172.21.15.83:0/3043229269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041a8bd0 con 0x7f1404116640 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.332+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 0x7f14041a84a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005e30 tx=0x7f13fc0317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.333+0000 7f13f37fe640 1 -- 172.21.15.83:0/3043229269 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003b70 con 0x7f1404116640 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.333+0000 7f13f37fe640 1 -- 172.21.15.83:0/3043229269 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13fc003d10 con 0x7f1404116640 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.333+0000 7f13f37fe640 1 -- 172.21.15.83:0/3043229269 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc02f9e0 con 0x7f1404116640 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.333+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041a8e60 con 0x7f1404116640 2024-09-21T03:23:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.333+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14041af620 con 0x7f1404116640 2024-09-21T03:23:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.337+0000 7f140efd2740 1 Processor -- start 2024-09-21T03:23:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.337+0000 7f13f37fe640 1 -- 172.21.15.83:0/3043229269 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f13fc02fb80 con 0x7f1404116640 2024-09-21T03:23:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 -- start start 2024-09-21T03:23:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579281840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x5595792822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595791f3e60 con 0x559578ead490 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595791e3290 con 0x559578eb5670 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140efd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5595791e70a0 con 0x559578ed6350 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45576/0 (socket says 172.21.15.83:45576) 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.338+0000 7f140259c640 1 -- 172.21.15.83:0/4016869624 learned_addr learned my addr 172.21.15.83:0/4016869624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:32.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f13f37fe640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 0x7f13ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f1401d9b640 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579281840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f1401d9b640 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x5595792822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f140259c640 1 -- 172.21.15.83:0/4016869624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 msgr2=0x559579281840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f140259c640 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579281840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f140259c640 1 -- 172.21.15.83:0/4016869624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 msgr2=0x5595792822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f140259c640 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x5595792822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 0x7f13ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f140259c640 1 -- 172.21.15.83:0/4016869624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559578f41930 con 0x559578ead490 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.339+0000 7f13f37fe640 1 -- 172.21.15.83:0/3043229269 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f13fc108840 con 0x7f1404116640 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.340+0000 7f140259c640 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f13f800bde0 tx=0x7f13f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e437a99c753232 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.340+0000 7f1402d9d640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 0x7f13ec0b8c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f13fc1065e0 tx=0x7f13fc0c2f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.340+0000 7f13f0ff9640 1 -- 172.21.15.83:0/4016869624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8015070 con 0x559578ead490 2024-09-21T03:23:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.341+0000 7f13f0ff9640 1 -- 172.21.15.83:0/4016869624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f801a910 con 0x559578ead490 2024-09-21T03:23:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.341+0000 7f13f0ff9640 1 -- 172.21.15.83:0/4016869624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8011bf0 con 0x559578ead490 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 -- 172.21.15.83:0/4016869624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 msgr2=0x559579281d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f13f800bde0 tx=0x7f13f800f730 comp rx=0 tx=0).stop 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 -- 172.21.15.83:0/4016869624 shutdown_connections 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x5595792822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579281d80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 --2- 172.21.15.83:0/4016869624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579281840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.342+0000 7f140efd2740 1 -- 172.21.15.83:0/4016869624 >> 172.21.15.83:0/4016869624 conn(0x559578eca740 msgr2=0x55957927f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.343+0000 7f140efd2740 1 -- 172.21.15.83:0/4016869624 shutdown_connections 2024-09-21T03:23:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.343+0000 7f140efd2740 1 -- 172.21.15.83:0/4016869624 wait complete. 2024-09-21T03:23:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.344+0000 7f140efd2740 1 -- start start 2024-09-21T03:23:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.344+0000 7f140efd2740 1 Processor -- start 2024-09-21T03:23:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ead490 0x55957928d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x55957928d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559579044890 con 0x559578ed6350 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55957917dd40 con 0x559578eb5670 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559579290020 con 0x559578ead490 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140efd2740 10 client.0 _ll_register_callbacks cb 0x5595791e1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45592/0 (socket says 172.21.15.83:45592) 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.345+0000 7f140259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x55957928d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 -- 172.21.15.83:0/410482934 learned_addr learned my addr 172.21.15.83:0/410482934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ead490 0x55957928d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 -- 172.21.15.83:0/410482934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ead490 msgr2=0x55957928d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ead490 0x55957928d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 -- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 msgr2=0x55957928d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x55957928d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 -- 172.21.15.83:0/410482934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957927e150 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f1401d9b640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55957928fad0 tx=0x7f13f4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a50ae8804c86c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.346+0000 7f13e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f400ebe0 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f13f400ed80 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f801a910 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f13f4004030 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f140efd2740 10 client.16605 fetch_fsmap learned FSMap version 44 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5595792a0710 con 0x559578ed6350 2024-09-21T03:23:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f801a910 con 0x559578ed6350 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.347+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f13f402c480 con 0x559578ed6350 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 10 client.16605 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 20 client.16605 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e94cd0 0x5595792a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 4 client.16605 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595791f5db0 con 0x559578e94cd0 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.348+0000 7f140259c640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e94cd0 0x5595792a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.350+0000 7f140259c640 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e94cd0 0x5595792a3b30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94870ea68b8bd386 server_cookie=89df7f1038a24397 in_seq=0 out_seq=0 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.350+0000 7f13e9ffb640 10 client.16605 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.351+0000 7f13e9ffb640 1 -- 172.21.15.83:0/410482934 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595791f5db0 con 0x559578e94cd0 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.351+0000 7f13e9ffb640 10 client.16605 handle_command_reply: tid=0 2024-09-21T03:23:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.351+0000 7f140efd2740 1 client.16605 shutdown 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 msgr2=0x55957927db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55957928fad0 tx=0x7f13f4015040 comp rx=0 tx=0).stop 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 shutdown_connections 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e94cd0 0x5595792a3b30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ed6350 0x55957927db60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x55957928d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 --2- 172.21.15.83:0/410482934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ead490 0x55957928d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 >> 172.21.15.83:0/410482934 conn(0x559578eca740 msgr2=0x5595792849d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f13e9ffb640 0 client.16605 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 shutdown_connections 2024-09-21T03:23:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 1 -- 172.21.15.83:0/410482934 wait complete. 2024-09-21T03:23:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.352+0000 7f140efd2740 20 client.16605 trim_cache size 0 max 16384 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.384+0000 7f140efd2740 1 Processor -- start 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.384+0000 7f140efd2740 1 -- start start 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579286950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x559579286e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x55957927f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b5ce0 con 0x559578ead490 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b7820 con 0x559578eb5670 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140efd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b63b0 con 0x559578ed6350 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579286950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579286950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45600/0 (socket says 172.21.15.83:45600) 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f1401d9b640 1 -- 172.21.15.83:0/3865867121 learned_addr learned my addr 172.21.15.83:0/3865867121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f140259c640 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x559579286e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.385+0000 7f1401d9b640 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x55957927f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 -- 172.21.15.83:0/3865867121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 msgr2=0x55957927f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x55957927f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 -- 172.21.15.83:0/3865867121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 msgr2=0x559579286950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579286950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 -- 172.21.15.83:0/3865867121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559579283d60 con 0x559578eb5670 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.386+0000 7f140259c640 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x559579286e90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559579280670 tx=0x7f13f80116f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6b09bdbcedcc32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.388+0000 7f13ebfff640 1 -- 172.21.15.83:0/3865867121 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8015040 con 0x559578eb5670 2024-09-21T03:23:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.388+0000 7f13ebfff640 1 -- 172.21.15.83:0/3865867121 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f800ba00 con 0x559578eb5670 2024-09-21T03:23:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.388+0000 7f13ebfff640 1 -- 172.21.15.83:0/3865867121 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8011dd0 con 0x559578eb5670 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 -- 172.21.15.83:0/3865867121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 msgr2=0x559579286e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x559579286e90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559579280670 tx=0x7f13f80116f0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 -- 172.21.15.83:0/3865867121 shutdown_connections 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x55957927f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578eb5670 0x559579286e90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 --2- 172.21.15.83:0/3865867121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578ead490 0x559579286950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.389+0000 7f140efd2740 1 -- 172.21.15.83:0/3865867121 >> 172.21.15.83:0/3865867121 conn(0x559578eca740 msgr2=0x55957928bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.393+0000 7f140efd2740 1 -- 172.21.15.83:0/3865867121 shutdown_connections 2024-09-21T03:23:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.393+0000 7f140efd2740 1 -- 172.21.15.83:0/3865867121 wait complete. 2024-09-21T03:23:32.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.394+0000 7f140efd2740 1 -- start start 2024-09-21T03:23:32.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.394+0000 7f140efd2740 1 Processor -- start 2024-09-21T03:23:32.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 0x559579290260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 0x5595792907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579290ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b7150 con 0x559578eb5670 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b6a80 con 0x559578ead490 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14041b3ce0 con 0x559578ed6350 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.395+0000 7f140efd2740 10 client.0 _ll_register_callbacks cb 0x7f13f4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 0x559579290260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f1401d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 0x559579290260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36278/0 (socket says 172.21.15.83:36278) 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f1401d9b640 1 -- 172.21.15.83:0/1416060182 learned_addr learned my addr 172.21.15.83:0/1416060182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f1401d9b640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579290ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 0x5595792907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 -- 172.21.15.83:0/1416060182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 msgr2=0x559579290ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579290ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 -- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 msgr2=0x559579290260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:32.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 0x559579290260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.396+0000 7f140259c640 1 -- 172.21.15.83:0/1416060182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957929b490 con 0x559578eb5670 2024-09-21T03:23:32.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f140259c640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 0x5595792907a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182dd8ae73224865 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f13e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8015070 con 0x559578eb5670 2024-09-21T03:23:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f13f8004230 con 0x559578eb5670 2024-09-21T03:23:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f800ba00 con 0x559578eb5670 2024-09-21T03:23:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f13f8044050 con 0x559578eb5670 2024-09-21T03:23:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f140efd2740 10 client.16611 fetch_fsmap learned FSMap version 44 2024-09-21T03:23:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.397+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55957929c740 con 0x559578eb5670 2024-09-21T03:23:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f800b680 con 0x559578eb5670 2024-09-21T03:23:32.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f13f80047a0 con 0x559578eb5670 2024-09-21T03:23:32.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 10 client.16611 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:23:32.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:32.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 20 client.16611 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:32.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e6a080 0x55957929ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 4 client.16611 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:23:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f13f800ba00 con 0x559578e6a080 2024-09-21T03:23:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.398+0000 7f1401d9b640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e6a080 0x55957929ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.399+0000 7f13e9ffb640 10 client.16611 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.399+0000 7f1401d9b640 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e6a080 0x55957929ccc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed0ceb182246b52 server_cookie=d6f1c885db83f5b4 in_seq=0 out_seq=0 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.401+0000 7f13e9ffb640 1 -- 172.21.15.83:0/1416060182 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f13f800ba00 con 0x559578e6a080 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.401+0000 7f13e9ffb640 10 client.16611 handle_command_reply: tid=0 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.401+0000 7f140efd2740 1 client.16611 shutdown 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.401+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 msgr2=0x5595792907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.401+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 0x5595792907a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8005510 comp rx=0 tx=0).stop 2024-09-21T03:23:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 shutdown_connections 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559578e6a080 0x55957929ccc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559578ed6350 0x559579290ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f13e9ffb640 0 client.16611 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559578eb5670 0x5595792907a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 --2- 172.21.15.83:0/1416060182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559578ead490 0x559579290260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.403+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 >> 172.21.15.83:0/1416060182 conn(0x559578eca740 msgr2=0x55957927efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.404+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 shutdown_connections 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.404+0000 7f140efd2740 1 -- 172.21.15.83:0/1416060182 wait complete. 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.404+0000 7f140efd2740 20 client.16611 trim_cache size 0 max 16384 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:23:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:23:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc", 2024-09-21T03:23:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:23:32.411 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 msgr2=0x7f13ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 0x7f13ec0b8c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f13fc1065e0 tx=0x7f13fc0c2f40 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 msgr2=0x7f14041a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 0x7f14041a84a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005e30 tx=0x7f13fc0317c0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 shutdown_connections 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f13ec0b6700 0x7f13ec0b8c10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1404116640 0x7f14041a84a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f140410aee0 0x7f14041a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 --2- 172.21.15.83:0/3043229269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f140410a590 0x7f14041a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 >> 172.21.15.83:0/3043229269 conn(0x7f1404070ec0 msgr2=0x7f1404071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 shutdown_connections 2024-09-21T03:23:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:32.405+0000 7f1408a5e640 1 -- 172.21.15.83:0/3043229269 wait complete. 2024-09-21T03:23:33.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: pgmap v533: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-21T03:23:33.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:33.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: pgmap v533: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:33.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: pgmap v533: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:33.603 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:34.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:34 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:34 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:23:34.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:34.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:34.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:23:34.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:34 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:23:34.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 -- 172.21.15.83:0/131071656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 msgr2=0x7f86a0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/131071656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a0104cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8690009900 tx=0x7f869002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 -- 172.21.15.83:0/131071656 shutdown_connections 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/131071656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/131071656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 0x7f86a010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/131071656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a0104cd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 -- 172.21.15.83:0/131071656 >> 172.21.15.83:0/131071656 conn(0x7f86a0079750 msgr2=0x7f86a0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 -- 172.21.15.83:0/131071656 shutdown_connections 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.204+0000 7f86a4f7c640 1 -- 172.21.15.83:0/131071656 wait complete. 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 Processor -- start 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 -- start start 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 0x7f86a0102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0113b00 con 0x7f86a010de70 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0113980 con 0x7f86a01048c0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f86a4f7c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0113c80 con 0x7f86a01052f0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f869e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f869e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36302/0 (socket says 172.21.15.83:36302) 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f869e59c640 1 -- 172.21.15.83:0/2805608669 learned_addr learned my addr 172.21.15.83:0/2805608669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.206+0000 7f869dd9b640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 0x7f86a0102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 -- 172.21.15.83:0/2805608669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 msgr2=0x7f86a0102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 0x7f86a0102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 -- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 msgr2=0x7f86a0103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 -- 172.21.15.83:0/2805608669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a01a8770 con 0x7f86a01048c0 2024-09-21T03:23:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869ed9d640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.207+0000 7f869e59c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8690005ab0 tx=0x7f8690003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.208+0000 7f868f7fe640 1 -- 172.21.15.83:0/2805608669 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869003f070 con 0x7f86a01048c0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.208+0000 7f868f7fe640 1 -- 172.21.15.83:0/2805608669 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8690031b60 con 0x7f86a01048c0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.208+0000 7f868f7fe640 1 -- 172.21.15.83:0/2805608669 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690035030 con 0x7f86a01048c0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.208+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a01a8a00 con 0x7f86a01048c0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.208+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a01a8e90 con 0x7f86a01048c0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.211+0000 7f86ab4f0740 1 Processor -- start 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 -- start start 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a97390 0x55d884e5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d884dd5e50 con 0x55d884a8f1b0 2024-09-21T03:23:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d884dc5280 con 0x55d884ab8070 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d884dc9090 con 0x55d884a97390 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.212+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36316/0 (socket says 172.21.15.83:36316) 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 -- 172.21.15.83:0/3492043756 learned_addr learned my addr 172.21.15.83:0/3492043756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869dd9b640 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a97390 0x55d884e5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 -- 172.21.15.83:0/3492043756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a97390 msgr2=0x55d884e5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a97390 0x55d884e5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 -- 172.21.15.83:0/3492043756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 msgr2=0x55d884e5f7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e5f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 -- 172.21.15.83:0/3492043756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884b23650 con 0x55d884ab8070 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f869ed9d640 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8698011810 tx=0x7f8698011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8681fb7167b460 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f868cff9640 1 -- 172.21.15.83:0/3492043756 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698011310 con 0x55d884ab8070 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f868cff9640 1 -- 172.21.15.83:0/3492043756 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869800f9a0 con 0x55d884ab8070 2024-09-21T03:23:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.213+0000 7f868cff9640 1 -- 172.21.15.83:0/3492043756 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698005610 con 0x55d884ab8070 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.214+0000 7f868f7fe640 1 -- 172.21.15.83:0/2805608669 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8690042070 con 0x7f86a01048c0 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- 172.21.15.83:0/3492043756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 msgr2=0x55d884e5f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8698011810 tx=0x7f8698011cd0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- 172.21.15.83:0/3492043756 shutdown_connections 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a97390 0x55d884e5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/3492043756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e5f270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- 172.21.15.83:0/3492043756 >> 172.21.15.83:0/3492043756 conn(0x55d884aac460 msgr2=0x55d884e5d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- 172.21.15.83:0/3492043756 shutdown_connections 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- 172.21.15.83:0/3492043756 wait complete. 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f868f7fe640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 0x7f86880b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 -- start start 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f869dd9b640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 0x7f86880b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f868f7fe640 1 -- 172.21.15.83:0/2805608669 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f8690107d80 con 0x7f86a01048c0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.215+0000 7f86ab4f0740 1 Processor -- start 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884ab8070 0x55d884e70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d884c26700 con 0x55d884a8f1b0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d884d60110 con 0x55d884a97390 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.217+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d884e65920 con 0x55d884ab8070 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f86ab4f0740 10 client.0 _ll_register_callbacks cb 0x55d884d79990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869dd9b640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 0x7f86880b8bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86a0102490 tx=0x7f8694005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45626/0 (socket says 172.21.15.83:45626) 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869e59c640 1 -- 172.21.15.83:0/2276065202 learned_addr learned my addr 172.21.15.83:0/2276065202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.218+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e6b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36322/0 (socket says 172.21.15.83:36322) 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869ed9d640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884ab8070 0x55d884e70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 -- 172.21.15.83:0/2276065202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884ab8070 msgr2=0x55d884e70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884ab8070 0x55d884e70810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 -- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 msgr2=0x55d884e6b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e6b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 -- 172.21.15.83:0/2276065202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d884d8dc40 con 0x55d884a8f1b0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f869e59c640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8690105b20 tx=0x7f86900cdb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc80ba3adcac95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.219+0000 7f8685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690049070 con 0x55d884a8f1b0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f86900f10b0 con 0x55d884a8f1b0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869800f9a0 con 0x55d884a8f1b0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f86900cd730 con 0x55d884a8f1b0 2024-09-21T03:23:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f86ab4f0740 10 client.16620 fetch_fsmap learned FSMap version 44 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.220+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d884e8a770 con 0x55d884a8f1b0 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.221+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869800f9a0 con 0x55d884a8f1b0 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.221+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f86900d71b0 con 0x55d884a8f1b0 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.221+0000 7f86ab4f0740 10 client.16620 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.221+0000 7f86ab4f0740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.221+0000 7f86ab4f0740 20 client.16620 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a769f0 0x55d884e8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f86ab4f0740 4 client.16620 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d884dd7da0 con 0x55d884a769f0 2024-09-21T03:23:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f869dd9b640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a769f0 0x55d884e8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f8685ffb640 10 client.16620 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.222+0000 7f869dd9b640 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a769f0 0x55d884e8dce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccdb5664f4a00ad1 server_cookie=43772827f9ff5f5d in_seq=0 out_seq=0 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f8685ffb640 1 -- 172.21.15.83:0/2276065202 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d884dd7da0 con 0x55d884a769f0 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f8685ffb640 10 client.16620 handle_command_reply: tid=0 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 client.16620 shutdown 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 msgr2=0x55d884e6b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8690105b20 tx=0x7f86900cdb20 comp rx=0 tx=0).stop 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 shutdown_connections 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a769f0 0x55d884e8dce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884ab8070 0x55d884e70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e6b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2276065202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a8f1b0 0x55d884e6b210 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 >> 172.21.15.83:0/2276065202 conn(0x55d884aac460 msgr2=0x55d884e5b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.223+0000 7f8685ffb640 0 client.16620 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.224+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 shutdown_connections 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.224+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2276065202 wait complete. 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.224+0000 7f86ab4f0740 20 client.16620 trim_cache size 0 max 16384 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.248+0000 7f86ab4f0740 1 Processor -- start 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 -- start start 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 0x55d884e68430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ad340 con 0x55d884ab8070 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01aee80 con 0x55d884a97390 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ada10 con 0x55d884a8f1b0 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 0x55d884e68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 0x55d884e68430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45636/0 (socket says 172.21.15.83:45636) 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f869ed9d640 1 -- 172.21.15.83:0/58620775 learned_addr learned my addr 172.21.15.83:0/58620775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.249+0000 7f869e59c640 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869dd9b640 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 -- 172.21.15.83:0/58620775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 msgr2=0x55d884e69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e69df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 -- 172.21.15.83:0/58620775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 msgr2=0x55d884e68430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 0x55d884e68430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 -- 172.21.15.83:0/58620775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884e5df90 con 0x55d884a8f1b0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f869e59c640 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e698b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86900cd920 tx=0x7f8690031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a0ee4f1278fc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.250+0000 7f8687fff640 1 -- 172.21.15.83:0/58620775 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690049070 con 0x55d884a8f1b0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.251+0000 7f8687fff640 1 -- 172.21.15.83:0/58620775 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86900d2700 con 0x55d884a8f1b0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.251+0000 7f8687fff640 1 -- 172.21.15.83:0/58620775 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86900f10f0 con 0x55d884a8f1b0 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 -- 172.21.15.83:0/58620775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 msgr2=0x55d884e698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e698b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86900cd920 tx=0x7f8690031400 comp rx=0 tx=0).stop 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 -- 172.21.15.83:0/58620775 shutdown_connections 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884ab8070 0x55d884e68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884a97390 0x55d884e69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/58620775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e698b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 -- 172.21.15.83:0/58620775 >> 172.21.15.83:0/58620775 conn(0x55d884aac460 msgr2=0x55d884e66210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 -- 172.21.15.83:0/58620775 shutdown_connections 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.252+0000 7f86ab4f0740 1 -- 172.21.15.83:0/58620775 wait complete. 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.253+0000 7f86ab4f0740 1 -- start start 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.253+0000 7f86ab4f0740 1 Processor -- start 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.253+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a97390 0x55d884e9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ae7b0 con 0x55d884a97390 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ae0e0 con 0x55d884ab8070 2024-09-21T03:23:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ab380 con 0x55d884a8f1b0 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f86ab4f0740 10 client.0 _ll_register_callbacks cb 0x7f8690009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36350/0 (socket says 172.21.15.83:36350) 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 -- 172.21.15.83:0/2584380856 learned_addr learned my addr 172.21.15.83:0/2584380856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 -- 172.21.15.83:0/2584380856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 msgr2=0x55d884e9f400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e9f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 -- 172.21.15.83:0/2584380856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a97390 msgr2=0x55d884e9f940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a97390 0x55d884e9f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.254+0000 7f869ed9d640 1 -- 172.21.15.83:0/2584380856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d884e9a8f0 con 0x55d884ab8070 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.255+0000 7f869ed9d640 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8698006010 tx=0x7f8698003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7c4a51bff7ef33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.255+0000 7f8685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.256+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869802c060 con 0x55d884ab8070 2024-09-21T03:23:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.256+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f869800fba0 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.256+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d884dd3850 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.262+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f869801e040 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.266+0000 7f86ab4f0740 10 client.26180 fetch_fsmap learned FSMap version 44 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.266+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d884f09cf0 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869801f040 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f8698024040 con 0x55d884ab8070 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 10 client.26180 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 20 client.26180 populate_metadata read hostname 'smithi083' 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a4bda0 0x55d884f0c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 4 client.26180 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.268+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d884dd3850 con 0x55d884a4bda0 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.272+0000 7f869e59c640 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a4bda0 0x55d884f0c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.277+0000 7f8685ffb640 10 client.26180 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.277+0000 7f869e59c640 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a4bda0 0x55d884f0c2d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b2320a75214239 server_cookie=ec33ecf95949f10b in_seq=0 out_seq=0 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.278+0000 7f8685ffb640 1 -- 172.21.15.83:0/2584380856 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x7f86900f50c0 con 0x55d884a4bda0 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.278+0000 7f8685ffb640 10 client.26180 handle_command_reply: tid=0 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 client.26180 shutdown 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 msgr2=0x55d884e9a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8698006010 tx=0x7f8698003ba0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 shutdown_connections 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d884a4bda0 0x55d884f0c2d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f8685ffb640 0 client.26180 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d884ab8070 0x55d884e9a3b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d884a97390 0x55d884e9f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.279+0000 7f86ab4f0740 1 --2- 172.21.15.83:0/2584380856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d884a8f1b0 0x55d884e9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.280+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 >> 172.21.15.83:0/2584380856 conn(0x55d884aac460 msgr2=0x55d884e66210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.281+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 shutdown_connections 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.281+0000 7f86ab4f0740 1 -- 172.21.15.83:0/2584380856 wait complete. 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.282+0000 7f86ab4f0740 20 client.26180 trim_cache size 0 max 16384 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc", 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:23:35.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.013 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:23:35.014 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.016 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:23:35.017 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 msgr2=0x7f86880b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 0x7f86880b8bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86a0102490 tx=0x7f8694005f90 comp rx=0 tx=0).stop 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 msgr2=0x7f86a01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8690005ab0 tx=0x7f8690003c10 comp rx=0 tx=0).stop 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 shutdown_connections 2024-09-21T03:23:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f86880b66e0 0x7f86880b8bf0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86a0102490 tx=0x7f8694005f90 comp rx=0 tx=0).stop 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a010de70 0x7f86a0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a01052f0 0x7f86a0102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 --2- 172.21.15.83:0/2805608669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a01048c0 0x7f86a01027d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.284+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 >> 172.21.15.83:0/2805608669 conn(0x7f86a0079750 msgr2=0x7f86a0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.286+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 shutdown_connections 2024-09-21T03:23:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:23:34.287+0000 7f86a4f7c640 1 -- 172.21.15.83:0/2805608669 wait complete. 2024-09-21T03:23:35.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:35 smithi193 ceph-mon[35224]: pgmap v534: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-09-21T03:23:35.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:35 smithi083 ceph-mon[31219]: pgmap v534: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-09-21T03:23:35.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:35 smithi142 ceph-mon[34512]: pgmap v534: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-09-21T03:23:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 375.0 s, 2297.8 tps, lat 43.630 ms stddev 48.302 2024-09-21T03:23:36.133 INFO:tasks.cephfs.filesystem:scrub status for tag:81bdb2f6-a018-40f8-b3a8-29c75c89e9fc - {'path': '/', 'tag': '81bdb2f6-a018-40f8-b3a8-29c75c89e9fc', 'options': 'recursive,force'} 2024-09-21T03:23:37.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:37 smithi193 ceph-mon[35224]: pgmap v535: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-21T03:23:37.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:37 smithi083 ceph-mon[31219]: pgmap v535: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-21T03:23:37.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:37 smithi142 ceph-mon[34512]: pgmap v535: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-21T03:23:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:39 smithi193 ceph-mon[35224]: pgmap v536: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 858 op/s 2024-09-21T03:23:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:23:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:23:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:23:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:23:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:39 smithi083 ceph-mon[31219]: pgmap v536: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 858 op/s 2024-09-21T03:23:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:23:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:23:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:23:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:23:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:39 smithi142 ceph-mon[34512]: pgmap v536: 97 pgs: 97 active+clean; 4.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 858 op/s 2024-09-21T03:23:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:23:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:23:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:23:39.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:23:40.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:40 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:23:40.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:40 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:23:40.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:40 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:23:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 380.0 s, 2604.0 tps, lat 38.274 ms stddev 43.945 2024-09-21T03:23:41.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:41 smithi193 ceph-mon[35224]: pgmap v537: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-09-21T03:23:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:41 smithi083 ceph-mon[31219]: pgmap v537: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-09-21T03:23:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:41 smithi142 ceph-mon[34512]: pgmap v537: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-09-21T03:23:43.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:43 smithi193 ceph-mon[35224]: pgmap v538: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 903 op/s 2024-09-21T03:23:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:43 smithi083 ceph-mon[31219]: pgmap v538: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 903 op/s 2024-09-21T03:23:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:43 smithi142 ceph-mon[34512]: pgmap v538: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 903 op/s 2024-09-21T03:23:45.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:45 smithi193 ceph-mon[35224]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 913 op/s 2024-09-21T03:23:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:45 smithi083 ceph-mon[31219]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 913 op/s 2024-09-21T03:23:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:45 smithi142 ceph-mon[34512]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 913 op/s 2024-09-21T03:23:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 385.0 s, 2021.8 tps, lat 49.280 ms stddev 101.644 2024-09-21T03:23:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:47 smithi193 ceph-mon[35224]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-09-21T03:23:47.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:47 smithi083 ceph-mon[31219]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-09-21T03:23:47.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:47 smithi142 ceph-mon[34512]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 850 op/s 2024-09-21T03:23:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:49 smithi193 ceph-mon[35224]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-09-21T03:23:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:49 smithi083 ceph-mon[31219]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-09-21T03:23:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:49 smithi142 ceph-mon[34512]: pgmap v541: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.05k op/s 2024-09-21T03:23:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 390.0 s, 2467.4 tps, lat 40.531 ms stddev 45.891 2024-09-21T03:23:51.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:51 smithi083 ceph-mon[31219]: pgmap v542: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.77k op/s 2024-09-21T03:23:51.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:51 smithi193 ceph-mon[35224]: pgmap v542: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.77k op/s 2024-09-21T03:23:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:51 smithi142 ceph-mon[34512]: pgmap v542: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.77k op/s 2024-09-21T03:23:53.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:53 smithi193 ceph-mon[35224]: pgmap v543: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.67k op/s 2024-09-21T03:23:53.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:53 smithi083 ceph-mon[31219]: pgmap v543: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.67k op/s 2024-09-21T03:23:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:53 smithi142 ceph-mon[34512]: pgmap v543: 97 pgs: 97 active+clean; 4.8 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.67k op/s 2024-09-21T03:23:54.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:54 smithi193 ceph-mon[35224]: pgmap v544: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-09-21T03:23:54.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:54 smithi083 ceph-mon[31219]: pgmap v544: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-09-21T03:23:54.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:54 smithi142 ceph-mon[34512]: pgmap v544: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 1.74k op/s 2024-09-21T03:23:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:55 smithi142 ceph-mon[34512]: pgmap v545: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.67k op/s 2024-09-21T03:23:55.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:55 smithi193 ceph-mon[35224]: pgmap v545: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.67k op/s 2024-09-21T03:23:55.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:55 smithi083 ceph-mon[31219]: pgmap v545: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.67k op/s 2024-09-21T03:23:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 395.0 s, 2216.0 tps, lat 45.362 ms stddev 53.280 2024-09-21T03:23:57.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:23:57 smithi142 ceph-mon[34512]: pgmap v546: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.56k op/s 2024-09-21T03:23:57.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:23:57 smithi193 ceph-mon[35224]: pgmap v546: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.56k op/s 2024-09-21T03:23:57.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:23:57 smithi083 ceph-mon[31219]: pgmap v546: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.56k op/s 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: pgmap v547: 97 pgs: 97 active+clean; 4.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:00.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:24:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: pgmap v547: 97 pgs: 97 active+clean; 4.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T03:24:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T03:24:00.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:24:00.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:00.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:00.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:24:00.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: pgmap v547: 97 pgs: 97 active+clean; 4.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T03:24:00.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T03:24:00.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T03:24:00.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-21T03:24:00.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:00.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:00.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:24:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 400.0 s, 1897.4 tps, lat 51.713 ms stddev 103.640 2024-09-21T03:24:02.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:02 smithi142 ceph-mon[34512]: pgmap v548: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-09-21T03:24:02.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:02 smithi193 ceph-mon[35224]: pgmap v548: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-09-21T03:24:02.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:02 smithi083 ceph-mon[31219]: pgmap v548: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-09-21T03:24:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:04 smithi193 ceph-mon[35224]: pgmap v549: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 27 MiB/s wr, 2.00k op/s 2024-09-21T03:24:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:04 smithi193 ceph-mon[35224]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:24:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:04 smithi083 ceph-mon[31219]: pgmap v549: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 27 MiB/s wr, 2.00k op/s 2024-09-21T03:24:04.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:04 smithi083 ceph-mon[31219]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:24:04.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:04 smithi142 ceph-mon[34512]: pgmap v549: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 27 MiB/s wr, 2.00k op/s 2024-09-21T03:24:04.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:04 smithi142 ceph-mon[34512]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:24:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 405.0 s, 2585.6 tps, lat 39.404 ms stddev 48.391 2024-09-21T03:24:06.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:24:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:06.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:06 smithi083 ceph-mon[31219]: pgmap v551: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 2.28k op/s 2024-09-21T03:24:06.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:06 smithi193 ceph-mon[35224]: pgmap v551: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 2.28k op/s 2024-09-21T03:24:06.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:06 smithi142 ceph-mon[34512]: pgmap v551: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 2.28k op/s 2024-09-21T03:24:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 -- 172.21.15.83:0/2327430318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8105170 msgr2=0x7f3ed8105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/2327430318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8105170 0x7f3ed8105580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009900 tx=0x7f3ec802a410 comp rx=0 tx=0).stop 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 -- 172.21.15.83:0/2327430318 shutdown_connections 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/2327430318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105ac0 0x7f3ed810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/2327430318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8105170 0x7f3ed8105580 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/2327430318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8103160 0x7f3ed8103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.192+0000 7f3ee07b7640 1 -- 172.21.15.83:0/2327430318 >> 172.21.15.83:0/2327430318 conn(0x7f3ed8077c60 msgr2=0x7f3ed80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.194+0000 7f3ee07b7640 1 -- 172.21.15.83:0/2327430318 shutdown_connections 2024-09-21T03:24:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.194+0000 7f3ee07b7640 1 -- 172.21.15.83:0/2327430318 wait complete. 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.194+0000 7f3ee07b7640 1 Processor -- start 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 -- start start 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105170 0x7f3ed819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8105ac0 0x7f3ed819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8113ab0 con 0x7f3ed8103160 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8113930 con 0x7f3ed8105ac0 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ee07b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8113c30 con 0x7f3ed8105170 2024-09-21T03:24:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ede553640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ede553640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44498/0 (socket says 172.21.15.83:44498) 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3ede553640 1 -- 172.21.15.83:0/3737489426 learned_addr learned my addr 172.21.15.83:0/3737489426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105170 0x7f3ed819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.195+0000 7f3eded54640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8105ac0 0x7f3ed819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 -- 172.21.15.83:0/3737489426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105170 msgr2=0x7f3ed819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105170 0x7f3ed819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 -- 172.21.15.83:0/3737489426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8105ac0 msgr2=0x7f3ed819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8105ac0 0x7f3ed819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 -- 172.21.15.83:0/3737489426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a0410 con 0x7f3ed8103160 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ede553640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400d950 tx=0x7f3ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ecf7fe640 1 -- 172.21.15.83:0/3737489426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3ed8103160 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ecf7fe640 1 -- 172.21.15.83:0/3737489426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed4009c30 con 0x7f3ed8103160 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a0700 con 0x7f3ed8103160 2024-09-21T03:24:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ecf7fe640 1 -- 172.21.15.83:0/3737489426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400f620 con 0x7f3ed8103160 2024-09-21T03:24:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.196+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a6e60 con 0x7f3ed8103160 2024-09-21T03:24:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.199+0000 7f3ecf7fe640 1 -- 172.21.15.83:0/3737489426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3ed4010460 con 0x7f3ed8103160 2024-09-21T03:24:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.200+0000 7f3ecf7fe640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 0x7f3ec40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 0x7f3ec40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 Processor -- start 2024-09-21T03:24:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ecf7fe640 1 -- 172.21.15.83:0/3737489426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f3ed40e7230 con 0x7f3ed8103160 2024-09-21T03:24:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 -- start start 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 0x557889bbd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897ed1b0 0x557889bbdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 0x557889bbe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557889b33e50 con 0x5578897f5390 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557889b23280 con 0x557889816070 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.201+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557889b27090 con 0x5578897ed1b0 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 0x557889bbd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 0x7f3ec40b8c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009230 tx=0x7f3ec8031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 0x557889bbd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41974/0 (socket says 172.21.15.83:41974) 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3ede553640 1 -- 172.21.15.83:0/2157644191 learned_addr learned my addr 172.21.15.83:0/2157644191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 0x557889bbe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eddd52640 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897ed1b0 0x557889bbdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 -- 172.21.15.83:0/2157644191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897ed1b0 msgr2=0x557889bbdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897ed1b0 0x557889bbdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 -- 172.21.15.83:0/2157644191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 msgr2=0x557889bbd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 0x557889bbd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.202+0000 7f3eded54640 1 -- 172.21.15.83:0/2157644191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557889881650 con 0x5578897f5390 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.203+0000 7f3eded54640 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 0x557889bbe0d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ab0 tx=0x7f3ed0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8b19361a1ef9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.203+0000 7f3eccff9640 1 -- 172.21.15.83:0/2157644191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0004d20 con 0x5578897f5390 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.204+0000 7f3eccff9640 1 -- 172.21.15.83:0/2157644191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed0005020 con 0x5578897f5390 2024-09-21T03:24:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.204+0000 7f3eccff9640 1 -- 172.21.15.83:0/2157644191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0021600 con 0x5578897f5390 2024-09-21T03:24:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.208+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2157644191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 msgr2=0x557889bbe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.208+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 0x557889bbe0d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ab0 tx=0x7f3ed0009f70 comp rx=0 tx=0).stop 2024-09-21T03:24:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.212+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2157644191 shutdown_connections 2024-09-21T03:24:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.212+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897f5390 0x557889bbe0d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.212+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897ed1b0 0x557889bbdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.212+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2157644191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557889816070 0x557889bbd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.213+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2157644191 >> 172.21.15.83:0/2157644191 conn(0x55788980a460 msgr2=0x557889bbb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.218+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2157644191 shutdown_connections 2024-09-21T03:24:07.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.218+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2157644191 wait complete. 2024-09-21T03:24:07.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.218+0000 7f3ee6d2b740 1 -- start start 2024-09-21T03:24:07.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.218+0000 7f3ee6d2b740 1 Processor -- start 2024-09-21T03:24:07.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 0x557889bc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557889816070 0x557889bce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 0x557889bc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 0x557889bc9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41976/0 (socket says 172.21.15.83:41976) 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39864/0 (socket says 172.21.15.83:39864) 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ede553640 1 -- 172.21.15.83:0/3985688522 learned_addr learned my addr 172.21.15.83:0/3985688522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557889984700 con 0x557889816070 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557889abe110 con 0x5578897ed1b0 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.219+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557889bc3bc0 con 0x5578897f5390 2024-09-21T03:24:07.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.220+0000 7f3ee6d2b740 10 client.0 _ll_register_callbacks cb 0x557889ad7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:07.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eded54640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557889816070 0x557889bce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eddd52640 1 -- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 msgr2=0x557889bc9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 0x557889bc9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eddd52640 1 -- 172.21.15.83:0/3985688522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557889816070 msgr2=0x557889bce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557889816070 0x557889bce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.224+0000 7f3eddd52640 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557889aebc40 con 0x5578897f5390 2024-09-21T03:24:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.226+0000 7f3eddd52640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ec802a930 tx=0x7f3ec8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a23805c58f4255b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.227+0000 7f3ec1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.228+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8035040 con 0x5578897f5390 2024-09-21T03:24:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.228+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f3ec8005040 con 0x5578897f5390 2024-09-21T03:24:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.228+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed0005020 con 0x5578897f5390 2024-09-21T03:24:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.229+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3ec8005240 con 0x5578897f5390 2024-09-21T03:24:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.229+0000 7f3ee6d2b740 10 client.26164 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.229+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557889bd7ab0 con 0x5578897f5390 2024-09-21T03:24:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.229+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed0005020 con 0x5578897f5390 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec80035f0 con 0x5578897f5390 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 10 client.26164 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 20 client.26164 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5578897d49f0 0x557889bdaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 4 client.26164 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557889b35da0 con 0x5578897d49f0 2024-09-21T03:24:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.230+0000 7f3ede553640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5578897d49f0 0x557889bdaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.231+0000 7f3ec1ffb640 10 client.26164 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.231+0000 7f3ede553640 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5578897d49f0 0x557889bdaed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96f8af5ae5ddf1b server_cookie=58021b18047a7dcf in_seq=0 out_seq=0 2024-09-21T03:24:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.232+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/3985688522 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557889b35da0 con 0x5578897d49f0 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.232+0000 7f3ec1ffb640 10 client.26164 handle_command_reply: tid=0 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.232+0000 7f3ee6d2b740 1 client.26164 shutdown 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 msgr2=0x557889bc9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ec802a930 tx=0x7f3ec8009260 comp rx=0 tx=0).stop 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 shutdown_connections 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5578897d49f0 0x557889bdaed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557889816070 0x557889bce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889bc9750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/3985688522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897ed1b0 0x557889bc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ec1ffb640 0 client.26164 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 >> 172.21.15.83:0/3985688522 conn(0x55788980a460 msgr2=0x557889bb9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 shutdown_connections 2024-09-21T03:24:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/3985688522 wait complete. 2024-09-21T03:24:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.233+0000 7f3ee6d2b740 20 client.26164 trim_cache size 0 max 16384 2024-09-21T03:24:07.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 Processor -- start 2024-09-21T03:24:07.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 -- start start 2024-09-21T03:24:07.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 0x557889bc7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897f5390 0x557889bc6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81ad500 con 0x5578897ed1b0 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81af040 con 0x5578897f5390 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ee6d2b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81adbd0 con 0x5578897d49f0 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44526/0 (socket says 172.21.15.83:44526) 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3eddd52640 1 -- 172.21.15.83:0/1696540441 learned_addr learned my addr 172.21.15.83:0/1696540441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.256+0000 7f3ede553640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 0x557889bc7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eded54640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897f5390 0x557889bc6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 -- 172.21.15.83:0/1696540441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 msgr2=0x557889bc7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 0x557889bc7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 -- 172.21.15.83:0/1696540441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897f5390 msgr2=0x557889bc6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897f5390 0x557889bc6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 -- 172.21.15.83:0/1696540441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557889bc7710 con 0x5578897ed1b0 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3ede553640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 0x557889bc7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3eddd52640 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80316d0 tx=0x7f3ec8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6484a35dedf20286 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3ec3fff640 1 -- 172.21.15.83:0/1696540441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8035050 con 0x5578897ed1b0 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3ec3fff640 1 -- 172.21.15.83:0/1696540441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec80035f0 con 0x5578897ed1b0 2024-09-21T03:24:07.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.257+0000 7f3ec3fff640 1 -- 172.21.15.83:0/1696540441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8004e60 con 0x5578897ed1b0 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/1696540441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 msgr2=0x557889bc7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80316d0 tx=0x7f3ec8032b20 comp rx=0 tx=0).stop 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/1696540441 shutdown_connections 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897f5390 0x557889bc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bc7fd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/1696540441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897d49f0 0x557889bc7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/1696540441 >> 172.21.15.83:0/1696540441 conn(0x55788980a460 msgr2=0x557889bc4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.259+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/1696540441 shutdown_connections 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.260+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/1696540441 wait complete. 2024-09-21T03:24:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.260+0000 7f3ee6d2b740 1 -- start start 2024-09-21T03:24:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.260+0000 7f3ee6d2b740 1 Processor -- start 2024-09-21T03:24:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 0x557889bd1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889c44920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44536/0 (socket says 172.21.15.83:44536) 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 0x557889bd1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ede553640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 0x557889bd1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42000/0 (socket says 172.21.15.83:42000) 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 -- 172.21.15.83:0/2110236708 learned_addr learned my addr 172.21.15.83:0/2110236708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81ae970 con 0x5578897ed1b0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81ae2a0 con 0x5578897d49f0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81ab500 con 0x5578897f5390 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eded54640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889c44920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 -- 172.21.15.83:0/2110236708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 msgr2=0x557889c44920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889c44920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 -- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 msgr2=0x557889bd1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 0x557889bd1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.261+0000 7f3eddd52640 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557889bd2160 con 0x5578897ed1b0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ee6d2b740 10 client.0 _ll_register_callbacks cb 0x7f3ec8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3eddd52640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009900 tx=0x7f3ec8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e7f923f18e3c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ec1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec80035f0 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/2110236708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8072060 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/2110236708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f3ec8004b60 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/2110236708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3ec8035040 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ee6d2b740 10 client.16641 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.262+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=44}) -- 0x557889c45bd0 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/2110236708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec8004da0 con 0x5578897ed1b0 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 10 client.16641 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 20 client.16641 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557889816070 0x557889c48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 4 client.16641 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:24:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3ec80035f0 con 0x557889816070 2024-09-21T03:24:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.263+0000 7f3ede553640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557889816070 0x557889c48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.264+0000 7f3ede553640 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557889816070 0x557889c48330 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e0e38b67748e1b server_cookie=e6144f31e668f6f2 in_seq=0 out_seq=0 2024-09-21T03:24:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.264+0000 7f3ec1ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ec1ffb640 1 -- 172.21.15.83:0/2110236708 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x7f3ec80035f0 con 0x557889816070 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ec1ffb640 10 client.16641 handle_command_reply: tid=0 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 client.16641 shutdown 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 msgr2=0x557889bd1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009900 tx=0x7f3ec8032b90 comp rx=0 tx=0).stop 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 shutdown_connections 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557889816070 0x557889c48330 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578897f5390 0x557889c44920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ec1ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578897ed1b0 0x557889bd1870 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 --2- 172.21.15.83:0/2110236708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578897d49f0 0x557889bd1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.266+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 >> 172.21.15.83:0/2110236708 conn(0x55788980a460 msgr2=0x557889bc7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 shutdown_connections 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee6d2b740 1 -- 172.21.15.83:0/2110236708 wait complete. 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee6d2b740 20 client.16641 trim_cache size 0 max 16384 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.270 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:24:07.271 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.272 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.273 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.274 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:07.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:24:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 msgr2=0x7f3ec40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 0x7f3ec40b8c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009230 tx=0x7f3ec8031400 comp rx=0 tx=0).stop 2024-09-21T03:24:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 msgr2=0x7f3ed819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400d950 tx=0x7f3ed400de10 comp rx=0 tx=0).stop 2024-09-21T03:24:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 shutdown_connections 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3ec40b6700 0x7f3ec40b8c10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed8105ac0 0x7f3ed819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8105170 0x7f3ed819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 --2- 172.21.15.83:0/3737489426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed8103160 0x7f3ed819a4a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.267+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 >> 172.21.15.83:0/3737489426 conn(0x7f3ed8077c60 msgr2=0x7f3ed81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.268+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 shutdown_connections 2024-09-21T03:24:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:07.268+0000 7f3ee07b7640 1 -- 172.21.15.83:0/3737489426 wait complete. 2024-09-21T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:08 smithi193 ceph-mon[35224]: pgmap v552: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 31 MiB/s wr, 2.33k op/s 2024-09-21T03:24:08.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:08 smithi083 ceph-mon[31219]: pgmap v552: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 31 MiB/s wr, 2.33k op/s 2024-09-21T03:24:08.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:08 smithi142 ceph-mon[34512]: pgmap v552: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 31 MiB/s wr, 2.33k op/s 2024-09-21T03:24:10.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:10 smithi193 ceph-mon[35224]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 25 MiB/s wr, 1.95k op/s 2024-09-21T03:24:10.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:10 smithi083 ceph-mon[31219]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 25 MiB/s wr, 1.95k op/s 2024-09-21T03:24:10.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:10 smithi142 ceph-mon[34512]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 25 MiB/s wr, 1.95k op/s 2024-09-21T03:24:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 410.0 s, 2074.4 tps, lat 48.186 ms stddev 64.397 2024-09-21T03:24:11.751 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:24:11.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:24:12.119 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:12.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:12 smithi083 ceph-mon[31219]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-21T03:24:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.671+0000 7f0d9cb17640 1 -- 172.21.15.83:0/3117722085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9811cba0 msgr2=0x7f0d9811efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.671+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/3117722085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9811cba0 0x7f0d9811efc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009860 tx=0x7f0d8802a380 comp rx=0 tx=0).stop 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.672+0000 7f0d9cb17640 1 -- 172.21.15.83:0/3117722085 shutdown_connections 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.672+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/3117722085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9811cba0 0x7f0d9811efc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.672+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/3117722085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d9810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.672+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/3117722085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d98111cd0 0x7f0d9810d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.672+0000 7f0d9cb17640 1 -- 172.21.15.83:0/3117722085 >> 172.21.15.83:0/3117722085 conn(0x7f0d9806e9f0 msgr2=0x7f0d9806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.675+0000 7f0d9cb17640 1 -- 172.21.15.83:0/3117722085 shutdown_connections 2024-09-21T03:24:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.676+0000 7f0d9cb17640 1 -- 172.21.15.83:0/3117722085 wait complete. 2024-09-21T03:24:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.677+0000 7f0d9cb17640 1 Processor -- start 2024-09-21T03:24:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.677+0000 7f0d9cb17640 1 -- start start 2024-09-21T03:24:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.677+0000 7f0d9cb17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.677+0000 7f0d9cb17640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 0x7f0d981a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d9cb17640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9811cba0 0x7f0d981ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d9cb17640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98122710 con 0x7f0d9810dbf0 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d9cb17640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98122590 con 0x7f0d9811cba0 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d9cb17640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98122890 con 0x7f0d98111cd0 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45706/0 (socket says 172.21.15.83:45706) 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d977fe640 1 -- 172.21.15.83:0/2613307088 learned_addr learned my addr 172.21.15.83:0/2613307088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 0x7f0d981a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.678+0000 7f0d97fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9811cba0 0x7f0d981ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2613307088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9811cba0 msgr2=0x7f0d981ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9811cba0 0x7f0d981ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 msgr2=0x7f0d981a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2613307088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d981aef90 con 0x7f0d98111cd0 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d977fe640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 0x7f0d981a95c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806c910 tx=0x7f0d8c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00cc90 con 0x7f0d98111cd0 2024-09-21T03:24:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d981af280 con 0x7f0d98111cd0 2024-09-21T03:24:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.679+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d981b59e0 con 0x7f0d98111cd0 2024-09-21T03:24:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.681+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c00ee70 con 0x7f0d98111cd0 2024-09-21T03:24:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.681+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c018680 con 0x7f0d98111cd0 2024-09-21T03:24:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.683+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9810e000 con 0x7f0d98111cd0 2024-09-21T03:24:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.686+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0d8c010430 con 0x7f0d98111cd0 2024-09-21T03:24:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.687+0000 7f0d94ff9640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 0x7f0d800b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.687+0000 7f0d977fe640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 0x7f0d800b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.687+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f0d8c0e7a50 con 0x7f0d98111cd0 2024-09-21T03:24:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.688+0000 7f0d977fe640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 0x7f0d800b8cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d90010480 tx=0x7f0d900096d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:12.692+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d8c0b2340 con 0x7f0d98111cd0 2024-09-21T03:24:12.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:12 smithi193 ceph-mon[35224]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-21T03:24:12.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:12 smithi142 ceph-mon[34512]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-21T03:24:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.020+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d981aa150 con 0x7f0d98111cd0 2024-09-21T03:24:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.021+0000 7f0d94ff9640 1 -- 172.21.15.83:0/2613307088 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11982 (secure 0 0 0) 0x7f0d8c0b7280 con 0x7f0d98111cd0 2024-09-21T03:24:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:24:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"btime":"2024-09-21T03:23:29:976813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:23:29.048942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:24:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-09-21T03:24:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.023+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 msgr2=0x7f0d800b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.023+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 0x7f0d800b8cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d90010480 tx=0x7f0d900096d0 comp rx=0 tx=0).stop 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.023+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 msgr2=0x7f0d981a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.023+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 0x7f0d981a95c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806c910 tx=0x7f0d8c00ec20 comp rx=0 tx=0).stop 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 shutdown_connections 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0d800b67b0 0x7f0d800b8cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9811cba0 0x7f0d981ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d98111cd0 0x7f0d981a95c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 --2- 172.21.15.83:0/2613307088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d9810dbf0 0x7f0d981a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 >> 172.21.15.83:0/2613307088 conn(0x7f0d9806e9f0 msgr2=0x7f0d98110a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 shutdown_connections 2024-09-21T03:24:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:13.024+0000 7f0d9cb17640 1 -- 172.21.15.83:0/2613307088 wait complete. 2024-09-21T03:24:13.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:13 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2613307088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:24:13.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:24:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:13 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2613307088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:24:13.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:13 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2613307088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:24:14.105 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.687+0000 7fe258cb3640 1 -- 172.21.15.83:0/210532894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c033e30 con 0x7fe254107950 2024-09-21T03:24:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 -- 172.21.15.83:0/210532894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254107950 msgr2=0x7fe254107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 --2- 172.21.15.83:0/210532894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254107950 0x7fe254107d60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe24c005fd0 tx=0x7fe24c02c8c0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 -- 172.21.15.83:0/210532894 shutdown_connections 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 --2- 172.21.15.83:0/210532894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25410ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 --2- 172.21.15.83:0/210532894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254107950 0x7fe254107d60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 --2- 172.21.15.83:0/210532894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe254102010 0x7fe254102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 -- 172.21.15.83:0/210532894 >> 172.21.15.83:0/210532894 conn(0x7fe254077aa0 msgr2=0x7fe254077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 -- 172.21.15.83:0/210532894 shutdown_connections 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.688+0000 7fe25acb7640 1 -- 172.21.15.83:0/210532894 wait complete. 2024-09-21T03:24:14.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.689+0000 7fe25acb7640 1 Processor -- start 2024-09-21T03:24:14.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.689+0000 7fe25acb7640 1 -- start start 2024-09-21T03:24:14.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254102010 0x7fe2541981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe254107950 0x7fe254198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541116e0 con 0x7fe254102010 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe254111560 con 0x7fe254107950 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.690+0000 7fe25acb7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe254111860 con 0x7fe2541082a0 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.691+0000 7fe25a4b6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.691+0000 7fe25a4b6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60398/0 (socket says 172.21.15.83:60398) 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.691+0000 7fe25a4b6640 1 -- 172.21.15.83:0/3711515167 learned_addr learned my addr 172.21.15.83:0/3711515167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.691+0000 7fe25a4b6640 1 -- 172.21.15.83:0/3711515167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe254107950 msgr2=0x7fe254198750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe254107950 0x7fe254198750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25a4b6640 1 -- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254102010 msgr2=0x7fe2541981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254102010 0x7fe2541981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25a4b6640 1 -- 172.21.15.83:0/3711515167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25419e0f0 con 0x7fe2541082a0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe250009db0 tx=0x7fe25000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe24affd640 1 -- 172.21.15.83:0/3711515167 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250005020 con 0x7fe2541082a0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe24affd640 1 -- 172.21.15.83:0/3711515167 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250005670 con 0x7fe2541082a0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe24affd640 1 -- 172.21.15.83:0/3711515167 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250004a20 con 0x7fe2541082a0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25419e3e0 con 0x7fe2541082a0 2024-09-21T03:24:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.692+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25406cde0 con 0x7fe2541082a0 2024-09-21T03:24:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.695+0000 7fe24affd640 1 -- 172.21.15.83:0/3711515167 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe250004c40 con 0x7fe2541082a0 2024-09-21T03:24:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.695+0000 7fe26248d740 1 Processor -- start 2024-09-21T03:24:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.697+0000 7fe24affd640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 0x7fe2400c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.697+0000 7fe2594b4640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 0x7fe2400c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.697+0000 7fe24affd640 1 -- 172.21.15.83:0/3711515167 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe2500eb430 con 0x7fe2541082a0 2024-09-21T03:24:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 -- start start 2024-09-21T03:24:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 0x5614451b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444de91b0 0x5614451b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56144512fea0 con 0x561444df1390 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56144511f2d0 con 0x561444de91b0 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.698+0000 7fe26248d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614451230e0 con 0x561444e12070 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe25a4b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe25a4b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451ba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45740/0 (socket says 172.21.15.83:45740) 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe25a4b6640 1 -- 172.21.15.83:0/1869101790 learned_addr learned my addr 172.21.15.83:0/1869101790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe2594b4640 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444de91b0 0x5614451b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe259cb5640 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 0x5614451b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe2594b4640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 0x7fe2400c14f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02cde0 tx=0x7fe24c00a500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe259cb5640 1 -- 172.21.15.83:0/1869101790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444de91b0 msgr2=0x5614451b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.699+0000 7fe259cb5640 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444de91b0 0x5614451b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe259cb5640 1 -- 172.21.15.83:0/1869101790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 msgr2=0x5614451ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe259cb5640 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451ba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe259cb5640 1 -- 172.21.15.83:0/1869101790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561444e7d650 con 0x561444e12070 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe259cb5640 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 0x5614451b9860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe244009b40 tx=0x7fe24400e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81418139e244ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe23ffff640 1 -- 172.21.15.83:0/1869101790 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400ed80 con 0x561444e12070 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe23ffff640 1 -- 172.21.15.83:0/1869101790 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244005020 con 0x561444e12070 2024-09-21T03:24:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.700+0000 7fe23ffff640 1 -- 172.21.15.83:0/1869101790 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244004a00 con 0x561444e12070 2024-09-21T03:24:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 -- 172.21.15.83:0/1869101790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 msgr2=0x5614451b9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 0x5614451b9860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe244009b40 tx=0x7fe24400e350 comp rx=0 tx=0).stop 2024-09-21T03:24:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 -- 172.21.15.83:0/1869101790 shutdown_connections 2024-09-21T03:24:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444de91b0 0x5614451b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 --2- 172.21.15.83:0/1869101790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444e12070 0x5614451b9860 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 -- 172.21.15.83:0/1869101790 >> 172.21.15.83:0/1869101790 conn(0x561444e06460 msgr2=0x5614451b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 -- 172.21.15.83:0/1869101790 shutdown_connections 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.701+0000 7fe26248d740 1 -- 172.21.15.83:0/1869101790 wait complete. 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.702+0000 7fe26248d740 1 -- start start 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.702+0000 7fe26248d740 1 Processor -- start 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.702+0000 7fe26248d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e12070 0x5614451d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561444f80700 con 0x561444df1390 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5614450ba160 con 0x561444e12070 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614451c8040 con 0x561444de91b0 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe2594b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe2594b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45752/0 (socket says 172.21.15.83:45752) 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe2594b4640 1 -- 172.21.15.83:0/4110677777 learned_addr learned my addr 172.21.15.83:0/4110677777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe26248d740 10 client.0 _ll_register_callbacks cb 0x5614450d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.703+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 -- 172.21.15.83:0/4110677777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e12070 msgr2=0x5614451d1330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e12070 0x5614451d1330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 -- 172.21.15.83:0/4110677777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 msgr2=0x5614451c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 -- 172.21.15.83:0/4110677777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614450e7c90 con 0x561444de91b0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe2594b4640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c4f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e850 tx=0x7fe244005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1699cc54bc04e868 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.704+0000 7fe23d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244029060 con 0x561444de91b0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe2440049b0 con 0x561444de91b0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe244005020 con 0x561444de91b0 2024-09-21T03:24:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe244004bb0 con 0x561444de91b0 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe26248d740 10 client.26188 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.705+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5614451d3d40 con 0x561444de91b0 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244005020 con 0x561444de91b0 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fe244010430 con 0x561444de91b0 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 10 client.26188 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 10 client.26188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 20 client.26188 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444dd09f0 0x5614451d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 4 client.26188 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561445131df0 con 0x561444dd09f0 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.706+0000 7fe2594b4640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444dd09f0 0x5614451d71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.707+0000 7fe23d7fa640 10 client.26188 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.707+0000 7fe2594b4640 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444dd09f0 0x5614451d71b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d0dd64f1aab1159 server_cookie=e524dc39195c90a8 in_seq=0 out_seq=0 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4110677777 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561445131df0 con 0x561444dd09f0 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe23d7fa640 10 client.26188 handle_command_reply: tid=0 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 client.26188 shutdown 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 msgr2=0x5614451c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c4f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e850 tx=0x7fe244005400 comp rx=0 tx=0).stop 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 shutdown_connections 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444dd09f0 0x5614451d71b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e12070 0x5614451d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe23d7fa640 0 client.26188 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 --2- 172.21.15.83:0/4110677777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c4f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 >> 172.21.15.83:0/4110677777 conn(0x561444e06460 msgr2=0x5614451bc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 shutdown_connections 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.708+0000 7fe26248d740 1 -- 172.21.15.83:0/4110677777 wait complete. 2024-09-21T03:24:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.709+0000 7fe26248d740 20 client.26188 trim_cache size 0 max 16384 2024-09-21T03:24:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.728+0000 7fe26248d740 1 Processor -- start 2024-09-21T03:24:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.728+0000 7fe26248d740 1 -- start start 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b5c70 con 0x561444df1390 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b77b0 con 0x561444e06460 2024-09-21T03:24:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.729+0000 7fe26248d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b6340 con 0x561444de91b0 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe25a4b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe259cb5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60426/0 (socket says 172.21.15.83:60426) 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 -- 172.21.15.83:0/1757477866 learned_addr learned my addr 172.21.15.83:0/1757477866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 -- 172.21.15.83:0/1757477866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 msgr2=0x5614451bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 -- 172.21.15.83:0/1757477866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 msgr2=0x5614451c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.730+0000 7fe2594b4640 1 -- 172.21.15.83:0/1757477866 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614451bc6c0 con 0x561444de91b0 2024-09-21T03:24:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.731+0000 7fe2594b4640 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000f80 tx=0x7fe24c03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24695da2715fe11f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.731+0000 7fe23f7fe640 1 -- 172.21.15.83:0/1757477866 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c040070 con 0x561444de91b0 2024-09-21T03:24:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.731+0000 7fe23f7fe640 1 -- 172.21.15.83:0/1757477866 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe24c038070 con 0x561444de91b0 2024-09-21T03:24:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.731+0000 7fe23f7fe640 1 -- 172.21.15.83:0/1757477866 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004570 con 0x561444de91b0 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- 172.21.15.83:0/1757477866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 msgr2=0x5614451c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000f80 tx=0x7fe24c03d000 comp rx=0 tx=0).stop 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- 172.21.15.83:0/1757477866 shutdown_connections 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451c2120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 --2- 172.21.15.83:0/1757477866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- 172.21.15.83:0/1757477866 >> 172.21.15.83:0/1757477866 conn(0x5614451c55f0 msgr2=0x5614451ba890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- 172.21.15.83:0/1757477866 shutdown_connections 2024-09-21T03:24:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- 172.21.15.83:0/1757477866 wait complete. 2024-09-21T03:24:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 -- start start 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.733+0000 7fe26248d740 1 Processor -- start 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b70e0 con 0x561444df1390 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2541b6a10 con 0x561444e06460 2024-09-21T03:24:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2540737e0 con 0x561444de91b0 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe26248d740 10 client.0 _ll_register_callbacks cb 0x7fe250004f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe259cb5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe259cb5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60428/0 (socket says 172.21.15.83:60428) 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe259cb5640 1 -- 172.21.15.83:0/4080932289 learned_addr learned my addr 172.21.15.83:0/4080932289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.734+0000 7fe2594b4640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 -- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 msgr2=0x5614451cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 -- 172.21.15.83:0/4080932289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 msgr2=0x5614451cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 -- 172.21.15.83:0/4080932289 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614451cbe90 con 0x561444e06460 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe25a4b6640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451cb720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2500b1200 tx=0x7fe250009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e05afc9f69b8b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe23d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500b0d80 con 0x561444e06460 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe2500acbc0 con 0x561444e06460 2024-09-21T03:24:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.735+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c038070 con 0x561444e06460 2024-09-21T03:24:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2500b1970 con 0x561444e06460 2024-09-21T03:24:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 10 client.26216 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561445241aa0 con 0x561444e06460 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2500b5d50 con 0x561444e06460 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fe2500b6f70 con 0x561444e06460 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 10 client.26216 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 10 client.26216 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 20 client.26216 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444e12070 0x561445244250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 4 client.26216 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.736+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe24c038070 con 0x561444e12070 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.737+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444e12070 0x561445244250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.738+0000 7fe23d7fa640 10 client.26216 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:24:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.738+0000 7fe259cb5640 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444e12070 0x561445244250 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91c4c551d3dda91 server_cookie=b2339ad96b2ad35a in_seq=0 out_seq=0 2024-09-21T03:24:14.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.738+0000 7fe23d7fa640 1 -- 172.21.15.83:0/4080932289 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe24c038070 con 0x561444e12070 2024-09-21T03:24:14.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.738+0000 7fe23d7fa640 10 client.26216 handle_command_reply: tid=0 2024-09-21T03:24:14.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.738+0000 7fe26248d740 1 client.26216 shutdown 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.756+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 msgr2=0x5614451cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.756+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451cb720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2500b1200 tx=0x7fe250009f90 comp rx=0 tx=0).stop 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 shutdown_connections 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x561444e12070 0x561445244250 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561444e06460 0x5614451cb720 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561444df1390 0x5614451cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 --2- 172.21.15.83:0/4080932289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561444de91b0 0x5614451cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe23d7fa640 0 client.26216 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:24:14.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.757+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 >> 172.21.15.83:0/4080932289 conn(0x5614451c55f0 msgr2=0x5614451bb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.758+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 shutdown_connections 2024-09-21T03:24:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.761+0000 7fe26248d740 1 -- 172.21.15.83:0/4080932289 wait complete. 2024-09-21T03:24:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.765+0000 7fe26248d740 20 client.26216 trim_cache size 0 max 16384 2024-09-21T03:24:14.767 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.765+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 msgr2=0x7fe2400c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.765+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 0x7fe2400c14f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02cde0 tx=0x7fe24c00a500 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 msgr2=0x7fe25419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe250009db0 tx=0x7fe25000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 shutdown_connections 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe2400befe0 0x7fe2400c14f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2541082a0 0x7fe25419d9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe254107950 0x7fe254198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 --2- 172.21.15.83:0/3711515167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254102010 0x7fe2541981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 >> 172.21.15.83:0/3711515167 conn(0x7fe254077aa0 msgr2=0x7fe254105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 shutdown_connections 2024-09-21T03:24:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:14.766+0000 7fe25acb7640 1 -- 172.21.15.83:0/3711515167 wait complete. 2024-09-21T03:24:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:14 smithi142 ceph-mon[34512]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-21T03:24:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:14 smithi142 ceph-mon[34512]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:24:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:14 smithi083 ceph-mon[31219]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-21T03:24:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:14 smithi083 ceph-mon[31219]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:24:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:14 smithi193 ceph-mon[35224]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-21T03:24:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:14 smithi193 ceph-mon[35224]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:24:15.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:24:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 415.0 s, 2677.0 tps, lat 37.183 ms stddev 41.767 2024-09-21T03:24:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.569+0000 7fc42b968640 1 -- 172.21.15.83:0/1217777917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424109e00 msgr2=0x7fc424110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.569+0000 7fc42b968640 1 --2- 172.21.15.83:0/1217777917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424109e00 0x7fc424110630 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009ca0 tx=0x7fc41c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.570+0000 7fc42b968640 1 -- 172.21.15.83:0/1217777917 shutdown_connections 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.570+0000 7fc42b968640 1 --2- 172.21.15.83:0/1217777917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424109e00 0x7fc424110630 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.570+0000 7fc42b968640 1 --2- 172.21.15.83:0/1217777917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4241094b0 0x7fc4241098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.570+0000 7fc42b968640 1 --2- 172.21.15.83:0/1217777917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc424103bc0 0x7fc424103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.570+0000 7fc42b968640 1 -- 172.21.15.83:0/1217777917 >> 172.21.15.83:0/1217777917 conn(0x7fc42407b910 msgr2=0x7fc42407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.571+0000 7fc42b968640 1 -- 172.21.15.83:0/1217777917 shutdown_connections 2024-09-21T03:24:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.571+0000 7fc42b968640 1 -- 172.21.15.83:0/1217777917 wait complete. 2024-09-21T03:24:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 Processor -- start 2024-09-21T03:24:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 -- start start 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 0x7fc424075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 0x7fc424076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc424113ac0 con 0x7fc424103bc0 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc424113940 con 0x7fc424109e00 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.572+0000 7fc42b968640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc424113c40 con 0x7fc4241094b0 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc428f03640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 0x7fc424075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc428f03640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 0x7fc424075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60446/0 (socket says 172.21.15.83:60446) 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45796/0 (socket says 172.21.15.83:45796) 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 -- 172.21.15.83:0/1676601072 learned_addr learned my addr 172.21.15.83:0/1676601072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429f05640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 0x7fc424076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 -- 172.21.15.83:0/1676601072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 msgr2=0x7fc424075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 0x7fc424075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 -- 172.21.15.83:0/1676601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 msgr2=0x7fc424076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 0x7fc424076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 -- 172.21.15.83:0/1676601072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4241a8710 con 0x7fc424103bc0 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429f05640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 0x7fc424076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc429704640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9f0 tx=0x7fc42000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc41a7fc640 1 -- 172.21.15.83:0/1676601072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42000cd80 con 0x7fc424103bc0 2024-09-21T03:24:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4241a8a00 con 0x7fc424103bc0 2024-09-21T03:24:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.573+0000 7fc41a7fc640 1 -- 172.21.15.83:0/1676601072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4200054d0 con 0x7fc424103bc0 2024-09-21T03:24:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.574+0000 7fc41a7fc640 1 -- 172.21.15.83:0/1676601072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420010690 con 0x7fc424103bc0 2024-09-21T03:24:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.574+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241a8ef0 con 0x7fc424103bc0 2024-09-21T03:24:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.577+0000 7fc41a7fc640 1 -- 172.21.15.83:0/1676601072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc420010830 con 0x7fc424103bc0 2024-09-21T03:24:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.577+0000 7fc431edc740 1 Processor -- start 2024-09-21T03:24:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.577+0000 7fc431edc740 1 -- start start 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11df800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1155ea0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c4d11452d0 con 0x55c4d0e38070 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc431edc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4d11490e0 con 0x55c4d0e17390 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc429f05640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc429f05640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38432/0 (socket says 172.21.15.83:38432) 2024-09-21T03:24:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc429f05640 1 -- 172.21.15.83:0/3572771254 learned_addr learned my addr 172.21.15.83:0/3572771254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc428f03640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc41a7fc640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 0x7fc4100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc429704640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc41a7fc640 1 -- 172.21.15.83:0/1676601072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc4200e7b00 con 0x7fc424103bc0 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc429f05640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 0x7fc4100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.578+0000 7fc428f03640 1 -- 172.21.15.83:0/3572771254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 msgr2=0x55c4d11df2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc428f03640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc428f03640 1 -- 172.21.15.83:0/3572771254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e0f1b0 msgr2=0x55c4d11df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc428f03640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11df800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc428f03640 1 -- 172.21.15.83:0/3572771254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d0ea3650 con 0x55c4d0e17390 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc429f05640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc428f03640 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11dfd40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc42406b8a0 tx=0x7fc414011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf23d16394838429 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc40f7fe640 1 -- 172.21.15.83:0/3572771254 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414004050 con 0x55c4d0e17390 2024-09-21T03:24:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc429f05640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 0x7fc4100b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc41c035000 tx=0x7fc41c02a8c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc40f7fe640 1 -- 172.21.15.83:0/3572771254 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4140054d0 con 0x55c4d0e17390 2024-09-21T03:24:16.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.579+0000 7fc40f7fe640 1 -- 172.21.15.83:0/3572771254 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414004c60 con 0x55c4d0e17390 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 -- 172.21.15.83:0/3572771254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 msgr2=0x55c4d11dfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11dfd40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc42406b8a0 tx=0x7fc414011810 comp rx=0 tx=0).stop 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 -- 172.21.15.83:0/3572771254 shutdown_connections 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11dfd40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 --2- 172.21.15.83:0/3572771254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 -- 172.21.15.83:0/3572771254 >> 172.21.15.83:0/3572771254 conn(0x55c4d0e2c460 msgr2=0x55c4d11dd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 -- 172.21.15.83:0/3572771254 shutdown_connections 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.580+0000 7fc431edc740 1 -- 172.21.15.83:0/3572771254 wait complete. 2024-09-21T03:24:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.582+0000 7fc431edc740 1 -- start start 2024-09-21T03:24:16.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.582+0000 7fc431edc740 1 Processor -- start 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 0x55c4d11ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c4d0fa6700 con 0x55c4d0e38070 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c4d10e0160 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4d11edaa0 con 0x55c4d0e17390 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc431edc740 10 client.0 _ll_register_callbacks cb 0x55c4d10f99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc429704640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc428f03640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc429704640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38436/0 (socket says 172.21.15.83:38436) 2024-09-21T03:24:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc429704640 1 -- 172.21.15.83:0/421259743 learned_addr learned my addr 172.21.15.83:0/421259743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.583+0000 7fc429f05640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 0x55c4d11ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 -- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 msgr2=0x55c4d11eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 -- 172.21.15.83:0/421259743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 msgr2=0x55c4d11ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 0x55c4d11ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 -- 172.21.15.83:0/421259743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d110dc90 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429f05640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 0x55c4d11ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc428f03640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc429704640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9c0 tx=0x7fc42000a8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b58e04ac2a150b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200ad2f0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc4200ace00 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.584+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4140054d0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc4200a9110 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 10 client.26228 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c4d1201fe0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4140054d0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc4200b0ce0 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 10 client.26228 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 10 client.26228 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 20 client.26228 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0df69f0 0x55c4d12055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 4 client.26228 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d1157df0 con 0x55c4d0df69f0 2024-09-21T03:24:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.585+0000 7fc428f03640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0df69f0 0x55c4d12055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.586+0000 7fc40cff9640 10 client.26228 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.586+0000 7fc428f03640 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0df69f0 0x55c4d12055b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e53d544ea5fbec5 server_cookie=32889e7e84994333 in_seq=0 out_seq=0 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.586+0000 7fc40cff9640 1 -- 172.21.15.83:0/421259743 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c4d1157df0 con 0x55c4d0df69f0 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.586+0000 7fc40cff9640 10 client.26228 handle_command_reply: tid=0 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 client.26228 shutdown 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 msgr2=0x55c4d11ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9c0 tx=0x7fc42000a8a0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 shutdown_connections 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e38070 0x55c4d11ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0df69f0 0x55c4d12055b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e17390 0x55c4d11eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 --2- 172.21.15.83:0/421259743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11ea9a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc40cff9640 0 client.26228 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:24:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 >> 172.21.15.83:0/421259743 conn(0x55c4d0e2c460 msgr2=0x55c4d11e24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 shutdown_connections 2024-09-21T03:24:16.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 1 -- 172.21.15.83:0/421259743 wait complete. 2024-09-21T03:24:16.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.587+0000 7fc431edc740 20 client.26228 trim_cache size 0 max 16384 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 Processor -- start 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 -- start start 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241ad3c0 con 0x55c4d0e17390 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241aef00 con 0x55c4d0e38070 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc431edc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241ada90 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc428f03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc428f03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45828/0 (socket says 172.21.15.83:45828) 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc428f03640 1 -- 172.21.15.83:0/652147762 learned_addr learned my addr 172.21.15.83:0/652147762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc429f05640 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc429704640 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.603+0000 7fc428f03640 1 -- 172.21.15.83:0/652147762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e0f1b0 msgr2=0x55c4d11e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc428f03640 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc428f03640 1 -- 172.21.15.83:0/652147762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 msgr2=0x55c4d11dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc428f03640 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc428f03640 1 -- 172.21.15.83:0/652147762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d11e00d0 con 0x55c4d0e17390 2024-09-21T03:24:16.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc428f03640 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc41401b040 tx=0x7fc414011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ee1471ed48a46f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc40effd640 1 -- 172.21.15.83:0/652147762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41402f020 con 0x55c4d0e17390 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc40effd640 1 -- 172.21.15.83:0/652147762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc414003aa0 con 0x55c4d0e17390 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.604+0000 7fc40effd640 1 -- 172.21.15.83:0/652147762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41401d040 con 0x55c4d0e17390 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- 172.21.15.83:0/652147762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 msgr2=0x55c4d11e0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc41401b040 tx=0x7fc414011810 comp rx=0 tx=0).stop 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- 172.21.15.83:0/652147762 shutdown_connections 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e38070 0x55c4d11dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11e0a00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 --2- 172.21.15.83:0/652147762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- 172.21.15.83:0/652147762 >> 172.21.15.83:0/652147762 conn(0x55c4d0e2c460 msgr2=0x55c4d11e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- 172.21.15.83:0/652147762 shutdown_connections 2024-09-21T03:24:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- 172.21.15.83:0/652147762 wait complete. 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.605+0000 7fc431edc740 1 -- start start 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 Processor -- start 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e38070 0x55c4d11f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241ae830 con 0x55c4d0e17390 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241ae160 con 0x55c4d0e0f1b0 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241ab3c0 con 0x55c4d0e38070 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc431edc740 10 client.0 _ll_register_callbacks cb 0x7fc42000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45834/0 (socket says 172.21.15.83:45834) 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 -- 172.21.15.83:0/2676420343 learned_addr learned my addr 172.21.15.83:0/2676420343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc429f05640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e38070 0x55c4d11f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc429704640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 -- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e38070 msgr2=0x55c4d11f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e38070 0x55c4d11f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 -- 172.21.15.83:0/2676420343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 msgr2=0x55c4d11fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.606+0000 7fc428f03640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc428f03640 1 -- 172.21.15.83:0/2676420343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d126efb0 con 0x55c4d0e17390 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc429704640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc428f03640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc414005dc0 tx=0x7fc414009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbe6dc77f3a2a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41401d040 con 0x55c4d0e17390 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc414003c50 con 0x55c4d0e17390 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc414003aa0 con 0x55c4d0e17390 2024-09-21T03:24:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc414003e50 con 0x55c4d0e17390 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc431edc740 10 client.16680 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c4d12700a0 con 0x55c4d0e17390 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc414003aa0 con 0x55c4d0e17390 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.607+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc414048480 con 0x55c4d0e17390 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 10 client.16680 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 10 client.16680 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 20 client.16680 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0dcbda0 0x55c4d1272590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 4 client.16680 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4d11538a0 con 0x55c4d0dcbda0 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc429704640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0dcbda0 0x55c4d1272590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc40cff9640 10 client.16680 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:24:16.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc429704640 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0dcbda0 0x55c4d1272590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c43be19d5dead59 server_cookie=239f7e8269d7bf6a in_seq=0 out_seq=0 2024-09-21T03:24:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc40cff9640 1 -- 172.21.15.83:0/2676420343 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4d11538a0 con 0x55c4d0dcbda0 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc40cff9640 10 client.16680 handle_command_reply: tid=0 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.608+0000 7fc431edc740 1 client.16680 shutdown 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 msgr2=0x55c4d11f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc414005dc0 tx=0x7fc414009640 comp rx=0 tx=0).stop 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 shutdown_connections 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c4d0dcbda0 0x55c4d1272590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4d0e38070 0x55c4d11f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc40cff9640 0 client.16680 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d0e17390 0x55c4d11f73c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 --2- 172.21.15.83:0/2676420343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c4d0e0f1b0 0x55c4d11fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 >> 172.21.15.83:0/2676420343 conn(0x55c4d0e2c460 msgr2=0x55c4d11ebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 shutdown_connections 2024-09-21T03:24:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 1 -- 172.21.15.83:0/2676420343 wait complete. 2024-09-21T03:24:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.609+0000 7fc431edc740 20 client.16680 trim_cache size 0 max 16384 2024-09-21T03:24:16.616 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 msgr2=0x7fc4100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 0x7fc4100b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc41c035000 tx=0x7fc41c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 msgr2=0x7fc424075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9f0 tx=0x7fc42000edf0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 shutdown_connections 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc4100b6680 0x7fc4100b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc424109e00 0x7fc424076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4241094b0 0x7fc424075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 --2- 172.21.15.83:0/1676601072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc424103bc0 0x7fc424075710 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.611+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 >> 172.21.15.83:0/1676601072 conn(0x7fc42407b910 msgr2=0x7fc424105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.612+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 shutdown_connections 2024-09-21T03:24:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:16.612+0000 7fc42b968640 1 -- 172.21.15.83:0/1676601072 wait complete. 2024-09-21T03:24:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:16 smithi142 ceph-mon[34512]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-09-21T03:24:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:16 smithi193 ceph-mon[35224]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-09-21T03:24:17.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:16 smithi083 ceph-mon[31219]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 665 op/s 2024-09-21T03:24:17.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:24:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 -- 172.21.15.83:0/1168953019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0010faf0 msgr2=0x7fcc0010ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 --2- 172.21.15.83:0/1168953019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0010faf0 0x7fcc0010ff00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf802a410 comp rx=0 tx=0).stop 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 -- 172.21.15.83:0/1168953019 shutdown_connections 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 --2- 172.21.15.83:0/1168953019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc00110440 0x7fcc00116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 --2- 172.21.15.83:0/1168953019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0010faf0 0x7fcc0010ff00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 --2- 172.21.15.83:0/1168953019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc0010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 -- 172.21.15.83:0/1168953019 >> 172.21.15.83:0/1168953019 conn(0x7fcc000780e0 msgr2=0x7fcc001081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 -- 172.21.15.83:0/1168953019 shutdown_connections 2024-09-21T03:24:18.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.465+0000 7fcc07514640 1 -- 172.21.15.83:0/1168953019 wait complete. 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.466+0000 7fcc07514640 1 Processor -- start 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.466+0000 7fcc07514640 1 -- start start 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc0010faf0 0x7fcc001a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc00110440 0x7fcc001a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0011a0d0 con 0x7fcc0010faf0 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00119f50 con 0x7fcc0010a1b0 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc07514640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0011a250 con 0x7fcc00110440 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc04aaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc0010faf0 0x7fcc001a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38492/0 (socket says 172.21.15.83:38492) 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 -- 172.21.15.83:0/3810334331 learned_addr learned my addr 172.21.15.83:0/3810334331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc00110440 0x7fcc001a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 -- 172.21.15.83:0/3810334331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc00110440 msgr2=0x7fcc001a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc00110440 0x7fcc001a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 -- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc0010faf0 msgr2=0x7fcc001a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc0010faf0 0x7fcc001a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 -- 172.21.15.83:0/3810334331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a6ad0 con 0x7fcc0010a1b0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.467+0000 7fcc052b0640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00eb90 tx=0x7fcbfc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.468+0000 7fcbf67fc640 1 -- 172.21.15.83:0/3810334331 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004b20 con 0x7fcc0010a1b0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.468+0000 7fcbf67fc640 1 -- 172.21.15.83:0/3810334331 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc004cc0 con 0x7fcc0010a1b0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.468+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a6dc0 con 0x7fcc0010a1b0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.468+0000 7fcbf67fc640 1 -- 172.21.15.83:0/3810334331 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc010690 con 0x7fcc0010a1b0 2024-09-21T03:24:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.468+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001ad520 con 0x7fcc0010a1b0 2024-09-21T03:24:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcbf67fc640 1 -- 172.21.15.83:0/3810334331 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcbfc010830 con 0x7fcc0010a1b0 2024-09-21T03:24:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 Processor -- start 2024-09-21T03:24:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 -- start start 2024-09-21T03:24:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56045f755ea0 con 0x56045f40f1b0 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56045f7452d0 con 0x56045f417390 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc0da88740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56045f7490e0 con 0x56045f438070 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc052b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60526/0 (socket says 172.21.15.83:60526) 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc05ab1640 1 -- 172.21.15.83:0/262016127 learned_addr learned my addr 172.21.15.83:0/262016127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.471+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 -- 172.21.15.83:0/262016127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 msgr2=0x56045f7e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 -- 172.21.15.83:0/262016127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 msgr2=0x56045f7dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7dfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 -- 172.21.15.83:0/262016127 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56045f4a3650 con 0x56045f438070 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc052b0640 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009c10 tx=0x7fcbf0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9bd3a7a05a220fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcbf67fc640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 0x7fcbec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcbeb7fe640 1 -- 172.21.15.83:0/262016127 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0004230 con 0x56045f438070 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcbeb7fe640 1 -- 172.21.15.83:0/262016127 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf00043d0 con 0x56045f438070 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcbeb7fe640 1 -- 172.21.15.83:0/262016127 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0019620 con 0x56045f438070 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.472+0000 7fcc052b0640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 0x7fcbec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.473+0000 7fcbf67fc640 1 -- 172.21.15.83:0/3810334331 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fcbfc0e7e00 con 0x7fcc0010a1b0 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.473+0000 7fcc0da88740 1 -- 172.21.15.83:0/262016127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 msgr2=0x56045f7df6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.473+0000 7fcc0da88740 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009c10 tx=0x7fcbf0011810 comp rx=0 tx=0).stop 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.474+0000 7fcc0da88740 1 -- 172.21.15.83:0/262016127 shutdown_connections 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.474+0000 7fcc0da88740 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.474+0000 7fcc0da88740 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.474+0000 7fcc0da88740 1 --2- 172.21.15.83:0/262016127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f438070 0x56045f7df6a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.474+0000 7fcc0da88740 1 -- 172.21.15.83:0/262016127 >> 172.21.15.83:0/262016127 conn(0x56045f42c460 msgr2=0x56045f7dd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.476+0000 7fcc0da88740 1 -- 172.21.15.83:0/262016127 shutdown_connections 2024-09-21T03:24:18.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.476+0000 7fcc052b0640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 0x7fcbec0b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56045f7de320 tx=0x7fcbfc00cea0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.476+0000 7fcc0da88740 1 -- 172.21.15.83:0/262016127 wait complete. 2024-09-21T03:24:18.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.477+0000 7fcc0da88740 1 -- start start 2024-09-21T03:24:18.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.477+0000 7fcc0da88740 1 Processor -- start 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60530/0 (socket says 172.21.15.83:60530) 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc05ab1640 1 -- 172.21.15.83:0/1563475293 learned_addr learned my addr 172.21.15.83:0/1563475293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56045f5a6700 con 0x56045f438070 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56045f6e0160 con 0x56045f417390 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.478+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56045f7ede80 con 0x56045f40f1b0 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc0da88740 10 client.0 _ll_register_callbacks cb 0x56045f6f99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 -- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 msgr2=0x56045f7eb2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7eb2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 -- 172.21.15.83:0/1563475293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 msgr2=0x56045f7f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045f70dc90 con 0x56045f40f1b0 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00098b0 tx=0x7fcbf00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f44558e618b0531 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf002e060 con 0x56045f40f1b0 2024-09-21T03:24:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.479+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fcbf0003b50 con 0x56045f40f1b0 2024-09-21T03:24:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf00043d0 con 0x56045f40f1b0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcbf0029480 con 0x56045f40f1b0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 10 client.26224 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56045f7f9b90 con 0x56045f40f1b0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0029ad0 con 0x56045f40f1b0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcbf0025050 con 0x56045f40f1b0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 10 client.26224 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 10 client.26224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 20 client.26224 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3f69f0 0x56045f7fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 4 client.26224 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.480+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcbf00043d0 con 0x56045f3f69f0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.481+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3f69f0 0x56045f7fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.481+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3f69f0 0x56045f7fcfa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75b50f572d31f6c7 server_cookie=3e9689509fc8e7ea in_seq=0 out_seq=0 2024-09-21T03:24:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.481+0000 7fcbe8ff9640 10 client.26224 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:24:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.482+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/1563475293 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcbf00043d0 con 0x56045f3f69f0 2024-09-21T03:24:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.482+0000 7fcbe8ff9640 10 client.26224 handle_command_reply: tid=0 2024-09-21T03:24:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.482+0000 7fcc0da88740 1 client.26224 shutdown 2024-09-21T03:24:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 msgr2=0x56045f7ead80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00098b0 tx=0x7fcbf00056d0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 shutdown_connections 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3f69f0 0x56045f7fcfa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f417390 0x56045f7eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1563475293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f40f1b0 0x56045f7ead80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 >> 172.21.15.83:0/1563475293 conn(0x56045f42c460 msgr2=0x56045f7e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.483+0000 7fcbe8ff9640 0 client.26224 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.484+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 shutdown_connections 2024-09-21T03:24:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.484+0000 7fcc0da88740 1 -- 172.21.15.83:0/1563475293 wait complete. 2024-09-21T03:24:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.485+0000 7fcc0da88740 20 client.26224 trim_cache size 0 max 16384 2024-09-21T03:24:18.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 Processor -- start 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 -- start start 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f40f1b0 0x56045f7eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b5c80 con 0x56045f438070 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b77c0 con 0x56045f40f1b0 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.507+0000 7fcc0da88740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b6350 con 0x56045f417390 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:60534/0 (socket says 172.21.15.83:60534) 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 -- 172.21.15.83:0/1478262872 learned_addr learned my addr 172.21.15.83:0/1478262872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45884/0 (socket says 172.21.15.83:45884) 2024-09-21T03:24:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f40f1b0 0x56045f7eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 -- 172.21.15.83:0/1478262872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f40f1b0 msgr2=0x56045f7eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f40f1b0 0x56045f7eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 -- 172.21.15.83:0/1478262872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 msgr2=0x56045f7dcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 -- 172.21.15.83:0/1478262872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56045f7e2500 con 0x56045f417390 2024-09-21T03:24:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8005ab0 tx=0x7fcbf8005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32688b579186c4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcbeaffd640 1 -- 172.21.15.83:0/1478262872 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8035040 con 0x56045f417390 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.508+0000 7fcbeaffd640 1 -- 172.21.15.83:0/1478262872 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf8005020 con 0x56045f417390 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.509+0000 7fcbeaffd640 1 -- 172.21.15.83:0/1478262872 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8004920 con 0x56045f417390 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 -- 172.21.15.83:0/1478262872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 msgr2=0x56045f7f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8005ab0 tx=0x7fcbf8005a20 comp rx=0 tx=0).stop 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 -- 172.21.15.83:0/1478262872 shutdown_connections 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f438070 0x56045f7dcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7f70d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 --2- 172.21.15.83:0/1478262872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f40f1b0 0x56045f7eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.510+0000 7fcc0da88740 1 -- 172.21.15.83:0/1478262872 >> 172.21.15.83:0/1478262872 conn(0x56045f42c460 msgr2=0x56045f7eab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.511+0000 7fcc0da88740 1 -- 172.21.15.83:0/1478262872 shutdown_connections 2024-09-21T03:24:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.511+0000 7fcc0da88740 1 -- 172.21.15.83:0/1478262872 wait complete. 2024-09-21T03:24:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.512+0000 7fcc0da88740 1 -- start start 2024-09-21T03:24:18.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.512+0000 7fcc0da88740 1 Processor -- start 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 0x56045f7ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b70f0 con 0x56045f40f1b0 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b6a20 con 0x56045f438070 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001b3c80 con 0x56045f417390 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc0da88740 10 client.0 _ll_register_callbacks cb 0x7fcbf0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc04aaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc05ab1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45896/0 (socket says 172.21.15.83:45896) 2024-09-21T03:24:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc05ab1640 1 -- 172.21.15.83:0/2475797888 learned_addr learned my addr 172.21.15.83:0/2475797888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.513+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 0x56045f7ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 -- 172.21.15.83:0/2475797888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 msgr2=0x56045f7eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 -- 172.21.15.83:0/2475797888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 msgr2=0x56045f7f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 -- 172.21.15.83:0/2475797888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045f7ef6f0 con 0x56045f438070 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcc05ab1640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 0x56045f7ef130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56045f7ed260 tx=0x7fcbf0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31804d53f6a90c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf002e040 con 0x56045f438070 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.514+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fcbf0003b50 con 0x56045f438070 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.515+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf8005020 con 0x56045f438070 2024-09-21T03:24:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.515+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcbf0003d50 con 0x56045f438070 2024-09-21T03:24:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.515+0000 7fcc0da88740 10 client.26249 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.515+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56045f867510 con 0x56045f438070 2024-09-21T03:24:18.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.515+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0048480 con 0x56045f438070 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcbf0025040 con 0x56045f438070 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 10 client.26249 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 10 client.26249 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 20 client.26249 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3cbda0 0x56045f869bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 4 client.26249 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcbf8005020 con 0x56045f3cbda0 2024-09-21T03:24:18.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.517+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3cbda0 0x56045f869bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcbe8ff9640 10 client.26249 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc04aaf640 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3cbda0 0x56045f869bc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a572cba991695265 server_cookie=96a5a1ee4dc04083 in_seq=0 out_seq=0 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcbe8ff9640 1 -- 172.21.15.83:0/2475797888 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcbf8005020 con 0x56045f3cbda0 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcbe8ff9640 10 client.26249 handle_command_reply: tid=0 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 client.26249 shutdown 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 msgr2=0x56045f7ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 0x56045f7ef130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56045f7ed260 tx=0x7fcbf0011810 comp rx=0 tx=0).stop 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 shutdown_connections 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56045f3cbda0 0x56045f869bc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcbe8ff9640 0 client.26249 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56045f438070 0x56045f7ef130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56045f417390 0x56045f7eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 --2- 172.21.15.83:0/2475797888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56045f40f1b0 0x56045f7f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 >> 172.21.15.83:0/2475797888 conn(0x56045f42c460 msgr2=0x56045f7ec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 shutdown_connections 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.518+0000 7fcc0da88740 1 -- 172.21.15.83:0/2475797888 wait complete. 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.519+0000 7fcc0da88740 20 client.26249 trim_cache size 0 max 16384 2024-09-21T03:24:18.521 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:24:18.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.520+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 msgr2=0x7fcbec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.520+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 0x7fcbec0b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56045f7de320 tx=0x7fcbfc00cea0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.520+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 msgr2=0x7fcc001a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:18.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.520+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00eb90 tx=0x7fcbfc00ef90 comp rx=0 tx=0).stop 2024-09-21T03:24:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 shutdown_connections 2024-09-21T03:24:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcbec0b6680 0x7fcbec0b8b90 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56045f7de320 tx=0x7fcbfc00cea0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc00110440 0x7fcc001a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc0010faf0 0x7fcc001a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:18.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 --2- 172.21.15.83:0/3810334331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0010a1b0 0x7fcc001a0b60 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00eb90 tx=0x7fcbfc00ef90 comp rx=0 tx=0).stop 2024-09-21T03:24:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.521+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 >> 172.21.15.83:0/3810334331 conn(0x7fcc000780e0 msgr2=0x7fcc001086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.522+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 shutdown_connections 2024-09-21T03:24:18.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:18.522+0000 7fcc07514640 1 -- 172.21.15.83:0/3810334331 wait complete. 2024-09-21T03:24:19.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:18 smithi083 ceph-mon[31219]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.4 MiB/s wr, 646 op/s 2024-09-21T03:24:19.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:24:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:18 smithi193 ceph-mon[35224]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.4 MiB/s wr, 646 op/s 2024-09-21T03:24:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:18 smithi142 ceph-mon[34512]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.4 MiB/s wr, 646 op/s 2024-09-21T03:24:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 -- 172.21.15.83:0/3540687559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb410de10 msgr2=0x7f6bb4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/3540687559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb410de10 0x7f6bb4110300 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009ca0 tx=0x7f6bac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 -- 172.21.15.83:0/3540687559 shutdown_connections 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/3540687559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb410de10 0x7f6bb4110300 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/3540687559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb4105300 0x7f6bb410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/3540687559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb41049b0 0x7f6bb4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 -- 172.21.15.83:0/3540687559 >> 172.21.15.83:0/3540687559 conn(0x7f6bb4079730 msgr2=0x7f6bb4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 -- 172.21.15.83:0/3540687559 shutdown_connections 2024-09-21T03:24:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.160+0000 7f6bbc5da640 1 -- 172.21.15.83:0/3540687559 wait complete. 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 Processor -- start 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 -- start start 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb4105300 0x7f6bb419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb410de10 0x7f6bb419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113a90 con 0x7f6bb41049b0 2024-09-21T03:24:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113910 con 0x7f6bb410de10 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbc5da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113c10 con 0x7f6bb4105300 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44884/0 (socket says 172.21.15.83:44884) 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 -- 172.21.15.83:0/2913186883 learned_addr learned my addr 172.21.15.83:0/2913186883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb4105300 0x7f6bb419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bbab77640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb410de10 0x7f6bb419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 -- 172.21.15.83:0/2913186883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb4105300 msgr2=0x7f6bb419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb4105300 0x7f6bb419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 -- 172.21.15.83:0/2913186883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb410de10 msgr2=0x7f6bb419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb410de10 0x7f6bb419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.161+0000 7f6bba376640 1 -- 172.21.15.83:0/2913186883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb41a03d0 con 0x7f6bb41049b0 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bba376640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e9c0 tx=0x7f6bb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bab7fe640 1 -- 172.21.15.83:0/2913186883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000cd80 con 0x7f6bb41049b0 2024-09-21T03:24:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bab7fe640 1 -- 172.21.15.83:0/2913186883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00035f0 con 0x7f6bb41049b0 2024-09-21T03:24:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bab7fe640 1 -- 172.21.15.83:0/2913186883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0010690 con 0x7f6bb41049b0 2024-09-21T03:24:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb41a06c0 con 0x7f6bb41049b0 2024-09-21T03:24:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.162+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41a6e20 con 0x7f6bb41049b0 2024-09-21T03:24:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.166+0000 7f6bc2b4e740 1 Processor -- start 2024-09-21T03:24:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 -- start start 2024-09-21T03:24:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bab7fe640 1 -- 172.21.15.83:0/2913186883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6bb0004000 con 0x7f6bb41049b0 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570bc1070 0x55d570f686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570b981b0 0x55d570f68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d570edeea0 con 0x55d570ba0390 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d570ece2d0 con 0x55d570b981b0 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d570ed20e0 con 0x55d570bc1070 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44888/0 (socket says 172.21.15.83:44888) 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 -- 172.21.15.83:0/326501292 learned_addr learned my addr 172.21.15.83:0/326501292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bbab77640 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570bc1070 0x55d570f686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bba376640 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570b981b0 0x55d570f68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 -- 172.21.15.83:0/326501292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570bc1070 msgr2=0x55d570f686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570bc1070 0x55d570f686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 -- 172.21.15.83:0/326501292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570b981b0 msgr2=0x55d570f68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570b981b0 0x55d570f68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.167+0000 7f6bb9b75640 1 -- 172.21.15.83:0/326501292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d570c2c650 con 0x55d570ba0390 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009c20 tx=0x7f6ba4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a62370322a2c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6bab7fe640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 0x7f6ba00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6ba8ff9640 1 -- 172.21.15.83:0/326501292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004230 con 0x55d570ba0390 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6bbab77640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 0x7f6ba00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6bab7fe640 1 -- 172.21.15.83:0/2913186883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f6bb00e7c00 con 0x7f6bb41049b0 2024-09-21T03:24:20.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6ba8ff9640 1 -- 172.21.15.83:0/326501292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba40043d0 con 0x55d570ba0390 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.168+0000 7f6ba8ff9640 1 -- 172.21.15.83:0/326501292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004d50 con 0x55d570ba0390 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.169+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/326501292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 msgr2=0x55d570f69120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.169+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009c20 tx=0x7f6ba4011810 comp rx=0 tx=0).stop 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bbab77640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 0x7f6ba00b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02f500 tx=0x7f6bac02f310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/326501292 shutdown_connections 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f69120 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570b981b0 0x55d570f68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/326501292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570bc1070 0x55d570f686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/326501292 >> 172.21.15.83:0/326501292 conn(0x55d570bb5460 msgr2=0x55d570f66590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/326501292 shutdown_connections 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/326501292 wait complete. 2024-09-21T03:24:20.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 -- start start 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.170+0000 7f6bc2b4e740 1 Processor -- start 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570f80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d570d2f700 con 0x55d570ba0390 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d570e69160 con 0x55d570bc1070 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d570f76e80 con 0x55d570b981b0 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bb9b75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bc2b4e740 10 client.0 _ll_register_callbacks cb 0x55d570e829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53118/0 (socket says 172.21.15.83:53118) 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 -- 172.21.15.83:0/2921179315 learned_addr learned my addr 172.21.15.83:0/2921179315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bb9b75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44892/0 (socket says 172.21.15.83:44892) 2024-09-21T03:24:20.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bbab77640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570f80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 -- 172.21.15.83:0/2921179315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 msgr2=0x55d570f80170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570f80170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 -- 172.21.15.83:0/2921179315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 msgr2=0x55d570f742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.171+0000 7f6bba376640 1 -- 172.21.15.83:0/2921179315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d570e96c90 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6bba376640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00ad7a0 tx=0x7f6bb00aa210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d607484199910d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ad3f0 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f6bb00b21c0 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba40043d0 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6bb00ad210 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6bc2b4e740 10 client.26242 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d570f82c00 con 0x55d570b981b0 2024-09-21T03:24:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.172+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba40043d0 con 0x55d570b981b0 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f6bb00ef040 con 0x55d570b981b0 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 10 client.26242 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 10 client.26242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 20 client.26242 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b7f9f0 0x55d570f86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 4 client.26242 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d570ee0df0 con 0x55d570b7f9f0 2024-09-21T03:24:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.173+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b7f9f0 0x55d570f86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.174+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b7f9f0 0x55d570f86060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb4bffaa07ceb273 server_cookie=3103dd7f2576b949 in_seq=0 out_seq=0 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.174+0000 7f6b9dffb640 10 client.26242 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.175+0000 7f6b9dffb640 1 -- 172.21.15.83:0/2921179315 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d570ee0df0 con 0x55d570b7f9f0 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.175+0000 7f6b9dffb640 10 client.26242 handle_command_reply: tid=0 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.175+0000 7f6bc2b4e740 1 client.26242 shutdown 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.175+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 msgr2=0x55d570f73d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.175+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00ad7a0 tx=0x7f6bb00aa210 comp rx=0 tx=0).stop 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 shutdown_connections 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b7f9f0 0x55d570f86060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570f80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6b9dffb640 0 client.26242 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:24:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/2921179315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f73d70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 >> 172.21.15.83:0/2921179315 conn(0x55d570bb5460 msgr2=0x55d570f6b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 shutdown_connections 2024-09-21T03:24:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/2921179315 wait complete. 2024-09-21T03:24:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.176+0000 7f6bc2b4e740 20 client.26242 trim_cache size 0 max 16384 2024-09-21T03:24:20.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 Processor -- start 2024-09-21T03:24:20.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 -- start start 2024-09-21T03:24:20.195 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:19 smithi083 ceph-mon[31219]: pgmap v559: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-21T03:24:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f69de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 0x55d570f65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41af540 con 0x55d570bc1070 2024-09-21T03:24:20.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b1080 con 0x55d570ba0390 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41afc10 con 0x55d570b981b0 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bbab77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bbab77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44904/0 (socket says 172.21.15.83:44904) 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bbab77640 1 -- 172.21.15.83:0/356399888 learned_addr learned my addr 172.21.15.83:0/356399888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.192+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 0x55d570f65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bba376640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f69de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 -- 172.21.15.83:0/356399888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 msgr2=0x55d570f69de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f69de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 -- 172.21.15.83:0/356399888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 msgr2=0x55d570f65bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 0x55d570f65bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 -- 172.21.15.83:0/356399888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d570f72270 con 0x55d570bc1070 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 0x55d570f65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:20.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6bbab77640 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bac035000 tx=0x7f6bac044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f68eed2c693570 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6b9ffff640 1 -- 172.21.15.83:0/356399888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac003970 con 0x55d570bc1070 2024-09-21T03:24:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6b9ffff640 1 -- 172.21.15.83:0/356399888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bac036050 con 0x55d570bc1070 2024-09-21T03:24:20.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.193+0000 7f6b9ffff640 1 -- 172.21.15.83:0/356399888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac02f8c0 con 0x55d570bc1070 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/356399888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 msgr2=0x55d570f66100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bac035000 tx=0x7f6bac044000 comp rx=0 tx=0).stop 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/356399888 shutdown_connections 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570bc1070 0x55d570f66100 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570ba0390 0x55d570f65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/356399888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f69de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/356399888 >> 172.21.15.83:0/356399888 conn(0x55d570bb5460 msgr2=0x55d570f696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/356399888 shutdown_connections 2024-09-21T03:24:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.194+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/356399888 wait complete. 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 -- start start 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 Processor -- start 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f78920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570fefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b09b0 con 0x55d570ba0390 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b02e0 con 0x55d570bc1070 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41ad540 con 0x55d570b981b0 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.195+0000 7f6bc2b4e740 10 client.0 _ll_register_callbacks cb 0x7f6bb0003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53146/0 (socket says 172.21.15.83:53146) 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 -- 172.21.15.83:0/3535747089 learned_addr learned my addr 172.21.15.83:0/3535747089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:20.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f78920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 -- 172.21.15.83:0/3535747089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 msgr2=0x55d570fefa70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570fefa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.196+0000 7f6bba376640 1 -- 172.21.15.83:0/3535747089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 msgr2=0x55d570f78920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.197+0000 7f6bba376640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f78920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.197+0000 7f6bba376640 1 -- 172.21.15.83:0/3535747089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d570f791f0 con 0x55d570b981b0 2024-09-21T03:24:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6bba376640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004400 tx=0x7f6bb000c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea49ed680dde9a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ad3f0 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f6bb00e5cf0 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bac036050 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0004480 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6bc2b4e740 10 client.26248 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.198+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d570ff1030 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bac036050 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f6bb00cfab0 con 0x55d570b981b0 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 10 client.26248 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 10 client.26248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 20 client.26248 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:20.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b54da0 0x55d570ff36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:20.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 4 client.26248 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:24:20.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.199+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d570edc8a0 con 0x55d570b54da0 2024-09-21T03:24:20.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.200+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b54da0 0x55d570ff36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.200+0000 7f6b9dffb640 10 client.26248 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.200+0000 7f6bb9b75640 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b54da0 0x55d570ff36b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3b747849c962eec server_cookie=e6ea12650716bf31 in_seq=0 out_seq=0 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6b9dffb640 1 -- 172.21.15.83:0/3535747089 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d570edc8a0 con 0x55d570b54da0 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6b9dffb640 10 client.26248 handle_command_reply: tid=0 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 client.26248 shutdown 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 msgr2=0x55d570f783e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004400 tx=0x7f6bb000c840 comp rx=0 tx=0).stop 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 shutdown_connections 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55d570b54da0 0x55d570ff36b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d570bc1070 0x55d570fefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d570ba0390 0x55d570f78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6b9dffb640 0 client.26248 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 --2- 172.21.15.83:0/3535747089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d570b981b0 0x55d570f783e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.201+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 >> 172.21.15.83:0/3535747089 conn(0x55d570bb5460 msgr2=0x55d570f696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 shutdown_connections 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bc2b4e740 1 -- 172.21.15.83:0/3535747089 wait complete. 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bc2b4e740 20 client.26248 trim_cache size 0 max 16384 2024-09-21T03:24:20.206 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 msgr2=0x7f6ba00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 0x7f6ba00b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6bac02f500 tx=0x7f6bac02f310 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 msgr2=0x7f6bb419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e9c0 tx=0x7f6bb000ee80 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 shutdown_connections 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6ba00b6680 0x7f6ba00b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bb410de10 0x7f6bb419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bb4105300 0x7f6bb419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.202+0000 7f6bbc5da640 1 --2- 172.21.15.83:0/2913186883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb41049b0 0x7f6bb419a460 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.203+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 >> 172.21.15.83:0/2913186883 conn(0x7f6bb4079730 msgr2=0x7f6bb40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.203+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 shutdown_connections 2024-09-21T03:24:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:20.203+0000 7f6bbc5da640 1 -- 172.21.15.83:0/2913186883 wait complete. 2024-09-21T03:24:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:19 smithi193 ceph-mon[35224]: pgmap v559: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-21T03:24:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:19 smithi142 ceph-mon[34512]: pgmap v559: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-21T03:24:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 420.0 s, 1814.0 tps, lat 54.567 ms stddev 59.128 2024-09-21T03:24:20.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:24:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:21.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:21 smithi083 ceph-mon[31219]: pgmap v560: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.3 MiB/s wr, 630 op/s 2024-09-21T03:24:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:21 smithi193 ceph-mon[35224]: pgmap v560: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.3 MiB/s wr, 630 op/s 2024-09-21T03:24:21.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:21 smithi142 ceph-mon[34512]: pgmap v560: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.3 MiB/s wr, 630 op/s 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.194+0000 7ff139ffb640 1 -- 172.21.15.83:0/993638728 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134040c10 con 0x7ff13c107be0 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.194+0000 7ff141716640 1 -- 172.21.15.83:0/993638728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c107be0 msgr2=0x7ff13c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.194+0000 7ff141716640 1 --2- 172.21.15.83:0/993638728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c107be0 0x7ff13c1080c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff13401ace0 tx=0x7ff13403b970 comp rx=0 tx=0).stop 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 -- 172.21.15.83:0/993638728 shutdown_connections 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 --2- 172.21.15.83:0/993638728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c107be0 0x7ff13c1080c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 --2- 172.21.15.83:0/993638728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 --2- 172.21.15.83:0/993638728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c1027a0 0x7ff13c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 -- 172.21.15.83:0/993638728 >> 172.21.15.83:0/993638728 conn(0x7ff13c079730 msgr2=0x7ff13c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 -- 172.21.15.83:0/993638728 shutdown_connections 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.195+0000 7ff141716640 1 -- 172.21.15.83:0/993638728 wait complete. 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.196+0000 7ff141716640 1 Processor -- start 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.196+0000 7ff141716640 1 -- start start 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c1027a0 0x7ff13c074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10b4e0 con 0x7ff13c1027a0 2024-09-21T03:24:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10b360 con 0x7ff13c107be0 2024-09-21T03:24:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.197+0000 7ff141716640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10b660 con 0x7ff13c1030f0 2024-09-21T03:24:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.198+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.198+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46500/0 (socket says 172.21.15.83:46500) 2024-09-21T03:24:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.198+0000 7ff13b7fe640 1 -- 172.21.15.83:0/1798515498 learned_addr learned my addr 172.21.15.83:0/1798515498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.198+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 -- 172.21.15.83:0/1798515498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 msgr2=0x7ff13c0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 -- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c1027a0 msgr2=0x7ff13c074320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c1027a0 0x7ff13c074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 -- 172.21.15.83:0/1798515498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c10f7e0 con 0x7ff13c107be0 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff13403be90 tx=0x7ff134041860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff123fff640 1 -- 172.21.15.83:0/1798515498 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134045030 con 0x7ff13c107be0 2024-09-21T03:24:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff123fff640 1 -- 172.21.15.83:0/1798515498 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff13404f070 con 0x7ff13c107be0 2024-09-21T03:24:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.199+0000 7ff123fff640 1 -- 172.21.15.83:0/1798515498 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134004330 con 0x7ff13c107be0 2024-09-21T03:24:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.200+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c10fa70 con 0x7ff13c107be0 2024-09-21T03:24:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.200+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c10ff80 con 0x7ff13c107be0 2024-09-21T03:24:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.204+0000 7ff123fff640 1 -- 172.21.15.83:0/1798515498 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff134048020 con 0x7ff13c107be0 2024-09-21T03:24:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.204+0000 7ff147c8a740 1 Processor -- start 2024-09-21T03:24:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.204+0000 7ff147c8a740 1 -- start start 2024-09-21T03:24:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9a2390 0x56355ed6b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56355ece0ea0 con 0x56355e9c3070 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56355ecd02d0 con 0x56355e9a2390 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff147c8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355ecd40e0 con 0x56355e99a1b0 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff13a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff13a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44944/0 (socket says 172.21.15.83:44944) 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff13a7fc640 1 -- 172.21.15.83:0/1171594939 learned_addr learned my addr 172.21.15.83:0/1171594939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.205+0000 7ff13affd640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9a2390 0x56355ed6b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 -- 172.21.15.83:0/1171594939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 msgr2=0x56355ed6abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed6abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 -- 172.21.15.83:0/1171594939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9a2390 msgr2=0x56355ed6b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9a2390 0x56355ed6b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 -- 172.21.15.83:0/1171594939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355ea2e650 con 0x56355e9c3070 2024-09-21T03:24:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff123fff640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 0x7ff1240b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13affd640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff123fff640 1 -- 172.21.15.83:0/1798515498 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff1341192b0 con 0x7ff13c107be0 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13affd640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 0x7ff1240b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c70 tx=0x7ff12c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74665b9806a1d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff1217fa640 1 -- 172.21.15.83:0/1171594939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01b070 con 0x56355e9c3070 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff1217fa640 1 -- 172.21.15.83:0/1171594939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c009e20 con 0x56355e9c3070 2024-09-21T03:24:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.206+0000 7ff1217fa640 1 -- 172.21.15.83:0/1171594939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00f040 con 0x56355e9c3070 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.207+0000 7ff147c8a740 1 -- 172.21.15.83:0/1171594939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 msgr2=0x56355ed6a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.207+0000 7ff13affd640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 0x7ff1240b8ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56355ed69320 tx=0x7ff130005d70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.207+0000 7ff147c8a740 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c70 tx=0x7ff12c009110 comp rx=0 tx=0).stop 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 -- 172.21.15.83:0/1171594939 shutdown_connections 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9a2390 0x56355ed6b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 --2- 172.21.15.83:0/1171594939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9c3070 0x56355ed6a6a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 -- 172.21.15.83:0/1171594939 >> 172.21.15.83:0/1171594939 conn(0x56355e9b7460 msgr2=0x56355ed68590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 -- 172.21.15.83:0/1171594939 shutdown_connections 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 -- 172.21.15.83:0/1171594939 wait complete. 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.208+0000 7ff147c8a740 1 -- start start 2024-09-21T03:24:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff147c8a740 1 Processor -- start 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff13affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff13affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53162/0 (socket says 172.21.15.83:53162) 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.209+0000 7ff13affd640 1 -- 172.21.15.83:0/114904897 learned_addr learned my addr 172.21.15.83:0/114904897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed82170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56355eb31700 con 0x56355e9a2390 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56355ec6b160 con 0x56355e9c3070 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355ed78e80 con 0x56355e99a1b0 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff147c8a740 10 client.0 _ll_register_callbacks cb 0x56355ec849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 -- 172.21.15.83:0/114904897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 msgr2=0x56355ed75d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 -- 172.21.15.83:0/114904897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 msgr2=0x56355ed762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355ec98c90 con 0x56355e9c3070 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13affd640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.210+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed82170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1341169f0 tx=0x7ff1340dea40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75598017b86d0f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340de550 con 0x56355e9c3070 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff1340ca1c0 con 0x56355e9c3070 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c009e20 con 0x56355e9c3070 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff1340dec00 con 0x56355e9c3070 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff147c8a740 10 client.26270 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56355ed84b00 con 0x56355e9c3070 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c009e20 con 0x56355e9c3070 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.211+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff13405c030 con 0x56355e9c3070 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 10 client.26270 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 10 client.26270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 20 client.26270 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e9819f0 0x56355ed88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 4 client.26270 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355ece2df0 con 0x56355e9819f0 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff13affd640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e9819f0 0x56355ed88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff11eff5640 10 client.26270 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.212+0000 7ff13affd640 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e9819f0 0x56355ed88030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63a31c9f44f8a80 server_cookie=5b0bb2a24dcc2e76 in_seq=0 out_seq=0 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff11eff5640 1 -- 172.21.15.83:0/114904897 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56355ece2df0 con 0x56355e9819f0 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff11eff5640 10 client.26270 handle_command_reply: tid=0 2024-09-21T03:24:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 client.26270 shutdown 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 msgr2=0x56355ed82170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed82170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1341169f0 tx=0x7ff1340dea40 comp rx=0 tx=0).stop 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 shutdown_connections 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e9819f0 0x56355ed88030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff11eff5640 0 client.26270 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed82170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 --2- 172.21.15.83:0/114904897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.213+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 >> 172.21.15.83:0/114904897 conn(0x56355e9b7460 msgr2=0x56355ed6d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.214+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 shutdown_connections 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.214+0000 7ff147c8a740 1 -- 172.21.15.83:0/114904897 wait complete. 2024-09-21T03:24:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.214+0000 7ff147c8a740 20 client.26270 trim_cache size 0 max 16384 2024-09-21T03:24:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.230+0000 7ff147c8a740 1 Processor -- start 2024-09-21T03:24:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.230+0000 7ff147c8a740 1 -- start start 2024-09-21T03:24:22.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed72f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b5ba0 con 0x56355e9a2390 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b76e0 con 0x56355e9c3070 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff147c8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b6270 con 0x56355e99a1b0 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44962/0 (socket says 172.21.15.83:44962) 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 -- 172.21.15.83:0/174114821 learned_addr learned my addr 172.21.15.83:0/174114821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13affd640 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed72f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 -- 172.21.15.83:0/174114821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 msgr2=0x56355ed76230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed76230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 -- 172.21.15.83:0/174114821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 msgr2=0x56355ed72f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed72f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.231+0000 7ff13a7fc640 1 -- 172.21.15.83:0/174114821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355ed6c990 con 0x56355e9a2390 2024-09-21T03:24:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.232+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9d0 tx=0x7ff12c009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8473a06974bd9c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.232+0000 7ff120ff9640 1 -- 172.21.15.83:0/174114821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c027030 con 0x56355e9a2390 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.232+0000 7ff120ff9640 1 -- 172.21.15.83:0/174114821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c01c040 con 0x56355e9a2390 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.232+0000 7ff120ff9640 1 -- 172.21.15.83:0/174114821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01b040 con 0x56355e9a2390 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.233+0000 7ff147c8a740 1 -- 172.21.15.83:0/174114821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 msgr2=0x56355ed82070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.233+0000 7ff147c8a740 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9d0 tx=0x7ff12c009180 comp rx=0 tx=0).stop 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.233+0000 7ff147c8a740 1 -- 172.21.15.83:0/174114821 shutdown_connections 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.233+0000 7ff147c8a740 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed72f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.233+0000 7ff147c8a740 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e9a2390 0x56355ed82070 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 --2- 172.21.15.83:0/174114821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e99a1b0 0x56355ed76230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 -- 172.21.15.83:0/174114821 >> 172.21.15.83:0/174114821 conn(0x56355e9b7460 msgr2=0x56355ed75b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 -- 172.21.15.83:0/174114821 shutdown_connections 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 -- 172.21.15.83:0/174114821 wait complete. 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 -- start start 2024-09-21T03:24:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.234+0000 7ff147c8a740 1 Processor -- start 2024-09-21T03:24:22.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 0x56355ed7a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed7ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b7010 con 0x56355e99a1b0 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b6940 con 0x56355e9c3070 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b3ba0 con 0x56355e9a2390 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff147c8a740 10 client.0 _ll_register_callbacks cb 0x7ff13403bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 0x56355ed7a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 0x56355ed7a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44972/0 (socket says 172.21.15.83:44972) 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13affd640 1 -- 172.21.15.83:0/4004160039 learned_addr learned my addr 172.21.15.83:0/4004160039 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13b7fe640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed7ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53174/0 (socket says 172.21.15.83:53174) 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 -- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 msgr2=0x56355ed7ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed7ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 -- 172.21.15.83:0/4004160039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 msgr2=0x56355ed7a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 0x56355ed7a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.235+0000 7ff13a7fc640 1 -- 172.21.15.83:0/4004160039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355edf1a40 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff13a7fc640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff12c019040 tx=0x7ff12c0093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3796666e0bc407 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c032020 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff12c0044b0 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c01c040 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff12c03d050 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff147c8a740 10 client.26260 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56355edf2c80 con 0x56355e9a2390 2024-09-21T03:24:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c003b60 con 0x56355e9a2390 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.236+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff12c012050 con 0x56355e9a2390 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 10 client.26260 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 10 client.26260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 20 client.26260 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e956da0 0x56355edf5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 4 client.26260 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff12c01c040 con 0x56355e956da0 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff13affd640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e956da0 0x56355edf5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff11eff5640 10 client.26260 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.237+0000 7ff13affd640 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e956da0 0x56355edf5140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b32a21be5af35c server_cookie=3153df16ad67b5e6 in_seq=0 out_seq=0 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff11eff5640 1 -- 172.21.15.83:0/4004160039 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff12c01c040 con 0x56355e956da0 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff11eff5640 10 client.26260 handle_command_reply: tid=0 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 client.26260 shutdown 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 msgr2=0x56355ed7a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff12c019040 tx=0x7ff12c0093c0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 shutdown_connections 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56355e956da0 0x56355edf5140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56355e9c3070 0x56355ed7ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355e9a2390 0x56355ed7a630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 --2- 172.21.15.83:0/4004160039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56355e99a1b0 0x56355ed7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff11eff5640 0 client.26260 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 >> 172.21.15.83:0/4004160039 conn(0x56355e9b7460 msgr2=0x56355ed6b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.238+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 shutdown_connections 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.239+0000 7ff147c8a740 1 -- 172.21.15.83:0/4004160039 wait complete. 2024-09-21T03:24:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.239+0000 7ff147c8a740 20 client.26260 trim_cache size 0 max 16384 2024-09-21T03:24:22.246 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 msgr2=0x7ff1240b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 0x7ff1240b8ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56355ed69320 tx=0x7ff130005d70 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 msgr2=0x7ff13c10f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff13403be90 tx=0x7ff134041860 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 shutdown_connections 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff1240b67d0 0x7ff1240b8ce0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c107be0 0x7ff13c10f170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff13c1030f0 0x7ff13c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 --2- 172.21.15.83:0/1798515498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c1027a0 0x7ff13c074320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 >> 172.21.15.83:0/1798515498 conn(0x7ff13c079730 msgr2=0x7ff13c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 shutdown_connections 2024-09-21T03:24:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:22.242+0000 7ff141716640 1 -- 172.21.15.83:0/1798515498 wait complete. 2024-09-21T03:24:24.096 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:24:24.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:24:24.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:24 smithi083 ceph-mon[31219]: pgmap v561: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 652 op/s 2024-09-21T03:24:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:24.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:24 smithi193 ceph-mon[35224]: pgmap v561: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 652 op/s 2024-09-21T03:24:24.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:24 smithi142 ceph-mon[34512]: pgmap v561: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 652 op/s 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 -- 172.21.15.83:0/1704581531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 msgr2=0x7f795010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 --2- 172.21.15.83:0/1704581531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795010f370 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7948009be0 tx=0x7f794802a6c0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 -- 172.21.15.83:0/1704581531 shutdown_connections 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 --2- 172.21.15.83:0/1704581531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795010f370 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 --2- 172.21.15.83:0/1704581531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501081f0 0x7f7950108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 --2- 172.21.15.83:0/1704581531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501028b0 0x7f7950102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.047+0000 7f7954b9f640 1 -- 172.21.15.83:0/1704581531 >> 172.21.15.83:0/1704581531 conn(0x7f79500fe320 msgr2=0x7f7950100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.048+0000 7f7954b9f640 1 -- 172.21.15.83:0/1704581531 shutdown_connections 2024-09-21T03:24:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.048+0000 7f7954b9f640 1 -- 172.21.15.83:0/1704581531 wait complete. 2024-09-21T03:24:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.049+0000 7f7954b9f640 1 Processor -- start 2024-09-21T03:24:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.049+0000 7f7954b9f640 1 -- start start 2024-09-21T03:24:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.049+0000 7f7954b9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.049+0000 7f7954b9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 0x7f7950199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f7954b9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f7954b9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79501127f0 con 0x7f7950108b40 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f7954b9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7950112670 con 0x7f79501028b0 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f7954b9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7950112970 con 0x7f79501081f0 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 0x7f7950199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46550/0 (socket says 172.21.15.83:46550) 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 -- 172.21.15.83:0/3017914799 learned_addr learned my addr 172.21.15.83:0/3017914799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794ffff640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 -- 172.21.15.83:0/3017914799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 msgr2=0x7f7950199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 0x7f7950199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 -- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 msgr2=0x7f795019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794f7fe640 1 -- 172.21.15.83:0/3017914799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795019f0d0 con 0x7f79501028b0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.050+0000 7f794effd640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 0x7f7950199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f794000c890 tx=0x7f794000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f794cff9640 1 -- 172.21.15.83:0/3017914799 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940004220 con 0x7f79501028b0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795019f3c0 con 0x7f79501028b0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f794cff9640 1 -- 172.21.15.83:0/3017914799 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79400043c0 con 0x7f79501028b0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79501a5b20 con 0x7f79501028b0 2024-09-21T03:24:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.051+0000 7f794cff9640 1 -- 172.21.15.83:0/3017914799 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940004a00 con 0x7f79501028b0 2024-09-21T03:24:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.056+0000 7f794cff9640 1 -- 172.21.15.83:0/3017914799 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7940004ba0 con 0x7f79501028b0 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.057+0000 7f794cff9640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 0x7f79380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.057+0000 7f795c375740 1 Processor -- start 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.057+0000 7f794effd640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 0x7f79380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.057+0000 7f795c375740 1 -- start start 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.057+0000 7f794cff9640 1 -- 172.21.15.83:0/3017914799 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f79400e70a0 con 0x7f79501028b0 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf138300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbcf0a9e60 con 0x55cbced6b670 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cbcf099290 con 0x55cbced63490 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f795c375740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cbcf09d0a0 con 0x55cbced8c350 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 0x7f79380b8c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f79440096c0 tx=0x7f7944009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46564/0 (socket says 172.21.15.83:46564) 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 -- 172.21.15.83:0/4200987294 learned_addr learned my addr 172.21.15.83:0/4200987294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794f7fe640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794ffff640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 -- 172.21.15.83:0/4200987294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 msgr2=0x55cbcf138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 -- 172.21.15.83:0/4200987294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 msgr2=0x55cbcf135eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf135eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 -- 172.21.15.83:0/4200987294 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcedf7930 con 0x55cbced63490 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794ffff640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf135eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794f7fe640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.058+0000 7f794effd640 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f794400f6d0 tx=0x7f7944014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c217e476d4d40555 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.059+0000 7f7935ffb640 1 -- 172.21.15.83:0/4200987294 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794401c750 con 0x55cbced63490 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.059+0000 7f7935ffb640 1 -- 172.21.15.83:0/4200987294 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794401cda0 con 0x55cbced63490 2024-09-21T03:24:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.059+0000 7f7935ffb640 1 -- 172.21.15.83:0/4200987294 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400ee40 con 0x55cbced63490 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.059+0000 7f795c375740 1 -- 172.21.15.83:0/4200987294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 msgr2=0x55cbcf135820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.059+0000 7f795c375740 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f794400f6d0 tx=0x7f7944014000 comp rx=0 tx=0).stop 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 -- 172.21.15.83:0/4200987294 shutdown_connections 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf135eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf135820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 --2- 172.21.15.83:0/4200987294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 -- 172.21.15.83:0/4200987294 >> 172.21.15.83:0/4200987294 conn(0x55cbced80740 msgr2=0x55cbcf133710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 -- 172.21.15.83:0/4200987294 shutdown_connections 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 -- 172.21.15.83:0/4200987294 wait complete. 2024-09-21T03:24:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.060+0000 7f795c375740 1 -- start start 2024-09-21T03:24:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 Processor -- start 2024-09-21T03:24:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf143710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf14f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbceefa890 con 0x55cbced6b670 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cbcf033d40 con 0x55cbced63490 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cbcf146140 con 0x55cbced8c350 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f794effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f794effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44994/0 (socket says 172.21.15.83:44994) 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f794effd640 1 -- 172.21.15.83:0/3764210080 learned_addr learned my addr 172.21.15.83:0/3764210080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.061+0000 7f795c375740 10 client.0 _ll_register_callbacks cb 0x55cbcf097c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794ffff640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf14f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 -- 172.21.15.83:0/3764210080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 msgr2=0x55cbcf14f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf14f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 -- 172.21.15.83:0/3764210080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 msgr2=0x55cbcf143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 -- 172.21.15.83:0/3764210080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcf14fad0 con 0x55cbced63490 2024-09-21T03:24:25.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794ffff640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf14f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf143710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79400e48e0 tx=0x7f79400acaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569f1fcdf746faff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f792effd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940092e60 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f79400a8be0 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f794401cda0 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7940098020 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f795c375740 10 client.26288 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cbcf13bd10 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.062+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794401cda0 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f79400b2c50 con 0x55cbced63490 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 10 client.26288 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 20 client.26288 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:25.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced4acd0 0x55cbcf13f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 4 client.26288 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbcf0abdb0 con 0x55cbced4acd0 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.063+0000 7f794effd640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced4acd0 0x55cbcf13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.064+0000 7f792effd640 10 client.26288 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.064+0000 7f794effd640 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced4acd0 0x55cbcf13f620 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a04a53c85f8193 server_cookie=d90e14bbf2b1a17a in_seq=0 out_seq=0 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.064+0000 7f792effd640 1 -- 172.21.15.83:0/3764210080 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cbcf0abdb0 con 0x55cbced4acd0 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.064+0000 7f792effd640 10 client.26288 handle_command_reply: tid=0 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 client.26288 shutdown 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 msgr2=0x55cbcf143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf143710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79400e48e0 tx=0x7f79400acaa0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 shutdown_connections 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced4acd0 0x55cbcf13f620 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf14f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f792effd640 0 client.26288 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 --2- 172.21.15.83:0/3764210080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf143710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 >> 172.21.15.83:0/3764210080 conn(0x55cbced80740 msgr2=0x55cbcf137210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 shutdown_connections 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 1 -- 172.21.15.83:0/3764210080 wait complete. 2024-09-21T03:24:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.065+0000 7f795c375740 20 client.26288 trim_cache size 0 max 16384 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.089+0000 7f795c375740 1 Processor -- start 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.089+0000 7f795c375740 1 -- start start 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced63490 0x55cbcf140e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf143c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00e670 con 0x55cbced63490 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0101b0 con 0x55cbced6b670 2024-09-21T03:24:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f795c375740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00ed40 con 0x55cbced8c350 2024-09-21T03:24:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f794effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f794effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46578/0 (socket says 172.21.15.83:46578) 2024-09-21T03:24:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f794effd640 1 -- 172.21.15.83:0/88853930 learned_addr learned my addr 172.21.15.83:0/88853930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.090+0000 7f794ffff640 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.094+0000 7f794effd640 1 -- 172.21.15.83:0/88853930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 msgr2=0x55cbcf143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.094+0000 7f794effd640 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.094+0000 7f794effd640 1 -- 172.21.15.83:0/88853930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced63490 msgr2=0x55cbcf140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.094+0000 7f794effd640 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced63490 0x55cbcf140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.094+0000 7f794effd640 1 -- 172.21.15.83:0/88853930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcf1442c0 con 0x55cbced6b670 2024-09-21T03:24:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.097+0000 7f794effd640 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7944009e10 tx=0x7f794400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625d2c3c1c932a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.098+0000 7f79357fa640 1 -- 172.21.15.83:0/88853930 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794402b040 con 0x55cbced6b670 2024-09-21T03:24:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.098+0000 7f79357fa640 1 -- 172.21.15.83:0/88853930 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794400e8e0 con 0x55cbced6b670 2024-09-21T03:24:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.098+0000 7f79357fa640 1 -- 172.21.15.83:0/88853930 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440048c0 con 0x55cbced6b670 2024-09-21T03:24:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 -- 172.21.15.83:0/88853930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 msgr2=0x55cbcf1436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7944009e10 tx=0x7f794400eb80 comp rx=0 tx=0).stop 2024-09-21T03:24:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 -- 172.21.15.83:0/88853930 shutdown_connections 2024-09-21T03:24:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced6b670 0x55cbcf1436f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.100+0000 7f795c375740 1 --2- 172.21.15.83:0/88853930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced63490 0x55cbcf140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.101+0000 7f795c375740 1 -- 172.21.15.83:0/88853930 >> 172.21.15.83:0/88853930 conn(0x55cbced80740 msgr2=0x55cbcf142a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.107+0000 7f795c375740 1 -- 172.21.15.83:0/88853930 shutdown_connections 2024-09-21T03:24:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.111+0000 7f795c375740 1 -- 172.21.15.83:0/88853930 wait complete. 2024-09-21T03:24:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.111+0000 7f795c375740 1 -- start start 2024-09-21T03:24:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.112+0000 7f795c375740 1 Processor -- start 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.113+0000 7f795c375740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.113+0000 7f795c375740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf1c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.113+0000 7f795c375740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f795c375740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00fae0 con 0x55cbced6b670 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f795c375740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00f410 con 0x55cbced63490 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f795c375740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00c670 con 0x55cbced8c350 2024-09-21T03:24:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f795c375740 10 client.0 _ll_register_callbacks cb 0x7f79400a8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f794ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f794ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53216/0 (socket says 172.21.15.83:53216) 2024-09-21T03:24:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.114+0000 7f794ffff640 1 -- 172.21.15.83:0/3495927981 learned_addr learned my addr 172.21.15.83:0/3495927981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794effd640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf1c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 -- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 msgr2=0x55cbcf160090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf160090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 -- 172.21.15.83:0/3495927981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 msgr2=0x55cbcf1c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf1c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 -- 172.21.15.83:0/3495927981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcf1c2780 con 0x55cbced8c350 2024-09-21T03:24:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f794ffff640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f794802abe0 tx=0x7f7948030e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9bc2bfc7ae9b832 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.115+0000 7f792effd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.116+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948036040 con 0x55cbced8c350 2024-09-21T03:24:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.116+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f794400e8e0 con 0x55cbced8c350 2024-09-21T03:24:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.116+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f7948003970 con 0x55cbced8c350 2024-09-21T03:24:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.120+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7948003b70 con 0x55cbced8c350 2024-09-21T03:24:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.122+0000 7f795c375740 10 client.26272 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.122+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cbcf135dc0 con 0x55cbced8c350 2024-09-21T03:24:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794400e8e0 con 0x55cbced8c350 2024-09-21T03:24:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7948031440 con 0x55cbced8c350 2024-09-21T03:24:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 10 client.26272 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 20 client.26272 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced20080 0x55cbcf14aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 4 client.26272 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:24:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.123+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbcf0a7860 con 0x55cbced20080 2024-09-21T03:24:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.125+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced20080 0x55cbcf14aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.125+0000 7f794f7fe640 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced20080 0x55cbcf14aad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c182059a7876c37e server_cookie=4cda4a64eb2c85a0 in_seq=0 out_seq=0 2024-09-21T03:24:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.125+0000 7f792effd640 10 client.26272 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.127+0000 7f792effd640 1 -- 172.21.15.83:0/3495927981 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbcf0a7860 con 0x55cbced20080 2024-09-21T03:24:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f792effd640 10 client.26272 handle_command_reply: tid=0 2024-09-21T03:24:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 client.26272 shutdown 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 msgr2=0x55cbcf1c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f794802abe0 tx=0x7f7948030e80 comp rx=0 tx=0).stop 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 shutdown_connections 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55cbced20080 0x55cbcf14aad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cbced8c350 0x55cbcf1c21c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbced6b670 0x55cbcf1c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 --2- 172.21.15.83:0/3495927981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbced63490 0x55cbcf160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 >> 172.21.15.83:0/3495927981 conn(0x55cbced80740 msgr2=0x55cbcf1436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.128+0000 7f792effd640 0 client.26272 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.129+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 shutdown_connections 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.129+0000 7f795c375740 1 -- 172.21.15.83:0/3495927981 wait complete. 2024-09-21T03:24:25.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.129+0000 7f795c375740 20 client.26272 trim_cache size 0 max 16384 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "90072a91-addd-4dbf-a10c-84c6ce031faf", 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.130+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 msgr2=0x7f79380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.131+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 0x7f79380b8c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f79440096c0 tx=0x7f7944009260 comp rx=0 tx=0).stop 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.131+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 msgr2=0x7f7950199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.131+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f794000c890 tx=0x7f794000cd50 comp rx=0 tx=0).stop 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 shutdown_connections 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f79380b6700 0x7f79380b8c10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7950108b40 0x7f795019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79501081f0 0x7f7950199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 --2- 172.21.15.83:0/3017914799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79501028b0 0x7f7950199160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:25.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.133+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 >> 172.21.15.83:0/3017914799 conn(0x7f79500fe320 msgr2=0x7f7950107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:25.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.134+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 shutdown_connections 2024-09-21T03:24:25.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:25.134+0000 7f7954b9f640 1 -- 172.21.15.83:0/3017914799 wait complete. 2024-09-21T03:24:25.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:25 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:24:25.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:25 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:25.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:25 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:25.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:25 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:25.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:25 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:24:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:25 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:25 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:25 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:25.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:24:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:25 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:24:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:25 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:25.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:25 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:25.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:25 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 425.0 s, 2030.6 tps, lat 49.893 ms stddev 61.898 2024-09-21T03:24:26.244 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: pgmap v562: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 600 op/s 2024-09-21T03:24:26.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:26 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: pgmap v562: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 600 op/s 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:26 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.815+0000 7fafde3c2640 1 -- 172.21.15.83:0/3837443869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 msgr2=0x7fafd810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.815+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3837443869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 0x7fafd810b8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc802a440 comp rx=0 tx=0).stop 2024-09-21T03:24:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.819+0000 7fafde3c2640 1 -- 172.21.15.83:0/3837443869 shutdown_connections 2024-09-21T03:24:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.819+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3837443869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd810be30 0x7fafd8112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.819+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3837443869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 0x7fafd810b8f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.819+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3837443869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd8105ba0 0x7fafd8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.819+0000 7fafde3c2640 1 -- 172.21.15.83:0/3837443869 >> 172.21.15.83:0/3837443869 conn(0x7fafd8101610 msgr2=0x7fafd8103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.825+0000 7fafde3c2640 1 -- 172.21.15.83:0/3837443869 shutdown_connections 2024-09-21T03:24:26.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.825+0000 7fafde3c2640 1 -- 172.21.15.83:0/3837443869 wait complete. 2024-09-21T03:24:26.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.826+0000 7fafde3c2640 1 Processor -- start 2024-09-21T03:24:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.826+0000 7fafde3c2640 1 -- start start 2024-09-21T03:24:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8105ba0 0x7fafd8074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 0x7fafd806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafd806d6a0 con 0x7fafd8105ba0 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafd806d520 con 0x7fafd810be30 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafde3c2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafd806d820 con 0x7fafd810b4e0 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafddbc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafddbc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:46608/0 (socket says 172.21.15.83:46608) 2024-09-21T03:24:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.827+0000 7fafddbc1640 1 -- 172.21.15.83:0/3665797640 learned_addr learned my addr 172.21.15.83:0/3665797640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8105ba0 0x7fafd8074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 -- 172.21.15.83:0/3665797640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 msgr2=0x7fafd806f1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 0x7fafd806f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 -- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8105ba0 msgr2=0x7fafd8074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8105ba0 0x7fafd8074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 -- 172.21.15.83:0/3665797640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd806ffc0 con 0x7fafd810be30 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.828+0000 7fafddbc1640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fafd000eaa0 tx=0x7fafd000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.829+0000 7fafce7fc640 1 -- 172.21.15.83:0/3665797640 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000cdb0 con 0x7fafd810be30 2024-09-21T03:24:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.829+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81b10a0 con 0x7fafd810be30 2024-09-21T03:24:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.829+0000 7fafce7fc640 1 -- 172.21.15.83:0/3665797640 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd0005020 con 0x7fafd810be30 2024-09-21T03:24:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.829+0000 7fafce7fc640 1 -- 172.21.15.83:0/3665797640 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00116d0 con 0x7fafd810be30 2024-09-21T03:24:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.829+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81b1550 con 0x7fafd810be30 2024-09-21T03:24:26.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.833+0000 7fafe5b98740 1 Processor -- start 2024-09-21T03:24:26.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 -- start start 2024-09-21T03:24:26.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6643c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a663b2e50 con 0x557a66074390 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557a663a2280 con 0x557a66095070 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafe5b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a663a6090 con 0x557a6606c1b0 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.836+0000 7fafce7fc640 1 -- 172.21.15.83:0/3665797640 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fafd00054d0 con 0x7fafd810be30 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.837+0000 7fafdd3c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.837+0000 7fafdd3c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53236/0 (socket says 172.21.15.83:53236) 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.837+0000 7fafdd3c0640 1 -- 172.21.15.83:0/3328454313 learned_addr learned my addr 172.21.15.83:0/3328454313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.837+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6643c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.837+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdd3c0640 1 -- 172.21.15.83:0/3328454313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 msgr2=0x557a6643c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6643c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdd3c0640 1 -- 172.21.15.83:0/3328454313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 msgr2=0x557a6643ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafce7fc640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 0x7fafc40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdd3c0640 1 -- 172.21.15.83:0/3328454313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a66100650 con 0x557a6606c1b0 2024-09-21T03:24:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafce7fc640 1 -- 172.21.15.83:0/3665797640 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fafd00e78f0 con 0x7fafd810be30 2024-09-21T03:24:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.838+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 0x7fafc40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.839+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd400abe0 tx=0x7fafd400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50dc7fb8354dbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.839+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 0x7fafc40b8c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a960 tx=0x7fafc8000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.839+0000 7fafc37fe640 1 -- 172.21.15.83:0/3328454313 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4005220 con 0x557a6606c1b0 2024-09-21T03:24:26.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.839+0000 7fafc37fe640 1 -- 172.21.15.83:0/3328454313 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd40047a0 con 0x557a6606c1b0 2024-09-21T03:24:26.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.840+0000 7fafc37fe640 1 -- 172.21.15.83:0/3328454313 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd40186a0 con 0x557a6606c1b0 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 -- 172.21.15.83:0/3328454313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 msgr2=0x557a6643c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd400abe0 tx=0x7fafd400f790 comp rx=0 tx=0).stop 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 -- 172.21.15.83:0/3328454313 shutdown_connections 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a6643c950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 --2- 172.21.15.83:0/3328454313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6643c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 -- 172.21.15.83:0/3328454313 >> 172.21.15.83:0/3328454313 conn(0x557a66089460 msgr2=0x557a6643a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 -- 172.21.15.83:0/3328454313 shutdown_connections 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.841+0000 7fafe5b98740 1 -- 172.21.15.83:0/3328454313 wait complete. 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 -- start start 2024-09-21T03:24:26.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 Processor -- start 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a66447940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 0x557a66447e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a66203700 con 0x557a66095070 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557a6633d110 con 0x557a66074390 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.842+0000 7fafe5b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a6644ac90 con 0x557a6606c1b0 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafe5b98740 10 client.0 _ll_register_callbacks cb 0x557a66356990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45060/0 (socket says 172.21.15.83:45060) 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 -- 172.21.15.83:0/786363744 learned_addr learned my addr 172.21.15.83:0/786363744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:26.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 0x557a66447e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a66447940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 -- 172.21.15.83:0/786363744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 msgr2=0x557a66447940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a66447940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 -- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 msgr2=0x557a66447e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 0x557a66447e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafddbc1640 1 -- 172.21.15.83:0/786363744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6636ac40 con 0x557a66095070 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.843+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 0x557a66447e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafddbc1640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fafd00a9d20 tx=0x7fafd00e55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8313bb46fc8bc985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd001f030 con 0x557a66095070 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fafd00ac420 con 0x557a66095070 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd40047a0 con 0x557a66095070 2024-09-21T03:24:26.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fafd00ade50 con 0x557a66095070 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafe5b98740 10 client.16758 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.844+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557a664568c0 con 0x557a66095070 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd40047a0 con 0x557a66095070 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fafd000f040 con 0x557a66095070 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 10 client.16758 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 20 client.16758 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a660539f0 0x557a66459e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 4 client.16758 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a663b4da0 con 0x557a660539f0 2024-09-21T03:24:26.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a660539f0 0x557a66459e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.845+0000 7fafc0ff9640 10 client.16758 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:26.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.846+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a660539f0 0x557a66459e60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ecd165639de03f server_cookie=d4b23a8caf7e7e6c in_seq=0 out_seq=0 2024-09-21T03:24:26.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.846+0000 7fafc0ff9640 1 -- 172.21.15.83:0/786363744 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a663b4da0 con 0x557a660539f0 2024-09-21T03:24:26.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.846+0000 7fafc0ff9640 10 client.16758 handle_command_reply: tid=0 2024-09-21T03:24:26.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.846+0000 7fafe5b98740 1 client.16758 shutdown 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 msgr2=0x557a664483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fafd00a9d20 tx=0x7fafd00e55f0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 shutdown_connections 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a660539f0 0x557a66459e60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66095070 0x557a664483c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66074390 0x557a66447e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 --2- 172.21.15.83:0/786363744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a6606c1b0 0x557a66447940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 >> 172.21.15.83:0/786363744 conn(0x557a66089460 msgr2=0x557a6643f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafc0ff9640 0 client.16758 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 shutdown_connections 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 1 -- 172.21.15.83:0/786363744 wait complete. 2024-09-21T03:24:26.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.848+0000 7fafe5b98740 20 client.16758 trim_cache size 0 max 16384 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.871+0000 7fafe5b98740 1 Processor -- start 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.871+0000 7fafe5b98740 1 -- start start 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 0x557a66449820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 0x557a66444b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81b9eb0 con 0x557a66074390 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81bb9f0 con 0x557a6606c1b0 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafe5b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81ba580 con 0x557a66095070 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45076/0 (socket says 172.21.15.83:45076) 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 -- 172.21.15.83:0/2893014227 learned_addr learned my addr 172.21.15.83:0/2893014227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafddbc1640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 0x557a66444b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 0x557a66449820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 -- 172.21.15.83:0/2893014227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 msgr2=0x557a66444b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 0x557a66444b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 -- 172.21.15.83:0/2893014227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 msgr2=0x557a66449820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 0x557a66449820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdcbbf640 1 -- 172.21.15.83:0/2893014227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a66053c20 con 0x557a66074390 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafddbc1640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 0x557a66444b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.872+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 0x557a66449820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.873+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fafd806c910 tx=0x7fafc802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542b02c1d6d96f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.873+0000 7fafc2ffd640 1 -- 172.21.15.83:0/2893014227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8005370 con 0x557a66074390 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.873+0000 7fafc2ffd640 1 -- 172.21.15.83:0/2893014227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc8050070 con 0x557a66074390 2024-09-21T03:24:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.873+0000 7fafc2ffd640 1 -- 172.21.15.83:0/2893014227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8031710 con 0x557a66074390 2024-09-21T03:24:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 -- 172.21.15.83:0/2893014227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 msgr2=0x557a6643db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fafd806c910 tx=0x7fafc802a440 comp rx=0 tx=0).stop 2024-09-21T03:24:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 -- 172.21.15.83:0/2893014227 shutdown_connections 2024-09-21T03:24:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66095070 0x557a66444b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a66074390 0x557a6643db50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 --2- 172.21.15.83:0/2893014227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a6606c1b0 0x557a66449820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 -- 172.21.15.83:0/2893014227 >> 172.21.15.83:0/2893014227 conn(0x557a66089460 msgr2=0x557a6643d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.874+0000 7fafe5b98740 1 -- 172.21.15.83:0/2893014227 shutdown_connections 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.875+0000 7fafe5b98740 1 -- 172.21.15.83:0/2893014227 wait complete. 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.875+0000 7fafe5b98740 1 -- start start 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.875+0000 7fafe5b98740 1 Processor -- start 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a6606c1b0 0x557a664507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6644c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81bb320 con 0x557a6606c1b0 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81bac50 con 0x557a66095070 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81b7eb0 con 0x557a66074390 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53272/0 (socket says 172.21.15.83:53272) 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 -- 172.21.15.83:0/1231620670 learned_addr learned my addr 172.21.15.83:0/1231620670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:26.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafe5b98740 10 client.0 _ll_register_callbacks cb 0x7fafd00a9da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a6606c1b0 0x557a664507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafddbc1640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6644c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 -- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 msgr2=0x557a6644c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6644c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 -- 172.21.15.83:0/1231620670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a6606c1b0 msgr2=0x557a664507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a6606c1b0 0x557a664507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 -- 172.21.15.83:0/1231620670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6644ca60 con 0x557a66074390 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafddbc1640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6644c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.876+0000 7fafdcbbf640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafc8040a80 tx=0x7fafc8005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34fafb9a3f4d83ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.877+0000 7fafc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.877+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8055670 con 0x557a66074390 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.877+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fafc8055810 con 0x557a66074390 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.877+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc8050070 con 0x557a66074390 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.877+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fafc8066050 con 0x557a66074390 2024-09-21T03:24:26.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.878+0000 7fafe5b98740 10 client.26287 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.878+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557a664c4820 con 0x557a66074390 2024-09-21T03:24:26.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.878+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc805d070 con 0x557a66074390 2024-09-21T03:24:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fafc80318c0 con 0x557a66074390 2024-09-21T03:24:26.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 10 client.26287 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 20 client.26287 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a66028da0 0x557a664c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 4 client.26287 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.879+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fafc8050070 con 0x557a66028da0 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.880+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a66028da0 0x557a664c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.880+0000 7fafdd3c0640 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a66028da0 0x557a664c6f50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1e5efd0164a25 server_cookie=49829fb6177d9e9c in_seq=0 out_seq=0 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.880+0000 7fafc0ff9640 10 client.26287 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafc0ff9640 1 -- 172.21.15.83:0/1231620670 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x7fafc8050070 con 0x557a66028da0 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafc0ff9640 10 client.26287 handle_command_reply: tid=0 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 client.26287 shutdown 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 msgr2=0x557a6644bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafc8040a80 tx=0x7fafc8005630 comp rx=0 tx=0).stop 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 shutdown_connections 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557a66028da0 0x557a664c6f50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a66095070 0x557a6644c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a66074390 0x557a6644bc80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 --2- 172.21.15.83:0/1231620670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a6606c1b0 0x557a664507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 >> 172.21.15.83:0/1231620670 conn(0x557a66089460 msgr2=0x557a66449580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafc0ff9640 0 client.26287 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 shutdown_connections 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 1 -- 172.21.15.83:0/1231620670 wait complete. 2024-09-21T03:24:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.881+0000 7fafe5b98740 20 client.26287 trim_cache size 0 max 16384 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "90072a91-addd-4dbf-a10c-84c6ce031faf", 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:24:26.888 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:24:26.891 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.892 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:26.893 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:26.894 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 msgr2=0x7fafc40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 0x7fafc40b8c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a960 tx=0x7fafc8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 msgr2=0x7fafd806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fafd000eaa0 tx=0x7fafd000ef60 comp rx=0 tx=0).stop 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 shutdown_connections 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafc40b6780 0x7fafc40b8c90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafd810be30 0x7fafd806f7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.882+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafd810b4e0 0x7fafd806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.883+0000 7fafde3c2640 1 --2- 172.21.15.83:0/3665797640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafd8105ba0 0x7fafd8074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.883+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 >> 172.21.15.83:0/3665797640 conn(0x7fafd8101610 msgr2=0x7fafd8103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.883+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 shutdown_connections 2024-09-21T03:24:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:26.883+0000 7fafde3c2640 1 -- 172.21.15.83:0/3665797640 wait complete. 2024-09-21T03:24:26.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: pgmap v562: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 600 op/s 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:26.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:26 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:24:27.674 INFO:tasks.cephfs.filesystem:scrub status for tag:90072a91-addd-4dbf-a10c-84c6ce031faf - {'path': '/', 'tag': '90072a91-addd-4dbf-a10c-84c6ce031faf', 'options': 'recursive,force'} 2024-09-21T03:24:28.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:28 smithi193 ceph-mon[35224]: pgmap v563: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.5 MiB/s wr, 654 op/s 2024-09-21T03:24:28.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:28 smithi083 ceph-mon[31219]: pgmap v563: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.5 MiB/s wr, 654 op/s 2024-09-21T03:24:28.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:28 smithi142 ceph-mon[34512]: pgmap v563: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.5 MiB/s wr, 654 op/s 2024-09-21T03:24:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:30 smithi193 ceph-mon[35224]: pgmap v564: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2024-09-21T03:24:30.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:30 smithi083 ceph-mon[31219]: pgmap v564: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2024-09-21T03:24:30.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:30 smithi142 ceph-mon[34512]: pgmap v564: 97 pgs: 97 active+clean; 4.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2024-09-21T03:24:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 430.0 s, 2541.6 tps, lat 38.989 ms stddev 43.722 2024-09-21T03:24:32.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:32 smithi193 ceph-mon[35224]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-21T03:24:32.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:32 smithi083 ceph-mon[31219]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-21T03:24:32.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:32 smithi142 ceph-mon[34512]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-21T03:24:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:33 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:24:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:33 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:24:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:33 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:24:33.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:33 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:24:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:33 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:24:33.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:33 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:24:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:24:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:24:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:34 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:24:34.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:34 smithi193 ceph-mon[35224]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.5 MiB/s wr, 759 op/s 2024-09-21T03:24:34.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:24:34.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:24:34.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:34 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:24:34.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:34 smithi083 ceph-mon[31219]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.5 MiB/s wr, 759 op/s 2024-09-21T03:24:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:24:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:24:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:34 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:24:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:34 smithi142 ceph-mon[34512]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.5 MiB/s wr, 759 op/s 2024-09-21T03:24:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 435.0 s, 2584.6 tps, lat 38.709 ms stddev 44.202 2024-09-21T03:24:36.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:36 smithi193 ceph-mon[35224]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 680 op/s 2024-09-21T03:24:36.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:36 smithi083 ceph-mon[31219]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 680 op/s 2024-09-21T03:24:36.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:36 smithi142 ceph-mon[34512]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 680 op/s 2024-09-21T03:24:38.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:38 smithi193 ceph-mon[35224]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T03:24:38.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:38 smithi083 ceph-mon[31219]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T03:24:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:38 smithi142 ceph-mon[34512]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T03:24:40.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:40 smithi193 ceph-mon[35224]: pgmap v569: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-09-21T03:24:40.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:40 smithi083 ceph-mon[31219]: pgmap v569: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-09-21T03:24:40.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:40 smithi142 ceph-mon[34512]: pgmap v569: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-09-21T03:24:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 440.0 s, 2115.2 tps, lat 47.402 ms stddev 93.343 2024-09-21T03:24:42.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:42 smithi193 ceph-mon[35224]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-09-21T03:24:42.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:42 smithi083 ceph-mon[31219]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-09-21T03:24:42.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:42 smithi142 ceph-mon[34512]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-09-21T03:24:45.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:44 smithi193 ceph-mon[35224]: pgmap v571: 97 pgs: 97 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-21T03:24:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:44 smithi083 ceph-mon[31219]: pgmap v571: 97 pgs: 97 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-21T03:24:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:44 smithi142 ceph-mon[34512]: pgmap v571: 97 pgs: 97 active+clean; 4.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 1.53k op/s 2024-09-21T03:24:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 445.0 s, 2027.6 tps, lat 49.365 ms stddev 52.638 2024-09-21T03:24:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:46 smithi193 ceph-mon[35224]: pgmap v572: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 1.62k op/s 2024-09-21T03:24:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:46 smithi083 ceph-mon[31219]: pgmap v572: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 1.62k op/s 2024-09-21T03:24:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:46 smithi142 ceph-mon[34512]: pgmap v572: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 MiB/s wr, 1.62k op/s 2024-09-21T03:24:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:48 smithi193 ceph-mon[35224]: pgmap v573: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-09-21T03:24:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:48 smithi083 ceph-mon[31219]: pgmap v573: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-09-21T03:24:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:48 smithi142 ceph-mon[34512]: pgmap v573: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-09-21T03:24:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 450.0 s, 2130.4 tps, lat 46.358 ms stddev 52.525 2024-09-21T03:24:51.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:50 smithi083 ceph-mon[31219]: pgmap v574: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.58k op/s 2024-09-21T03:24:51.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:50 smithi193 ceph-mon[35224]: pgmap v574: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.58k op/s 2024-09-21T03:24:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:50 smithi142 ceph-mon[34512]: pgmap v574: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.58k op/s 2024-09-21T03:24:53.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:52 smithi193 ceph-mon[35224]: pgmap v575: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 1.50k op/s 2024-09-21T03:24:53.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:52 smithi083 ceph-mon[31219]: pgmap v575: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 1.50k op/s 2024-09-21T03:24:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:52 smithi142 ceph-mon[34512]: pgmap v575: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 1.50k op/s 2024-09-21T03:24:55.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:54 smithi193 ceph-mon[35224]: pgmap v576: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.35k op/s 2024-09-21T03:24:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:54 smithi083 ceph-mon[31219]: pgmap v576: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.35k op/s 2024-09-21T03:24:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:54 smithi142 ceph-mon[34512]: pgmap v576: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 MiB/s wr, 1.35k op/s 2024-09-21T03:24:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 455.0 s, 2082.0 tps, lat 48.711 ms stddev 57.168 2024-09-21T03:24:57.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:56 smithi193 ceph-mon[35224]: pgmap v577: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T03:24:57.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:56 smithi083 ceph-mon[31219]: pgmap v577: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T03:24:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:56 smithi142 ceph-mon[34512]: pgmap v577: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T03:24:57.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:24:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 -- 172.21.15.83:0/4282544539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 msgr2=0x7fc06c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 --2- 172.21.15.83:0/4282544539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c102380 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a410 comp rx=0 tx=0).stop 2024-09-21T03:24:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 -- 172.21.15.83:0/4282544539 shutdown_connections 2024-09-21T03:24:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 --2- 172.21.15.83:0/4282544539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 --2- 172.21.15.83:0/4282544539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 --2- 172.21.15.83:0/4282544539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c102380 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.620+0000 7fc07336a640 1 -- 172.21.15.83:0/4282544539 >> 172.21.15.83:0/4282544539 conn(0x7fc06c077c60 msgr2=0x7fc06c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.621+0000 7fc07336a640 1 -- 172.21.15.83:0/4282544539 shutdown_connections 2024-09-21T03:24:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.621+0000 7fc07336a640 1 -- 172.21.15.83:0/4282544539 wait complete. 2024-09-21T03:24:58.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.622+0000 7fc07336a640 1 Processor -- start 2024-09-21T03:24:58.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.622+0000 7fc07336a640 1 -- start start 2024-09-21T03:24:58.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c113ac0 con 0x7fc06c101f70 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c113940 con 0x7fc06c109e00 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc07336a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c113c40 con 0x7fc06c1094b0 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc070905640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc071907640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.623+0000 7fc070905640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59472/0 (socket says 172.21.15.83:59472) 2024-09-21T03:24:58.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 -- 172.21.15.83:0/2396570038 learned_addr learned my addr 172.21.15.83:0/2396570038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:58.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071106640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 -- 172.21.15.83:0/2396570038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 msgr2=0x7fc06c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 -- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 msgr2=0x7fc06c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 -- 172.21.15.83:0/2396570038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1a0400 con 0x7fc06c109e00 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc070905640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071106640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.624+0000 7fc071907640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06400eb10 tx=0x7fc06400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.625+0000 7fc0627fc640 1 -- 172.21.15.83:0/2396570038 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004b20 con 0x7fc06c109e00 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.625+0000 7fc0627fc640 1 -- 172.21.15.83:0/2396570038 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064004cc0 con 0x7fc06c109e00 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.625+0000 7fc0627fc640 1 -- 172.21.15.83:0/2396570038 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064010670 con 0x7fc06c109e00 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.626+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c1a06f0 con 0x7fc06c109e00 2024-09-21T03:24:58.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.626+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1a6e50 con 0x7fc06c109e00 2024-09-21T03:24:58.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.629+0000 7fc0798de740 1 Processor -- start 2024-09-21T03:24:58.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.629+0000 7fc0627fc640 1 -- 172.21.15.83:0/2396570038 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc064005020 con 0x7fc06c109e00 2024-09-21T03:24:58.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.629+0000 7fc0798de740 1 -- start start 2024-09-21T03:24:58.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.629+0000 7fc0798de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 0x55b5a42e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0798de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 0x55b5a42e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0798de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f21390 0x55b5a42ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc070905640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 0x55b5a42e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0627fc640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 0x7fc0580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc070905640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 0x55b5a42e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:48108/0 (socket says 172.21.15.83:48108) 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc070905640 1 -- 172.21.15.83:0/3784675954 learned_addr learned my addr 172.21.15.83:0/3784675954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5a425fe50 con 0x55b5a3f21390 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5a424f280 con 0x55b5a3f42070 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b5a4253090 con 0x55b5a3f191b0 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc071106640 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 0x55b5a42e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc0627fc640 1 -- 172.21.15.83:0/2396570038 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc0640e7d00 con 0x7fc06c109e00 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc071907640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 0x7fc0580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.630+0000 7fc070905640 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f21390 0x55b5a42ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 -- 172.21.15.83:0/3784675954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 msgr2=0x55b5a42e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 0x55b5a42e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 -- 172.21.15.83:0/3784675954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f21390 msgr2=0x55b5a42ea0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f21390 0x55b5a42ea0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 -- 172.21.15.83:0/3784675954 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a3fad650 con 0x55b5a3f191b0 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071106640 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 0x55b5a42e9b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc068000c00 tx=0x7fc068032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8c298e8fca4960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc071907640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 0x7fc0580b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0640049d0 tx=0x7fc06407a850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc0577fe640 1 -- 172.21.15.83:0/3784675954 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06803f070 con 0x55b5a3f191b0 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc0577fe640 1 -- 172.21.15.83:0/3784675954 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680316a0 con 0x55b5a3f191b0 2024-09-21T03:24:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.631+0000 7fc0577fe640 1 -- 172.21.15.83:0/3784675954 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068035070 con 0x55b5a3f191b0 2024-09-21T03:24:58.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.632+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 msgr2=0x55b5a42e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 0x55b5a42e9b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc068000c00 tx=0x7fc068032c30 comp rx=0 tx=0).stop 2024-09-21T03:24:58.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 shutdown_connections 2024-09-21T03:24:58.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f21390 0x55b5a42ea0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f191b0 0x55b5a42e9b90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 --2- 172.21.15.83:0/3784675954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f42070 0x55b5a42e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.633+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 >> 172.21.15.83:0/3784675954 conn(0x55b5a3f36460 msgr2=0x55b5a42e7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.634+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 shutdown_connections 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.634+0000 7fc0798de740 1 -- 172.21.15.83:0/3784675954 wait complete. 2024-09-21T03:24:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.634+0000 7fc0798de740 1 -- start start 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.634+0000 7fc0798de740 1 Processor -- start 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc070905640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f42070 0x55b5a4303340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5a40b0700 con 0x55b5a3f191b0 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc070905640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48186/0 (socket says 172.21.15.83:48186) 2024-09-21T03:24:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc070905640 1 -- 172.21.15.83:0/3347284673 learned_addr learned my addr 172.21.15.83:0/3347284673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5a41ea110 con 0x55b5a3f21390 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b5a42fc180 con 0x55b5a3f42070 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc071106640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.636+0000 7fc0798de740 10 client.0 _ll_register_callbacks cb 0x55b5a4203990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc070905640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f42070 0x55b5a4303340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 -- 172.21.15.83:0/3347284673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f42070 msgr2=0x55b5a4303340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f42070 0x55b5a4303340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 -- 172.21.15.83:0/3347284673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 msgr2=0x55b5a42fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a4217c40 con 0x55b5a3f21390 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc071106640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fe320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55b5a42e82d0 tx=0x7fc06802ab20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e0f03bbbf59fba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:24:58.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068035070 con 0x55b5a3f21390 2024-09-21T03:24:58.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc068005560 con 0x55b5a3f21390 2024-09-21T03:24:58.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.637+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0680316a0 con 0x55b5a3f21390 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc068055050 con 0x55b5a3f21390 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 10 client.26318 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b5a430c270 con 0x55b5a3f21390 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06804c070 con 0x55b5a3f21390 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc068003bf0 con 0x55b5a3f21390 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 10 client.26318 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 20 client.26318 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f009f0 0x55b5a430f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 4 client.26318 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:24:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.638+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc0680316a0 con 0x55b5a3f009f0 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.639+0000 7fc070905640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f009f0 0x55b5a430f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.639+0000 7fc054ff9640 10 client.26318 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.639+0000 7fc070905640 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f009f0 0x55b5a430f5d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84761f173c101510 server_cookie=b71c76c17f022173 in_seq=0 out_seq=0 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc054ff9640 1 -- 172.21.15.83:0/3347284673 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc0680316a0 con 0x55b5a3f009f0 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc054ff9640 10 client.26318 handle_command_reply: tid=0 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 client.26318 shutdown 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 msgr2=0x55b5a42fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fe320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55b5a42e82d0 tx=0x7fc06802ab20 comp rx=0 tx=0).stop 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 shutdown_connections 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f009f0 0x55b5a430f5d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc054ff9640 0 client.26318 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f42070 0x55b5a4303340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fe320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 --2- 172.21.15.83:0/3347284673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.640+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 >> 172.21.15.83:0/3347284673 conn(0x55b5a3f36460 msgr2=0x55b5a42ec7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.641+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 shutdown_connections 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.641+0000 7fc0798de740 1 -- 172.21.15.83:0/3347284673 wait complete. 2024-09-21T03:24:58.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.641+0000 7fc0798de740 20 client.26318 trim_cache size 0 max 16384 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.667+0000 7fc0798de740 1 Processor -- start 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.667+0000 7fc0798de740 1 -- start start 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1ad400 con 0x55b5a3f191b0 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1aef40 con 0x55b5a3f21390 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:59504/0 (socket says 172.21.15.83:59504) 2024-09-21T03:24:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 -- 172.21.15.83:0/3813541974 learned_addr learned my addr 172.21.15.83:0/3813541974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc0798de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1adad0 con 0x55b5a3f009f0 2024-09-21T03:24:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 -- 172.21.15.83:0/3813541974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 msgr2=0x55b5a42fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.668+0000 7fc070905640 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc070905640 1 -- 172.21.15.83:0/3813541974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 msgr2=0x55b5a42ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc070905640 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc070905640 1 -- 172.21.15.83:0/3813541974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a42fc1b0 con 0x55b5a3f009f0 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc070905640 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2dcdd3c194ce0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc056ffd640 1 -- 172.21.15.83:0/3813541974 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0043d0 con 0x55b5a3f009f0 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc056ffd640 1 -- 172.21.15.83:0/3813541974 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05c0054e0 con 0x55b5a3f009f0 2024-09-21T03:24:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.669+0000 7fc056ffd640 1 -- 172.21.15.83:0/3813541974 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c004a90 con 0x55b5a3f009f0 2024-09-21T03:24:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 -- 172.21.15.83:0/3813541974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 msgr2=0x55b5a42f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c015040 comp rx=0 tx=0).stop 2024-09-21T03:24:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 -- 172.21.15.83:0/3813541974 shutdown_connections 2024-09-21T03:24:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f21390 0x55b5a42fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a42ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 --2- 172.21.15.83:0/3813541974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f009f0 0x55b5a42f7150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 -- 172.21.15.83:0/3813541974 >> 172.21.15.83:0/3813541974 conn(0x55b5a3f36460 msgr2=0x55b5a42ea680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 -- 172.21.15.83:0/3813541974 shutdown_connections 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.671+0000 7fc0798de740 1 -- 172.21.15.83:0/3813541974 wait complete. 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 -- start start 2024-09-21T03:24:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 Processor -- start 2024-09-21T03:24:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 0x55b5a4309c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f21390 0x55b5a4306690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1ae870 con 0x55b5a3f191b0 2024-09-21T03:24:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1ae1a0 con 0x55b5a3f009f0 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1ab400 con 0x55b5a3f21390 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.672+0000 7fc0798de740 10 client.0 _ll_register_callbacks cb 0x7fc05c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48202/0 (socket says 172.21.15.83:48202) 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 -- 172.21.15.83:0/3558153817 learned_addr learned my addr 172.21.15.83:0/3558153817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc070905640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 0x55b5a4309c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc070905640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f21390 0x55b5a4306690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 -- 172.21.15.83:0/3558153817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f21390 msgr2=0x55b5a4306690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f21390 0x55b5a4306690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 -- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 msgr2=0x55b5a4309c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 0x55b5a4309c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 -- 172.21.15.83:0/3558153817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a4306bd0 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc070905640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 0x55b5a4309c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc071106640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc068009250 tx=0x7fc068032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb7a82446251014 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.673+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06805e060 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fc068004190 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5a4303df0 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc068035040 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 10 client.16773 fetch_fsmap learned FSMap version 44 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b5a437aa20 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06804c070 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fc06803b5d0 con 0x55b5a3f191b0 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 10 client.16773 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:24:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 20 client.16773 populate_metadata read hostname 'smithi083' 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f42070 0x55b5a437d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 4 client.16773 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.674+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5a4303df0 con 0x55b5a3f42070 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.675+0000 7fc070905640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f42070 0x55b5a437d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.675+0000 7fc054ff9640 10 client.16773 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.675+0000 7fc070905640 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f42070 0x55b5a437d1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa5790dc0ca46c1 server_cookie=5877fa7277d85df1 in_seq=0 out_seq=0 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc054ff9640 1 -- 172.21.15.83:0/3558153817 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55b5a4303df0 con 0x55b5a3f42070 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc054ff9640 10 client.16773 handle_command_reply: tid=0 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 client.16773 shutdown 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 msgr2=0x55b5a4306150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc068009250 tx=0x7fc068032c30 comp rx=0 tx=0).stop 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 shutdown_connections 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55b5a3f42070 0x55b5a437d1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5a3f21390 0x55b5a4306690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5a3f191b0 0x55b5a4306150 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 --2- 172.21.15.83:0/3558153817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5a3f009f0 0x55b5a4309c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc054ff9640 0 client.16773 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 >> 172.21.15.83:0/3558153817 conn(0x55b5a3f36460 msgr2=0x55b5a42feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 shutdown_connections 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 1 -- 172.21.15.83:0/3558153817 wait complete. 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.676+0000 7fc0798de740 20 client.16773 trim_cache size 0 max 16384 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:24:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.685 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:24:58.688 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 msgr2=0x7fc0580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 0x7fc0580b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0640049d0 tx=0x7fc06407a850 comp rx=0 tx=0).stop 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 msgr2=0x7fc06c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06400eb10 tx=0x7fc06400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 shutdown_connections 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc0580b6700 0x7fc0580b8c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06c109e00 0x7fc06c19fcd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc06c1094b0 0x7fc06c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 --2- 172.21.15.83:0/2396570038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c101f70 0x7fc06c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 >> 172.21.15.83:0/2396570038 conn(0x7fc06c077c60 msgr2=0x7fc06c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 shutdown_connections 2024-09-21T03:24:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:24:58.677+0000 7fc07336a640 1 -- 172.21.15.83:0/2396570038 wait complete. 2024-09-21T03:24:59.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:24:58 smithi193 ceph-mon[35224]: pgmap v578: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 653 op/s 2024-09-21T03:24:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:24:58 smithi083 ceph-mon[31219]: pgmap v578: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 653 op/s 2024-09-21T03:24:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:24:58 smithi142 ceph-mon[34512]: pgmap v578: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 653 op/s 2024-09-21T03:24:59.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:24:59.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:24:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 -- 172.21.15.83:0/1641135580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c1083d0 msgr2=0x7f8c4c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 --2- 172.21.15.83:0/1641135580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c1087e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:25:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 -- 172.21.15.83:0/1641135580 shutdown_connections 2024-09-21T03:25:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 --2- 172.21.15.83:0/1641135580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 0x7f8c4c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 --2- 172.21.15.83:0/1641135580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c1087e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 --2- 172.21.15.83:0/1641135580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c102ae0 0x7f8c4c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 -- 172.21.15.83:0/1641135580 >> 172.21.15.83:0/1641135580 conn(0x7f8c4c0797b0 msgr2=0x7f8c4c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.530+0000 7f8c544e1640 1 -- 172.21.15.83:0/1641135580 shutdown_connections 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.531+0000 7f8c544e1640 1 -- 172.21.15.83:0/1641135580 wait complete. 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.531+0000 7f8c544e1640 1 Processor -- start 2024-09-21T03:25:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.532+0000 7f8c544e1640 1 -- start start 2024-09-21T03:25:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.532+0000 7f8c544e1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c102ae0 0x7f8c4c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c544e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c544e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 0x7f8c4c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c544e1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c112a30 con 0x7f8c4c102ae0 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c544e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1128b0 con 0x7f8c4c108d20 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c544e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c112bb0 con 0x7f8c4c1083d0 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c51a7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c51a7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:50998/0 (socket says 172.21.15.83:50998) 2024-09-21T03:25:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c51a7c640 1 -- 172.21.15.83:0/2486787777 learned_addr learned my addr 172.21.15.83:0/2486787777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.533+0000 7f8c52a7e640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 0x7f8c4c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c5227d640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c102ae0 0x7f8c4c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 -- 172.21.15.83:0/2486787777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 msgr2=0x7f8c4c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 0x7f8c4c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 -- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c102ae0 msgr2=0x7f8c4c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c102ae0 0x7f8c4c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 -- 172.21.15.83:0/2486787777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c1a0240 con 0x7f8c4c1083d0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.534+0000 7f8c51a7c640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02a930 tx=0x7f8c3c030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.535+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c03f070 con 0x7f8c4c1083d0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.535+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c004460 con 0x7f8c4c1083d0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.535+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c1a04d0 con 0x7f8c4c1083d0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.535+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c0ff550 con 0x7f8c4c1083d0 2024-09-21T03:25:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.535+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c035030 con 0x7f8c4c1083d0 2024-09-21T03:25:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.536+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c300052c0 con 0x7f8c4c1083d0 2024-09-21T03:25:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.538+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8c3c042070 con 0x7f8c4c1083d0 2024-09-21T03:25:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.539+0000 7f8c437fe640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 0x7f8c380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:00.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.539+0000 7f8c5227d640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 0x7f8c380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:00.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.539+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f8c3c107c60 con 0x7f8c4c1083d0 2024-09-21T03:25:00.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.540+0000 7f8c5227d640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 0x7f8c380b8c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005fd0 tx=0x7f8c48009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.541+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c3c0d25e0 con 0x7f8c4c1083d0 2024-09-21T03:25:00.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.849+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c300055b0 con 0x7f8c4c1083d0 2024-09-21T03:25:00.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.850+0000 7f8c437fe640 1 -- 172.21.15.83:0/2486787777 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11982 (secure 0 0 0) 0x7f8c3c0d7520 con 0x7f8c4c1083d0 2024-09-21T03:25:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:25:00.852 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"btime":"2024-09-21T03:23:29:976813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:23:29.048942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:25:00.852 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 msgr2=0x7f8c380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 0x7f8c380b8c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005fd0 tx=0x7f8c48009500 comp rx=0 tx=0).stop 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 msgr2=0x7f8c4c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02a930 tx=0x7f8c3c030650 comp rx=0 tx=0).stop 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 shutdown_connections 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8c380b6700 0x7f8c380b8c10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c4c108d20 0x7f8c4c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4c1083d0 0x7f8c4c19a870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 --2- 172.21.15.83:0/2486787777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c4c102ae0 0x7f8c4c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:00.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 >> 172.21.15.83:0/2486787777 conn(0x7f8c4c0797b0 msgr2=0x7f8c4c10a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:00.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 shutdown_connections 2024-09-21T03:25:00.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:00.853+0000 7f8c544e1640 1 -- 172.21.15.83:0/2486787777 wait complete. 2024-09-21T03:25:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 460.0 s, 1577.6 tps, lat 61.816 ms stddev 65.361 2024-09-21T03:25:01.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: pgmap v579: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-09-21T03:25:01.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:01.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:01.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:01.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:25:01.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:00 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2486787777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: pgmap v579: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:25:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:00 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2486787777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: pgmap v579: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-09-21T03:25:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:01.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:25:01.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:00 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2486787777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:01.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:25:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/1984535264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 msgr2=0x7fbbb01061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/1984535264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb01061a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009900 tx=0x7fbba002a410 comp rx=0 tx=0).stop 2024-09-21T03:25:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/1984535264 shutdown_connections 2024-09-21T03:25:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/1984535264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/1984535264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/1984535264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb01061a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/1984535264 >> 172.21.15.83:0/1984535264 conn(0x7fbbb01017e0 msgr2=0x7fbbb0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/1984535264 shutdown_connections 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.674+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/1984535264 wait complete. 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.675+0000 7fbbb4b4c640 1 Processor -- start 2024-09-21T03:25:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.675+0000 7fbbb4b4c640 1 -- start start 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0115ce0 con 0x7fbbb0105d90 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0115b60 con 0x7fbbb010c020 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbb4b4c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0115e60 con 0x7fbbb010b6d0 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51018/0 (socket says 172.21.15.83:51018) 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb006d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47464/0 (socket says 172.21.15.83:47464) 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbba7fff640 1 -- 172.21.15.83:0/586022681 learned_addr learned my addr 172.21.15.83:0/586022681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.676+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 -- 172.21.15.83:0/586022681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 msgr2=0x7fbbb006d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 -- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 msgr2=0x7fbbb006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 -- 172.21.15.83:0/586022681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb0073040 con 0x7fbbb010c020 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbba7fff640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0072910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009fd0 tx=0x7fbba800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbba77fe640 1 -- 172.21.15.83:0/586022681 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba800ce20 con 0x7fbbb010c020 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb00732d0 con 0x7fbbb010c020 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbba77fe640 1 -- 172.21.15.83:0/586022681 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba8005020 con 0x7fbbb010c020 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbba77fe640 1 -- 172.21.15.83:0/586022681 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8010690 con 0x7fbbb010c020 2024-09-21T03:25:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.677+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb0073a80 con 0x7fbbb010c020 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.680+0000 7fbba77fe640 1 -- 172.21.15.83:0/586022681 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbba8010830 con 0x7fbbb010c020 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.682+0000 7fbbbb0c0740 1 Processor -- start 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.682+0000 7fbbbb0c0740 1 -- start start 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cf2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72cf2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b72c5fea0 con 0x556b72942070 2024-09-21T03:25:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556b72c4f2d0 con 0x556b729191b0 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556b72c530e0 con 0x556b72921390 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47470/0 (socket says 172.21.15.83:47470) 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbba7fff640 1 -- 172.21.15.83:0/3511629783 learned_addr learned my addr 172.21.15.83:0/3511629783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72cf2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.683+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cf2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 -- 172.21.15.83:0/3511629783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 msgr2=0x556b72cf2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72cf2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 -- 172.21.15.83:0/3511629783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 msgr2=0x556b72cf2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cf2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba77fe640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 0x7fbb9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 -- 172.21.15.83:0/3511629783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b729ad650 con 0x556b72942070 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba77fe640 1 -- 172.21.15.83:0/586022681 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fbba80e7e20 con 0x7fbbb010c020 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbb98011c30 tx=0x7fbb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e185242853318e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.684+0000 7fbba7fff640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 0x7fbb9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.685+0000 7fbba4ff9640 1 -- 172.21.15.83:0/3511629783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800fc50 con 0x556b72942070 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.685+0000 7fbba4ff9640 1 -- 172.21.15.83:0/3511629783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fdf0 con 0x556b72942070 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbba4ff9640 1 -- 172.21.15.83:0/3511629783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800e7f0 con 0x556b72942070 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3511629783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 msgr2=0x556b72cf1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbb98011c30 tx=0x7fbb98009bb0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbba7fff640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 0x7fbb9c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb9801e000 tx=0x7fbb9800ea70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3511629783 shutdown_connections 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72cf2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cf2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3511629783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72cf1ad0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3511629783 >> 172.21.15.83:0/3511629783 conn(0x556b72936460 msgr2=0x556b72cef9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3511629783 shutdown_connections 2024-09-21T03:25:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.686+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3511629783 wait complete. 2024-09-21T03:25:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.687+0000 7fbbbb0c0740 1 -- start start 2024-09-21T03:25:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.687+0000 7fbbbb0c0740 1 Processor -- start 2024-09-21T03:25:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b72ab0700 con 0x556b72921390 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556b72bea160 con 0x556b729191b0 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556b72d00350 con 0x556b72942070 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51040/0 (socket says 172.21.15.83:51040) 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbaed9d640 1 -- 172.21.15.83:0/3368429121 learned_addr learned my addr 172.21.15.83:0/3368429121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.688+0000 7fbbbb0c0740 10 client.0 _ll_register_callbacks cb 0x556b72c039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 -- 172.21.15.83:0/3368429121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 msgr2=0x556b72cfd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 -- 172.21.15.83:0/3368429121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 msgr2=0x556b72cfd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 -- 172.21.15.83:0/3368429121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b72c17c90 con 0x556b72942070 2024-09-21T03:25:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbba80ab1a0 tx=0x7fbba80054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd5641ee7ee308f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.689+0000 7fbb95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba80b3380 con 0x556b72942070 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fbba80ade30 con 0x556b72942070 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9800fdf0 con 0x556b72942070 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbba80d12f0 con 0x556b72942070 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbbbb0c0740 10 client.26320 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556b72d0bf60 con 0x556b72942070 2024-09-21T03:25:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fdf0 con 0x556b72942070 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.690+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fbba80aa520 con 0x556b72942070 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 10 client.26320 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 10 client.26320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 20 client.26320 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b729009f0 0x556b72d0f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 4 client.26320 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b72c61df0 con 0x556b729009f0 2024-09-21T03:25:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.691+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b729009f0 0x556b72d0f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.692+0000 7fbb95ffb640 10 client.26320 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.692+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b729009f0 0x556b72d0f910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=513ceb96282b3549 server_cookie=44c802513b016b90 in_seq=0 out_seq=0 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.693+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3368429121 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b72c61df0 con 0x556b729009f0 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.693+0000 7fbb95ffb640 10 client.26320 handle_command_reply: tid=0 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.693+0000 7fbbbb0c0740 1 client.26320 shutdown 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.694+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 msgr2=0x556b72cfdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.694+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbba80ab1a0 tx=0x7fbba80054d0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 shutdown_connections 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b729009f0 0x556b72d0f910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfdae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3368429121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 >> 172.21.15.83:0/3368429121 conn(0x556b72936460 msgr2=0x556b72cf4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbb95ffb640 0 client.26320 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 shutdown_connections 2024-09-21T03:25:02.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3368429121 wait complete. 2024-09-21T03:25:02.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.695+0000 7fbbbb0c0740 20 client.26320 trim_cache size 0 max 16384 2024-09-21T03:25:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.712+0000 7fbbbb0c0740 1 Processor -- start 2024-09-21T03:25:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.712+0000 7fbbbb0c0740 1 -- start start 2024-09-21T03:25:02.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.712+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.712+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b5b80 con 0x556b72921390 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b76c0 con 0x556b729191b0 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b6250 con 0x556b72942070 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52476/0 (socket says 172.21.15.83:52476) 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbae59c640 1 -- 172.21.15.83:0/2618845553 learned_addr learned my addr 172.21.15.83:0/2618845553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 -- 172.21.15.83:0/2618845553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 msgr2=0x556b72cfa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 -- 172.21.15.83:0/2618845553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 msgr2=0x556b72cfef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.713+0000 7fbba7fff640 1 -- 172.21.15.83:0/2618845553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b72cfe100 con 0x556b72921390 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.714+0000 7fbba7fff640 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfa250 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbb980058c0 tx=0x7fbb9802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ab30436b30ddf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.714+0000 7fbb97fff640 1 -- 172.21.15.83:0/2618845553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9803a070 con 0x556b72921390 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.714+0000 7fbb97fff640 1 -- 172.21.15.83:0/2618845553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fc20 con 0x556b72921390 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.714+0000 7fbb97fff640 1 -- 172.21.15.83:0/2618845553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98031040 con 0x556b72921390 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/2618845553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 msgr2=0x556b72cfa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfa250 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbb980058c0 tx=0x7fbb9802b000 comp rx=0 tx=0).stop 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/2618845553 shutdown_connections 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72942070 0x556b72cfa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72921390 0x556b72cfa250 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/2618845553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72cfef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/2618845553 >> 172.21.15.83:0/2618845553 conn(0x556b72936460 msgr2=0x556b72cfb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.715+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/2618845553 shutdown_connections 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/2618845553 wait complete. 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 -- start start 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 Processor -- start 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72d01b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b6ff0 con 0x556b72942070 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01b6920 con 0x556b729191b0 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb007b190 con 0x556b72921390 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbbb0c0740 10 client.0 _ll_register_callbacks cb 0x7fbba80ab220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.716+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52484/0 (socket says 172.21.15.83:52484) 2024-09-21T03:25:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47486/0 (socket says 172.21.15.83:47486) 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 -- 172.21.15.83:0/3836116831 learned_addr learned my addr 172.21.15.83:0/3836116831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72d01b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 -- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 msgr2=0x556b72d01b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72d01b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 -- 172.21.15.83:0/3836116831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 msgr2=0x556b72d02050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 -- 172.21.15.83:0/3836116831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b72d79310 con 0x556b729191b0 2024-09-21T03:25:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72d01b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbaed9d640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbae59c640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbba0005ab0 tx=0x7fbba0005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa9d6b79df89631 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbb95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0054060 con 0x556b729191b0 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9800fc20 con 0x556b729191b0 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.717+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fbba0004040 con 0x556b729191b0 2024-09-21T03:25:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbba004b040 con 0x556b729191b0 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 10 client.26345 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556b72d7a620 con 0x556b729191b0 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800fc20 con 0x556b729191b0 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fbba003a040 con 0x556b729191b0 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 10 client.26345 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 10 client.26345 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 20 client.26345 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.718+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b728d5da0 0x556b72d7cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.719+0000 7fbbbb0c0740 4 client.26345 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.719+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b72c5d8a0 con 0x556b728d5da0 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.719+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b728d5da0 0x556b72d7cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbb95ffb640 10 client.26345 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbba7fff640 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b728d5da0 0x556b72d7cdc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa45476a4b95f723 server_cookie=fe47e75b4c3089ea in_seq=0 out_seq=0 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbb95ffb640 1 -- 172.21.15.83:0/3836116831 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b72c5d8a0 con 0x556b728d5da0 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbb95ffb640 10 client.26345 handle_command_reply: tid=0 2024-09-21T03:25:02.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 client.26345 shutdown 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 msgr2=0x556b72d015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbba0005ab0 tx=0x7fbba0005540 comp rx=0 tx=0).stop 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 shutdown_connections 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x556b728d5da0 0x556b72d7cdc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b72942070 0x556b72d02050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbb95ffb640 0 client.26345 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556b72921390 0x556b72d01b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 --2- 172.21.15.83:0/3836116831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b729191b0 0x556b72d015d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 >> 172.21.15.83:0/3836116831 conn(0x556b72936460 msgr2=0x556b72cfece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 shutdown_connections 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.720+0000 7fbbbb0c0740 1 -- 172.21.15.83:0/3836116831 wait complete. 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.721+0000 7fbbbb0c0740 20 client.26345 trim_cache size 0 max 16384 2024-09-21T03:25:02.735 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.721+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 msgr2=0x7fbb9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.721+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 0x7fbb9c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb9801e000 tx=0x7fbb9800ea70 comp rx=0 tx=0).stop 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.721+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 msgr2=0x7fbbb0072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.721+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0072910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009fd0 tx=0x7fbba800ef10 comp rx=0 tx=0).stop 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 shutdown_connections 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fbb9c0b6680 0x7fbb9c0b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb010c020 0x7fbbb0072910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb010b6d0 0x7fbbb006d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 --2- 172.21.15.83:0/586022681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb0105d90 0x7fbbb006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 >> 172.21.15.83:0/586022681 conn(0x7fbbb01017e0 msgr2=0x7fbbb0103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 shutdown_connections 2024-09-21T03:25:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:02.722+0000 7fbbb4b4c640 1 -- 172.21.15.83:0/586022681 wait complete. 2024-09-21T03:25:03.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:02 smithi083 ceph-mon[31219]: pgmap v580: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-21T03:25:03.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:02 smithi193 ceph-mon[35224]: pgmap v580: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-21T03:25:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:02 smithi142 ceph-mon[34512]: pgmap v580: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-21T03:25:03.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:25:03.888 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.674+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1646282977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0108d30 msgr2=0x7f32b010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.674+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1646282977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0108d30 0x7f32b010f560 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009ca0 tx=0x7f32a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1646282977 shutdown_connections 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1646282977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0108d30 0x7f32b010f560 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1646282977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b01083e0 0x7f32b01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1646282977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b0102aa0 0x7f32b0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1646282977 >> 172.21.15.83:0/1646282977 conn(0x7f32b00797e0 msgr2=0x7f32b0079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1646282977 shutdown_connections 2024-09-21T03:25:04.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.675+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1646282977 wait complete. 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.676+0000 7f32b5d6b640 1 Processor -- start 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.676+0000 7f32b5d6b640 1 -- start start 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 0x7f32b019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 0x7f32b019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b0108d30 0x7f32b01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01129a0 con 0x7f32b0102aa0 2024-09-21T03:25:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0112820 con 0x7f32b0108d30 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32b5d6b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0112b20 con 0x7f32b01083e0 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 0x7f32b019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 0x7f32b019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47510/0 (socket says 172.21.15.83:47510) 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 0x7f32b019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32af7fe640 1 -- 172.21.15.83:0/1512322961 learned_addr learned my addr 172.21.15.83:0/1512322961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32affff640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b0108d30 0x7f32b01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 -- 172.21.15.83:0/1512322961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b0108d30 msgr2=0x7f32b01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b0108d30 0x7f32b01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 -- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 msgr2=0x7f32b019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 0x7f32b019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.677+0000 7f32aeffd640 1 -- 172.21.15.83:0/1512322961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01a2630 con 0x7f32b01083e0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.678+0000 7f32aeffd640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 0x7f32b019cc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9c0 tx=0x7f32a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.678+0000 7f32acff9640 1 -- 172.21.15.83:0/1512322961 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a400ccc0 con 0x7f32b01083e0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.678+0000 7f32acff9640 1 -- 172.21.15.83:0/1512322961 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a40108e0 con 0x7f32b01083e0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.678+0000 7f32acff9640 1 -- 172.21.15.83:0/1512322961 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40187c0 con 0x7f32b01083e0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.679+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b01a2920 con 0x7f32b01083e0 2024-09-21T03:25:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.680+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a2d20 con 0x7f32b01083e0 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 Processor -- start 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 -- start start 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32acff9640 1 -- 172.21.15.83:0/1512322961 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f32a401f020 con 0x7f32b01083e0 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf1676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf168120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9bf0ddea0 con 0x55d9bedc0070 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.682+0000 7f32bc2df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d9bf0cd2d0 con 0x55d9bed971b0 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32bc2df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9bf0d10e0 con 0x55d9bed9f390 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf1676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52516/0 (socket says 172.21.15.83:52516) 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 -- 172.21.15.83:0/719031133 learned_addr learned my addr 172.21.15.83:0/719031133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32aeffd640 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 -- 172.21.15.83:0/719031133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 msgr2=0x55d9bf168120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf168120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 -- 172.21.15.83:0/719031133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 msgr2=0x55d9bf1676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf1676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32af7fe640 1 -- 172.21.15.83:0/719031133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bee2b650 con 0x55d9bed971b0 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.683+0000 7f32acff9640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 0x7f32980b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.684+0000 7f32acff9640 1 -- 172.21.15.83:0/1512322961 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f32a40e7750 con 0x7f32b01083e0 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.685+0000 7f32af7fe640 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009e10 tx=0x7f32a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e974088cdbee615c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.685+0000 7f3295ffb640 1 -- 172.21.15.83:0/719031133 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a001e070 con 0x55d9bed971b0 2024-09-21T03:25:04.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.685+0000 7f3295ffb640 1 -- 172.21.15.83:0/719031133 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a0019ae0 con 0x55d9bed971b0 2024-09-21T03:25:04.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.685+0000 7f3295ffb640 1 -- 172.21.15.83:0/719031133 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a000fb30 con 0x55d9bed971b0 2024-09-21T03:25:04.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.685+0000 7f32affff640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 0x7f32980b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 -- 172.21.15.83:0/719031133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 msgr2=0x55d9bf167be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32affff640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 0x7f32980b8ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f32a802f500 tx=0x7f32a802f310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009e10 tx=0x7f32a0011810 comp rx=0 tx=0).stop 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 -- 172.21.15.83:0/719031133 shutdown_connections 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf167be0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 --2- 172.21.15.83:0/719031133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf1676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 -- 172.21.15.83:0/719031133 >> 172.21.15.83:0/719031133 conn(0x55d9bedb4460 msgr2=0x55d9bf165590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 -- 172.21.15.83:0/719031133 shutdown_connections 2024-09-21T03:25:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.686+0000 7f32bc2df740 1 -- 172.21.15.83:0/719031133 wait complete. 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 -- start start 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 Processor -- start 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf17be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf181400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9bef2e700 con 0x55d9bedc0070 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d9bf068160 con 0x55d9bed971b0 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9bf175e80 con 0x55d9bed9f390 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf17be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32bc2df740 10 client.0 _ll_register_callbacks cb 0x55d9bf0819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32aeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf181400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.687+0000 7f32aeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51100/0 (socket says 172.21.15.83:51100) 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 -- 172.21.15.83:0/2497780040 learned_addr learned my addr 172.21.15.83:0/2497780040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 -- 172.21.15.83:0/2497780040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 msgr2=0x55d9bf17be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf17be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 -- 172.21.15.83:0/2497780040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 msgr2=0x55d9bf181400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf181400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 -- 172.21.15.83:0/2497780040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bf095c90 con 0x55d9bed9f390 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32affff640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf181400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32aeffd640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ad260 tx=0x7f32a40accd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da236aeb2e0d2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f328effd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40e52f0 con 0x55d9bed9f390 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f32a4010630 con 0x55d9bed9f390 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a0019ae0 con 0x55d9bed9f390 2024-09-21T03:25:04.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f32a40b2040 con 0x55d9bed9f390 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32bc2df740 10 client.26335 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.688+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d9bf18a3b0 con 0x55d9bed9f390 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a0019ae0 con 0x55d9bed9f390 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f32a40a8fd0 con 0x55d9bed9f390 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 10 client.26335 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 10 client.26335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 20 client.26335 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed7e9f0 0x55d9bf18d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 4 client.26335 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9bf0dfdf0 con 0x55d9bed7e9f0 2024-09-21T03:25:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.689+0000 7f32af7fe640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed7e9f0 0x55d9bf18d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.691+0000 7f328effd640 10 client.26335 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.691+0000 7f32af7fe640 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed7e9f0 0x55d9bf18d880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a3d37c0e93a3d5b server_cookie=85c1fe0a6f152348 in_seq=0 out_seq=0 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.691+0000 7f328effd640 1 -- 172.21.15.83:0/2497780040 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9bf0dfdf0 con 0x55d9bed7e9f0 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.691+0000 7f328effd640 10 client.26335 handle_command_reply: tid=0 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.691+0000 7f32bc2df740 1 client.26335 shutdown 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 msgr2=0x55d9bf17c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32a40ad260 tx=0x7f32a40accd0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 shutdown_connections 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bedc0070 0x55d9bf181400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed7e9f0 0x55d9bf18d880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bed9f390 0x55d9bf17c3e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 --2- 172.21.15.83:0/2497780040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf17be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 >> 172.21.15.83:0/2497780040 conn(0x55d9bedb4460 msgr2=0x55d9bf16a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f328effd640 0 client.26335 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 shutdown_connections 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 1 -- 172.21.15.83:0/2497780040 wait complete. 2024-09-21T03:25:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.692+0000 7f32bc2df740 20 client.26335 trim_cache size 0 max 16384 2024-09-21T03:25:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 Processor -- start 2024-09-21T03:25:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 -- start start 2024-09-21T03:25:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf1751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf17a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01ad410 con 0x55d9bed9f390 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01aef50 con 0x55d9bed971b0 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.710+0000 7f32bc2df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01adae0 con 0x55d9bedc0070 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf1751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47534/0 (socket says 172.21.15.83:47534) 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 -- 172.21.15.83:0/3451619829 learned_addr learned my addr 172.21.15.83:0/3451619829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32affff640 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf17a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 -- 172.21.15.83:0/3451619829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 msgr2=0x55d9bf17a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf17a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 -- 172.21.15.83:0/3451619829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 msgr2=0x55d9bf1751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf1751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 -- 172.21.15.83:0/3451619829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bf169990 con 0x55d9bed9f390 2024-09-21T03:25:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32af7fe640 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf1751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32aeffd640 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f32a4098040 tx=0x7f32a40ace00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4574cf3ad9d02440 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32957fa640 1 -- 172.21.15.83:0/3451619829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40a9bb0 con 0x55d9bed9f390 2024-09-21T03:25:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32957fa640 1 -- 172.21.15.83:0/3451619829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a40ac5d0 con 0x55d9bed9f390 2024-09-21T03:25:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.711+0000 7f32957fa640 1 -- 172.21.15.83:0/3451619829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40a9110 con 0x55d9bed9f390 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.712+0000 7f32bc2df740 1 -- 172.21.15.83:0/3451619829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 msgr2=0x55d9bf17a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.712+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f32a4098040 tx=0x7f32a40ace00 comp rx=0 tx=0).stop 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 -- 172.21.15.83:0/3451619829 shutdown_connections 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf17a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed9f390 0x55d9bf17a480 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3451619829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed971b0 0x55d9bf1751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 -- 172.21.15.83:0/3451619829 >> 172.21.15.83:0/3451619829 conn(0x55d9bedb4460 msgr2=0x55d9bf176d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 -- 172.21.15.83:0/3451619829 shutdown_connections 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.713+0000 7f32bc2df740 1 -- 172.21.15.83:0/3451619829 wait complete. 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.714+0000 7f32bc2df740 1 -- start start 2024-09-21T03:25:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.714+0000 7f32bc2df740 1 Processor -- start 2024-09-21T03:25:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 0x55d9bf181fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed9f390 0x55d9bf182510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 0x55d9bf181fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 0x55d9bf181fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47536/0 (socket says 172.21.15.83:47536) 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf187020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32af7fe640 1 -- 172.21.15.83:0/3548138337 learned_addr learned my addr 172.21.15.83:0/3548138337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01ae880 con 0x55d9bed971b0 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01ae1b0 con 0x55d9bed9f390 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01ab430 con 0x55d9bedc0070 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32affff640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf187020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32aeffd640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed9f390 0x55d9bf182510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.715+0000 7f32bc2df740 10 client.0 _ll_register_callbacks cb 0x7f32a400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 -- 172.21.15.83:0/3548138337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed9f390 msgr2=0x55d9bf182510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed9f390 0x55d9bf182510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 -- 172.21.15.83:0/3548138337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 msgr2=0x55d9bf181fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 0x55d9bf181fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bf187560 con 0x55d9bedc0070 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32affff640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf187020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd57e5b635f799aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f328effd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8058060 con 0x55d9bedc0070 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f32a80043d0 con 0x55d9bedc0070 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.716+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a40ac5d0 con 0x55d9bedc0070 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.717+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f32a80045d0 con 0x55d9bedc0070 2024-09-21T03:25:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.717+0000 7f32bc2df740 10 client.26341 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.717+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d9bf1f8240 con 0x55d9bedc0070 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.717+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a40ac5d0 con 0x55d9bedc0070 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f32a80305c0 con 0x55d9bedc0070 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 10 client.26341 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 10 client.26341 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 20 client.26341 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed53da0 0x55d9bf1fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 4 client.26341 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9bf0db8a0 con 0x55d9bed53da0 2024-09-21T03:25:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.718+0000 7f32af7fe640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed53da0 0x55d9bf1fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.719+0000 7f32af7fe640 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed53da0 0x55d9bf1fa8d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c455455a634ceb3 server_cookie=f935305a41221b6c in_seq=0 out_seq=0 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.719+0000 7f328effd640 10 client.26341 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.719+0000 7f328effd640 1 -- 172.21.15.83:0/3548138337 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9bf0db8a0 con 0x55d9bed53da0 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.719+0000 7f328effd640 10 client.26341 handle_command_reply: tid=0 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 client.26341 shutdown 2024-09-21T03:25:04.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 msgr2=0x55d9bf187020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf187020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a80095c0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 shutdown_connections 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d9bed53da0 0x55d9bf1fa8d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9bedc0070 0x55d9bf187020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9bed9f390 0x55d9bf182510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f328effd640 0 client.26341 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:04.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 --2- 172.21.15.83:0/3548138337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9bed971b0 0x55d9bf181fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 >> 172.21.15.83:0/3548138337 conn(0x55d9bedb4460 msgr2=0x55d9bf164e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.720+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 shutdown_connections 2024-09-21T03:25:04.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32bc2df740 1 -- 172.21.15.83:0/3548138337 wait complete. 2024-09-21T03:25:04.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32bc2df740 20 client.26341 trim_cache size 0 max 16384 2024-09-21T03:25:04.731 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 msgr2=0x7f32980b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 0x7f32980b8ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f32a802f500 tx=0x7f32a802f310 comp rx=0 tx=0).stop 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 msgr2=0x7f32b019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.721+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 0x7f32b019cc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9c0 tx=0x7f32a400ee80 comp rx=0 tx=0).stop 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 shutdown_connections 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f32980b67d0 0x7f32980b8ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b0108d30 0x7f32b01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b01083e0 0x7f32b019cc60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 --2- 172.21.15.83:0/1512322961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102aa0 0x7f32b019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 >> 172.21.15.83:0/1512322961 conn(0x7f32b00797e0 msgr2=0x7f32b006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 shutdown_connections 2024-09-21T03:25:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:04.722+0000 7f32b5d6b640 1 -- 172.21.15.83:0/1512322961 wait complete. 2024-09-21T03:25:05.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:04 smithi193 ceph-mon[35224]: pgmap v581: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:25:05.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:04 smithi193 ceph-mon[35224]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:25:05.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:04 smithi083 ceph-mon[31219]: pgmap v581: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:25:05.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:04 smithi083 ceph-mon[31219]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:25:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:04 smithi142 ceph-mon[34512]: pgmap v581: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:25:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:04 smithi142 ceph-mon[34512]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:25:05.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:25:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 465.0 s, 1591.6 tps, lat 64.550 ms stddev 130.128 2024-09-21T03:25:05.937 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:06.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.494+0000 7f92f27fc640 1 -- 172.21.15.83:0/3625252657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0318a0 con 0x7f92f4102630 2024-09-21T03:25:06.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.495+0000 7f92fa0fc640 1 -- 172.21.15.83:0/3625252657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 msgr2=0x7f92f4106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/3625252657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f4106190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009900 tx=0x7f92ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 -- 172.21.15.83:0/3625252657 shutdown_connections 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/3625252657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f4106190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/3625252657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4101ce0 0x7f92f41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/3625252657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f41053f0 0x7f92f4105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 -- 172.21.15.83:0/3625252657 >> 172.21.15.83:0/3625252657 conn(0x7f92f4077c60 msgr2=0x7f92f40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 -- 172.21.15.83:0/3625252657 shutdown_connections 2024-09-21T03:25:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 -- 172.21.15.83:0/3625252657 wait complete. 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.496+0000 7f92fa0fc640 1 Processor -- start 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 -- start start 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4101ce0 0x7f92f41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f41053f0 0x7f92f41a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92f411a1b0 con 0x7f92f4102630 2024-09-21T03:25:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92f411a030 con 0x7f92f4101ce0 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92fa0fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f411a330 con 0x7f92f41053f0 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47556/0 (socket says 172.21.15.83:47556) 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.497+0000 7f92f2ffd640 1 -- 172.21.15.83:0/4050245874 learned_addr learned my addr 172.21.15.83:0/4050245874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 -- 172.21.15.83:0/4050245874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f41053f0 msgr2=0x7f92f41a64a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:25:06.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f41053f0 0x7f92f41a64a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 -- 172.21.15.83:0/4050245874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4101ce0 msgr2=0x7f92f41a0c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:25:06.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4101ce0 0x7f92f41a0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 -- 172.21.15.83:0/4050245874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f41a6bd0 con 0x7f92f4102630 2024-09-21T03:25:06.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.498+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b450 tx=0x7f92e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.500+0000 7f92f0ff9640 1 -- 172.21.15.83:0/4050245874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8015020 con 0x7f92f4102630 2024-09-21T03:25:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.500+0000 7f92f0ff9640 1 -- 172.21.15.83:0/4050245874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8011d40 con 0x7f92f4102630 2024-09-21T03:25:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.500+0000 7f92f0ff9640 1 -- 172.21.15.83:0/4050245874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e800f340 con 0x7f92f4102630 2024-09-21T03:25:06.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.506+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f41a6ec0 con 0x7f92f4102630 2024-09-21T03:25:06.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.506+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41ad620 con 0x7f92f4102630 2024-09-21T03:25:06.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 Processor -- start 2024-09-21T03:25:06.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 -- start start 2024-09-21T03:25:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18be8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18be9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b18b5eea0 con 0x562b188181b0 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562b18b4e2d0 con 0x562b18841070 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.512+0000 7f9300670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b18b520e0 con 0x562b18820390 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.514+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.515+0000 7f92f0ff9640 1 -- 172.21.15.83:0/4050245874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f92e8018020 con 0x7f92f4102630 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.515+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52560/0 (socket says 172.21.15.83:52560) 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.515+0000 7f92f3fff640 1 -- 172.21.15.83:0/150375441 learned_addr learned my addr 172.21.15.83:0/150375441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.515+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18be9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.519+0000 7f92f3fff640 1 -- 172.21.15.83:0/150375441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 msgr2=0x562b18be9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.516+0000 7f92f37fe640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18be8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.522+0000 7f92f3fff640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18be9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.522+0000 7f92f3fff640 1 -- 172.21.15.83:0/150375441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 msgr2=0x562b18be8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.523+0000 7f92f3fff640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18be8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.523+0000 7f92f3fff640 1 -- 172.21.15.83:0/150375441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b188ac650 con 0x562b18841070 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.523+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18be9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.524+0000 7f92f37fe640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18be8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.529+0000 7f92f3fff640 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a930 tx=0x7f92ec0040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a588fe74f795d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.530+0000 7f92d9ffb640 1 -- 172.21.15.83:0/150375441 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec03e070 con 0x562b18841070 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.531+0000 7f92d9ffb640 1 -- 172.21.15.83:0/150375441 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ec005500 con 0x562b18841070 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.531+0000 7f92d9ffb640 1 -- 172.21.15.83:0/150375441 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec035040 con 0x562b18841070 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.531+0000 7f92f0ff9640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 0x7f92dc0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.531+0000 7f92f0ff9640 1 -- 172.21.15.83:0/4050245874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f92e80e6e80 con 0x7f92f4102630 2024-09-21T03:25:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.532+0000 7f9300670740 1 -- 172.21.15.83:0/150375441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 msgr2=0x562b18be86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.532+0000 7f9300670740 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a930 tx=0x7f92ec0040f0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f92f37fe640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 0x7f92dc0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f9300670740 1 -- 172.21.15.83:0/150375441 shutdown_connections 2024-09-21T03:25:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f9300670740 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18be9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f9300670740 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18be8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f9300670740 1 --2- 172.21.15.83:0/150375441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18841070 0x562b18be86a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.533+0000 7f9300670740 1 -- 172.21.15.83:0/150375441 >> 172.21.15.83:0/150375441 conn(0x562b18835460 msgr2=0x562b18be6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.536+0000 7f92f37fe640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 0x7f92dc0b8c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x562b18be7320 tx=0x7f92e4007340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.536+0000 7f9300670740 1 -- 172.21.15.83:0/150375441 shutdown_connections 2024-09-21T03:25:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.536+0000 7f9300670740 1 -- 172.21.15.83:0/150375441 wait complete. 2024-09-21T03:25:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.536+0000 7f9300670740 1 -- start start 2024-09-21T03:25:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.537+0000 7f9300670740 1 Processor -- start 2024-09-21T03:25:06.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.537+0000 7f9300670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 0x562b18bf3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18bf42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18c11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b189af700 con 0x562b18841070 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562b18ae9160 con 0x562b188181b0 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b18c07e30 con 0x562b18820390 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f9300670740 10 client.0 _ll_register_callbacks cb 0x562b18b029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:06.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.538+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18c11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.539+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18c11150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47570/0 (socket says 172.21.15.83:47570) 2024-09-21T03:25:06.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.539+0000 7f92f3fff640 1 -- 172.21.15.83:0/1555113227 learned_addr learned my addr 172.21.15.83:0/1555113227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:06.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.539+0000 7f92f37fe640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 0x562b18bf3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.539+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18bf42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 -- 172.21.15.83:0/1555113227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 msgr2=0x562b18bf42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18bf42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 -- 172.21.15.83:0/1555113227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 msgr2=0x562b18c11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18c11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 -- 172.21.15.83:0/1555113227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b18b16c90 con 0x562b188181b0 2024-09-21T03:25:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.540+0000 7f92f37fe640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 0x562b18bf3d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007540 tx=0x7f92e4004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c050722ec1b222b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.541+0000 7f92d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.541+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4020020 con 0x562b188181b0 2024-09-21T03:25:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.541+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f92e400e8f0 con 0x562b188181b0 2024-09-21T03:25:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.541+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec005500 con 0x562b188181b0 2024-09-21T03:25:06.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.543+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f92e400eaf0 con 0x562b188181b0 2024-09-21T03:25:06.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.547+0000 7f9300670740 10 client.26369 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:06.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.547+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562b18c13a90 con 0x562b188181b0 2024-09-21T03:25:06.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.551+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ec005500 con 0x562b188181b0 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.552+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f92e40243a0 con 0x562b188181b0 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 10 client.26369 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 10 client.26369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 20 client.26369 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187ff9f0 0x562b18c16f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 4 client.26369 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b18b60df0 con 0x562b187ff9f0 2024-09-21T03:25:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.553+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187ff9f0 0x562b18c16f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.556+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187ff9f0 0x562b18c16f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49be173c96d7ff3a server_cookie=f10743cc63cf07fc in_seq=0 out_seq=0 2024-09-21T03:25:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.556+0000 7f92d2ffd640 10 client.26369 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:06.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.557+0000 7f92d2ffd640 1 -- 172.21.15.83:0/1555113227 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b18b60df0 con 0x562b187ff9f0 2024-09-21T03:25:06.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.557+0000 7f92d2ffd640 10 client.26369 handle_command_reply: tid=0 2024-09-21T03:25:06.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.557+0000 7f9300670740 1 client.26369 shutdown 2024-09-21T03:25:06.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 msgr2=0x562b18bf3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 0x562b18bf3d70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007540 tx=0x7f92e4004b50 comp rx=0 tx=0).stop 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 shutdown_connections 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187ff9f0 0x562b18c16f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18c11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f92d2ffd640 0 client.26369 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18820390 0x562b18bf42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 --2- 172.21.15.83:0/1555113227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b188181b0 0x562b18bf3d70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 >> 172.21.15.83:0/1555113227 conn(0x562b18835460 msgr2=0x562b18beb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 shutdown_connections 2024-09-21T03:25:06.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.559+0000 7f9300670740 1 -- 172.21.15.83:0/1555113227 wait complete. 2024-09-21T03:25:06.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.560+0000 7f9300670740 20 client.26369 trim_cache size 0 max 16384 2024-09-21T03:25:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.582+0000 7f9300670740 1 Processor -- start 2024-09-21T03:25:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.582+0000 7f9300670740 1 -- start start 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.582+0000 7f9300670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 0x562b18bf5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.582+0000 7f9300670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.582+0000 7f9300670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18be5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f9300670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4112b00 con 0x562b18841070 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f9300670740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41af090 con 0x562b18820390 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f9300670740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41adc20 con 0x562b188181b0 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 0x562b18bf5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52584/0 (socket says 172.21.15.83:52584) 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 0x562b18bf5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51154/0 (socket says 172.21.15.83:51154) 2024-09-21T03:25:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 -- 172.21.15.83:0/770127023 learned_addr learned my addr 172.21.15.83:0/770127023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f3fff640 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18be5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 -- 172.21.15.83:0/770127023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 msgr2=0x562b18bf5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 0x562b18bf5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 -- 172.21.15.83:0/770127023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 msgr2=0x562b18be5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18be5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 -- 172.21.15.83:0/770127023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b18bf2270 con 0x562b18820390 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.583+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a9260 tx=0x7f92e80d0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de9e9dc784a21a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.584+0000 7f92d97fa640 1 -- 172.21.15.83:0/770127023 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80ad390 con 0x562b18820390 2024-09-21T03:25:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.584+0000 7f92d97fa640 1 -- 172.21.15.83:0/770127023 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e80b1770 con 0x562b18820390 2024-09-21T03:25:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.584+0000 7f92d97fa640 1 -- 172.21.15.83:0/770127023 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80ad390 con 0x562b18820390 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.585+0000 7f9300670740 1 -- 172.21.15.83:0/770127023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 msgr2=0x562b18be9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.585+0000 7f9300670740 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a9260 tx=0x7f92e80d0040 comp rx=0 tx=0).stop 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 -- 172.21.15.83:0/770127023 shutdown_connections 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b18841070 0x562b18be5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18be9de0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 --2- 172.21.15.83:0/770127023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b188181b0 0x562b18bf5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 -- 172.21.15.83:0/770127023 >> 172.21.15.83:0/770127023 conn(0x562b18835460 msgr2=0x562b18be96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 -- 172.21.15.83:0/770127023 shutdown_connections 2024-09-21T03:25:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 -- 172.21.15.83:0/770127023 wait complete. 2024-09-21T03:25:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 -- start start 2024-09-21T03:25:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.586+0000 7f9300670740 1 Processor -- start 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18c0d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18c0d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18c0d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47578/0 (socket says 172.21.15.83:47578) 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51160/0 (socket says 172.21.15.83:51160) 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:52588/0 (socket says 172.21.15.83:52588) 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f92f3fff640 1 -- 172.21.15.83:0/3793620610 learned_addr learned my addr 172.21.15.83:0/3793620610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ae9c0 con 0x562b188181b0 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ae2f0 con 0x562b18820390 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.587+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4106250 con 0x562b18841070 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f9300670740 10 client.0 _ll_register_callbacks cb 0x7f92e4007610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 -- 172.21.15.83:0/3793620610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 msgr2=0x562b18c09470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 -- 172.21.15.83:0/3793620610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 msgr2=0x562b18c0d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18c0d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b18c09c00 con 0x562b18820390 2024-09-21T03:25:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f3fff640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e80b1770 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92f2ffd640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a9260 tx=0x7f92e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29da477ecbaf560 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92d2ffd640 10 client.26375 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92d2ffd640 1 -- 172.21.15.83:0/3793620610 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80ad390 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92d2ffd640 1 -- 172.21.15.83:0/3793620610 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f92e8010590 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f92d2ffd640 1 -- 172.21.15.83:0/3793620610 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f92e80b66b0 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f9300670740 10 client.26375 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.588+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=44}) -- 0x562b18c818a0 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f92d2ffd640 1 -- 172.21.15.83:0/3793620610 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f92e80e4ce0 con 0x562b18820390 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 10 client.26375 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 10 client.26375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 20 client.26375 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187d4da0 0x562b18c83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 4 client.26375 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b18b5c8a0 con 0x562b187d4da0 2024-09-21T03:25:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.589+0000 7f92f37fe640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187d4da0 0x562b18c83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.590+0000 7f92d2ffd640 10 client.26375 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.590+0000 7f92f37fe640 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187d4da0 0x562b18c83e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=908b446c8c47acac server_cookie=5ac708fe97247a6d in_seq=0 out_seq=0 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f92d2ffd640 1 -- 172.21.15.83:0/3793620610 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b18b5c8a0 con 0x562b187d4da0 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f92d2ffd640 10 client.26375 handle_command_reply: tid=0 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 client.26375 shutdown 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 msgr2=0x562b18c08e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92e80a9260 tx=0x7f92e80046c0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 shutdown_connections 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x562b187d4da0 0x562b18c83e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b18841070 0x562b18c09470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f92d2ffd640 0 client.26375 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b18820390 0x562b18c08e40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 --2- 172.21.15.83:0/3793620610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b188181b0 0x562b18c0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.592+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 >> 172.21.15.83:0/3793620610 conn(0x562b18835460 msgr2=0x562b18bf59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 shutdown_connections 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f9300670740 1 -- 172.21.15.83:0/3793620610 wait complete. 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f9300670740 20 client.26375 trim_cache size 0 max 16384 2024-09-21T03:25:06.598 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:06.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 msgr2=0x7f92dc0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 0x7f92dc0b8c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x562b18be7320 tx=0x7f92e4007340 comp rx=0 tx=0).stop 2024-09-21T03:25:06.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 msgr2=0x7f92f41a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.593+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b450 tx=0x7f92e800b910 comp rx=0 tx=0).stop 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 shutdown_connections 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92dc0b6780 0x7f92dc0b8c90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f41053f0 0x7f92f41a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4102630 0x7f92f41a1200 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 --2- 172.21.15.83:0/4050245874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f4101ce0 0x7f92f41a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 >> 172.21.15.83:0/4050245874 conn(0x7f92f4077c60 msgr2=0x7f92f4100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 shutdown_connections 2024-09-21T03:25:06.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:06.594+0000 7f92fa0fc640 1 -- 172.21.15.83:0/4050245874 wait complete. 2024-09-21T03:25:07.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:06 smithi193 ceph-mon[35224]: pgmap v583: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-21T03:25:07.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:06 smithi083 ceph-mon[31219]: pgmap v583: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-21T03:25:07.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:06 smithi142 ceph-mon[34512]: pgmap v583: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-21T03:25:08.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:25:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:09.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:08 smithi083 ceph-mon[31219]: pgmap v584: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.98k op/s 2024-09-21T03:25:09.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:08 smithi193 ceph-mon[35224]: pgmap v584: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.98k op/s 2024-09-21T03:25:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:08 smithi142 ceph-mon[34512]: pgmap v584: 97 pgs: 97 active+clean; 4.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 23 MiB/s wr, 1.98k op/s 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 -- 172.21.15.83:0/1102476096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb65411cc30 msgr2=0x7fb65411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 --2- 172.21.15.83:0/1102476096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb65411cc30 0x7fb65411f140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009860 tx=0x7fb63c02a380 comp rx=0 tx=0).stop 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 -- 172.21.15.83:0/1102476096 shutdown_connections 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 --2- 172.21.15.83:0/1102476096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb65411cc30 0x7fb65411f140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 --2- 172.21.15.83:0/1102476096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65410a9e0 0x7fb65410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 --2- 172.21.15.83:0/1102476096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb654111f40 0x7fb65410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.532+0000 7fb65b05f640 1 -- 172.21.15.83:0/1102476096 >> 172.21.15.83:0/1102476096 conn(0x7fb654070ec0 msgr2=0x7fb6540712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.534+0000 7fb65b05f640 1 -- 172.21.15.83:0/1102476096 shutdown_connections 2024-09-21T03:25:09.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.534+0000 7fb65b05f640 1 -- 172.21.15.83:0/1102476096 wait complete. 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 Processor -- start 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 -- start start 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb65410a9e0 0x7fb6541a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb654111f40 0x7fb6541a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541228b0 con 0x7fb65411cc30 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb654122730 con 0x7fb65410a9e0 2024-09-21T03:25:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.536+0000 7fb65b05f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb654122a30 con 0x7fb654111f40 2024-09-21T03:25:09.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34696/0 (socket says 172.21.15.83:34696) 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 -- 172.21.15.83:0/470710685 learned_addr learned my addr 172.21.15.83:0/470710685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb653fff640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb654111f40 0x7fb6541a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb658dfb640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb65410a9e0 0x7fb6541a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 -- 172.21.15.83:0/470710685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb654111f40 msgr2=0x7fb6541a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb654111f40 0x7fb6541a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 -- 172.21.15.83:0/470710685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb65410a9e0 msgr2=0x7fb6541a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb65410a9e0 0x7fb6541a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.540+0000 7fb6595fc640 1 -- 172.21.15.83:0/470710685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6541af1b0 con 0x7fb65411cc30 2024-09-21T03:25:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.541+0000 7fb6595fc640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02a970 tx=0x7fb63c030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.541+0000 7fb651ffb640 1 -- 172.21.15.83:0/470710685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c037050 con 0x7fb65411cc30 2024-09-21T03:25:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.542+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6541af440 con 0x7fb65411cc30 2024-09-21T03:25:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.542+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6541b5c00 con 0x7fb65411cc30 2024-09-21T03:25:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.542+0000 7fb651ffb640 1 -- 172.21.15.83:0/470710685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c043070 con 0x7fb65411cc30 2024-09-21T03:25:09.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.543+0000 7fb651ffb640 1 -- 172.21.15.83:0/470710685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003be0 con 0x7fb65411cc30 2024-09-21T03:25:09.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.547+0000 7fb651ffb640 1 -- 172.21.15.83:0/470710685 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb63c004040 con 0x7fb65411cc30 2024-09-21T03:25:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.548+0000 7fb651ffb640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 0x7fb6400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.549+0000 7fb651ffb640 1 -- 172.21.15.83:0/470710685 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb63c107d10 con 0x7fb65411cc30 2024-09-21T03:25:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.549+0000 7fb658dfb640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 0x7fb6400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.549+0000 7fb6615d3740 1 Processor -- start 2024-09-21T03:25:09.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.549+0000 7fb6615d3740 1 -- start start 2024-09-21T03:25:09.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.550+0000 7fb658dfb640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 0x7fb6400b8a70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.550+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e78e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.550+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.550+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e78f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.550+0000 7fb6615d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a88e704ea0 con 0x55a88e3c6390 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb6615d3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a88e6f42d0 con 0x55a88e3e7070 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb6615d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a88e6f80e0 con 0x55a88e3be1b0 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e78e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e78e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38070/0 (socket says 172.21.15.83:38070) 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb658dfb640 1 -- 172.21.15.83:0/144347667 learned_addr learned my addr 172.21.15.83:0/144347667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb653fff640 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.551+0000 7fb6595fc640 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e78f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 -- 172.21.15.83:0/144347667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 msgr2=0x55a88e78ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 -- 172.21.15.83:0/144347667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 msgr2=0x55a88e78e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e78e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 -- 172.21.15.83:0/144347667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88e452650 con 0x55a88e3c6390 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.552+0000 7fb6595fc640 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e78f120 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb63c105420 tx=0x7fb63c0cd8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2af5a91620c6da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.553+0000 7fb64affd640 1 -- 172.21.15.83:0/144347667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0328f0 con 0x55a88e3c6390 2024-09-21T03:25:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.553+0000 7fb64affd640 1 -- 172.21.15.83:0/144347667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c038040 con 0x55a88e3c6390 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.553+0000 7fb64affd640 1 -- 172.21.15.83:0/144347667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0d2600 con 0x55a88e3c6390 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.553+0000 7fb6615d3740 1 -- 172.21.15.83:0/144347667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 msgr2=0x55a88e78f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.553+0000 7fb6615d3740 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e78f120 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb63c105420 tx=0x7fb63c0cd8f0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 -- 172.21.15.83:0/144347667 shutdown_connections 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e78f120 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 --2- 172.21.15.83:0/144347667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e78e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 -- 172.21.15.83:0/144347667 >> 172.21.15.83:0/144347667 conn(0x55a88e3db460 msgr2=0x55a88e78c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 -- 172.21.15.83:0/144347667 shutdown_connections 2024-09-21T03:25:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 -- 172.21.15.83:0/144347667 wait complete. 2024-09-21T03:25:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 -- start start 2024-09-21T03:25:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.554+0000 7fb6615d3740 1 Processor -- start 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e799bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e79a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6615d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a88e555700 con 0x55a88e3c6390 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb653fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e799bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e799bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39254/0 (socket says 172.21.15.83:39254) 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb658dfb640 1 -- 172.21.15.83:0/3979634790 learned_addr learned my addr 172.21.15.83:0/3979634790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e79a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.555+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a88e68f160 con 0x55a88e3e7070 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a88e79cc40 con 0x55a88e3be1b0 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6615d3740 10 client.0 _ll_register_callbacks cb 0x55a88e6a89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 -- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 msgr2=0x55a88e799bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e799bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 -- 172.21.15.83:0/3979634790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 msgr2=0x55a88e79a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88e6bcc90 con 0x55a88e3e7070 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb653fff640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63c038040 con 0x55a88e3e7070 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e79a670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb63c105420 tx=0x7fb63c02a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ba9a75a2fc69b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.556+0000 7fb633fff640 10 client.26387 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.558+0000 7fb633fff640 1 -- 172.21.15.83:0/3979634790 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0d0e60 con 0x55a88e3e7070 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.558+0000 7fb633fff640 1 -- 172.21.15.83:0/3979634790 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb63c0c9d90 con 0x55a88e3e7070 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.558+0000 7fb633fff640 1 -- 172.21.15.83:0/3979634790 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb63c0328f0 con 0x55a88e3e7070 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.558+0000 7fb6615d3740 10 client.26387 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.558+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55a88e7a86a0 con 0x55a88e3e7070 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb633fff640 1 -- 172.21.15.83:0/3979634790 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fb63c1056a0 con 0x55a88e3e7070 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 10 client.26387 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 10 client.26387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:25:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 20 client.26387 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3a59f0 0x55a88e7aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 4 client.26387 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a88e706df0 con 0x55a88e3a59f0 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.559+0000 7fb658dfb640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3a59f0 0x55a88e7aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.560+0000 7fb633fff640 10 client.26387 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:25:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.560+0000 7fb658dfb640 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3a59f0 0x55a88e7aba00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe07a3f7d7212465 server_cookie=4965928db15c1749 in_seq=0 out_seq=0 2024-09-21T03:25:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb633fff640 1 -- 172.21.15.83:0/3979634790 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a88e706df0 con 0x55a88e3a59f0 2024-09-21T03:25:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb633fff640 10 client.26387 handle_command_reply: tid=0 2024-09-21T03:25:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 client.26387 shutdown 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 msgr2=0x55a88e79a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e79a670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb63c105420 tx=0x7fb63c02a380 comp rx=0 tx=0).stop 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 shutdown_connections 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3a59f0 0x55a88e7aba00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb633fff640 0 client.26387 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3e7070 0x55a88e79a670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3979634790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e799bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 >> 172.21.15.83:0/3979634790 conn(0x55a88e3db460 msgr2=0x55a88e791830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 shutdown_connections 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 1 -- 172.21.15.83:0/3979634790 wait complete. 2024-09-21T03:25:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.561+0000 7fb6615d3740 20 client.26387 trim_cache size 0 max 16384 2024-09-21T03:25:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.579+0000 7fb6615d3740 1 Processor -- start 2024-09-21T03:25:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.579+0000 7fb6615d3740 1 -- start start 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.579+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e796de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38098/0 (socket says 172.21.15.83:38098) 2024-09-21T03:25:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb658dfb640 1 -- 172.21.15.83:0/3541736403 learned_addr learned my addr 172.21.15.83:0/3541736403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb653fff640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb63400e820 con 0x55a88e3c6390 2024-09-21T03:25:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb634010360 con 0x55a88e3a59f0 2024-09-21T03:25:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb63400eef0 con 0x55a88e3be1b0 2024-09-21T03:25:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.580+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e796de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 -- 172.21.15.83:0/3541736403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 msgr2=0x55a88e78c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 -- 172.21.15.83:0/3541736403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 msgr2=0x55a88e796de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e796de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 -- 172.21.15.83:0/3541736403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88e7a5e80 con 0x55a88e3a59f0 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb6595fc640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e796de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb658dfb640 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb644011ef0 tx=0x7fb644014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178fceb3e65835da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb64a7fc640 1 -- 172.21.15.83:0/3541736403 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440211d0 con 0x55a88e3a59f0 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb64a7fc640 1 -- 172.21.15.83:0/3541736403 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6440219b0 con 0x55a88e3a59f0 2024-09-21T03:25:09.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.581+0000 7fb64a7fc640 1 -- 172.21.15.83:0/3541736403 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64402ba20 con 0x55a88e3a59f0 2024-09-21T03:25:09.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.582+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 msgr2=0x55a88e78bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.582+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb644011ef0 tx=0x7fb644014000 comp rx=0 tx=0).stop 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 shutdown_connections 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e796de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e78c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 --2- 172.21.15.83:0/3541736403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e78bef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 >> 172.21.15.83:0/3541736403 conn(0x55a88e3db460 msgr2=0x55a88e798470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 shutdown_connections 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.583+0000 7fb6615d3740 1 -- 172.21.15.83:0/3541736403 wait complete. 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.584+0000 7fb6615d3740 1 -- start start 2024-09-21T03:25:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.584+0000 7fb6615d3740 1 Processor -- start 2024-09-21T03:25:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e79ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb63400fc90 con 0x55a88e3c6390 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb63400f5c0 con 0x55a88e3a59f0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb63400c820 con 0x55a88e3be1b0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6615d3740 10 client.0 _ll_register_callbacks cb 0x7fb644009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6595fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39272/0 (socket says 172.21.15.83:39272) 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6595fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34742/0 (socket says 172.21.15.83:34742) 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb658dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e79ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 -- 172.21.15.83:0/4024063253 learned_addr learned my addr 172.21.15.83:0/4024063253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 -- 172.21.15.83:0/4024063253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 msgr2=0x55a88e79ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e79ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 -- 172.21.15.83:0/4024063253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 msgr2=0x55a88e79e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb653fff640 1 -- 172.21.15.83:0/4024063253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88e815260 con 0x55a88e3be1b0 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.585+0000 7fb6595fc640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.586+0000 7fb653fff640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005fd0 tx=0x7fb64c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbbbec7c2e59835f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.586+0000 7fb633fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.586+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c022060 con 0x55a88e3be1b0 2024-09-21T03:25:09.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.586+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb64c003b70 con 0x55a88e3be1b0 2024-09-21T03:25:09.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.586+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6440219b0 con 0x55a88e3be1b0 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb64c011e80 con 0x55a88e3be1b0 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 10 client.26368 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a88e8164e0 con 0x55a88e3be1b0 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c003d70 con 0x55a88e3be1b0 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fb64c01c070 con 0x55a88e3be1b0 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 10 client.26368 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 10 client.26368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 20 client.26368 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3e7070 0x55a88e818cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 4 client.26368 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.587+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb6440219b0 con 0x55a88e3e7070 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb658dfb640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3e7070 0x55a88e818cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb633fff640 10 client.26368 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb658dfb640 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3e7070 0x55a88e818cb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79c2ea3b7035ecd7 server_cookie=819ba5df9ce1a198 in_seq=0 out_seq=0 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb633fff640 1 -- 172.21.15.83:0/4024063253 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb6440219b0 con 0x55a88e3e7070 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb633fff640 10 client.26368 handle_command_reply: tid=0 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb6615d3740 1 client.26368 shutdown 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 msgr2=0x55a88e79e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.588+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005fd0 tx=0x7fb64c0117b0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 shutdown_connections 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55a88e3e7070 0x55a88e818cb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88e3c6390 0x55a88e79e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a88e3be1b0 0x55a88e79e410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 --2- 172.21.15.83:0/4024063253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a88e3a59f0 0x55a88e79ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 >> 172.21.15.83:0/4024063253 conn(0x55a88e3db460 msgr2=0x55a88e78be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb633fff640 0 client.26368 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 shutdown_connections 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 1 -- 172.21.15.83:0/4024063253 wait complete. 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.589+0000 7fb6615d3740 20 client.26368 trim_cache size 0 max 16384 2024-09-21T03:25:09.594 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.591+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 msgr2=0x7fb6400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.591+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 0x7fb6400b8a70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644009260 comp rx=0 tx=0).stop 2024-09-21T03:25:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 msgr2=0x7fb6541aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02a970 tx=0x7fb63c030d70 comp rx=0 tx=0).stop 2024-09-21T03:25:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 shutdown_connections 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb6400b6560 0x7fb6400b8a70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb65411cc30 0x7fb6541aea80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb654111f40 0x7fb6541a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 --2- 172.21.15.83:0/470710685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb65410a9e0 0x7fb6541a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 >> 172.21.15.83:0/470710685 conn(0x7fb654070ec0 msgr2=0x7fb65411e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 shutdown_connections 2024-09-21T03:25:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:09.592+0000 7fb65b05f640 1 -- 172.21.15.83:0/470710685 wait complete. 2024-09-21T03:25:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 470.0 s, 2770.9 tps, lat 36.106 ms stddev 42.121 2024-09-21T03:25:11.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:10 smithi193 ceph-mon[35224]: pgmap v585: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-09-21T03:25:11.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:10 smithi083 ceph-mon[31219]: pgmap v585: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-09-21T03:25:11.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:10 smithi142 ceph-mon[34512]: pgmap v585: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-09-21T03:25:12.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:25:12.966 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:13.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:12 smithi083 ceph-mon[31219]: pgmap v586: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 24 MiB/s wr, 1.97k op/s 2024-09-21T03:25:13.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:12 smithi193 ceph-mon[35224]: pgmap v586: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 24 MiB/s wr, 1.97k op/s 2024-09-21T03:25:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:12 smithi142 ceph-mon[34512]: pgmap v586: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 24 MiB/s wr, 1.97k op/s 2024-09-21T03:25:13.677 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:13.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.545+0000 7fb270d2d640 1 -- 172.21.15.83:0/2778108911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 msgr2=0x7fb26c1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.545+0000 7fb270d2d640 1 --2- 172.21.15.83:0/2778108911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1085c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb26002a410 comp rx=0 tx=0).stop 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 -- 172.21.15.83:0/2778108911 shutdown_connections 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 --2- 172.21.15.83:0/2778108911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c108b00 0x7fb26c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 --2- 172.21.15.83:0/2778108911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1085c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 --2- 172.21.15.83:0/2778108911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c102870 0x7fb26c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 -- 172.21.15.83:0/2778108911 >> 172.21.15.83:0/2778108911 conn(0x7fb26c0fe2c0 msgr2=0x7fb26c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 -- 172.21.15.83:0/2778108911 shutdown_connections 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.546+0000 7fb270d2d640 1 -- 172.21.15.83:0/2778108911 wait complete. 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.547+0000 7fb270d2d640 1 Processor -- start 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb270d2d640 1 -- start start 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb270d2d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb270d2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb270d2d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c108b00 0x7fb26c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39302/0 (socket says 172.21.15.83:39302) 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb26b7fe640 1 -- 172.21.15.83:0/3762587492 learned_addr learned my addr 172.21.15.83:0/3762587492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.548+0000 7fb26affd640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26bfff640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c108b00 0x7fb26c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c112790 con 0x7fb26c1081b0 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c112610 con 0x7fb26c108b00 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c112910 con 0x7fb26c102870 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26b7fe640 1 -- 172.21.15.83:0/3762587492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c108b00 msgr2=0x7fb26c19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c108b00 0x7fb26c19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26b7fe640 1 -- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 msgr2=0x7fb26c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.549+0000 7fb26b7fe640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c19f1b0 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009f50 tx=0x7fb25c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb268ff9640 1 -- 172.21.15.83:0/3762587492 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c01e070 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb268ff9640 1 -- 172.21.15.83:0/3762587492 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c00cce0 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c19f4a0 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb268ff9640 1 -- 172.21.15.83:0/3762587492 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c019830 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.550+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c06ae60 con 0x7fb26c102870 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.552+0000 7fb278503740 1 Processor -- start 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.552+0000 7fb278503740 1 -- start start 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a31a16aea0 con 0x55a319e2c390 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a31a15a2d0 con 0x55a319e241b0 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb278503740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a31a15e0e0 con 0x55a319e4d070 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39306/0 (socket says 172.21.15.83:39306) 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 -- 172.21.15.83:0/282076569 learned_addr learned my addr 172.21.15.83:0/282076569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 -- 172.21.15.83:0/282076569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 msgr2=0x55a31a1f4da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a1f4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 -- 172.21.15.83:0/282076569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 msgr2=0x55a31a1f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a1f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.553+0000 7fb26bfff640 1 -- 172.21.15.83:0/282076569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a319eb8650 con 0x55a319e4d070 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.554+0000 7fb26bfff640 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb264009f50 tx=0x7fb2640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3d3908f3e2d7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.554+0000 7fb251ffb640 1 -- 172.21.15.83:0/282076569 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264004d20 con 0x55a319e4d070 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.554+0000 7fb251ffb640 1 -- 172.21.15.83:0/282076569 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb26400f9f0 con 0x55a319e4d070 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.554+0000 7fb251ffb640 1 -- 172.21.15.83:0/282076569 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264005580 con 0x55a319e4d070 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb268ff9640 1 -- 172.21.15.83:0/3762587492 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb25c021070 con 0x7fb26c102870 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 -- 172.21.15.83:0/282076569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 msgr2=0x55a31a1f4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb264009f50 tx=0x7fb2640118d0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 -- 172.21.15.83:0/282076569 shutdown_connections 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a1f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 --2- 172.21.15.83:0/282076569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e4d070 0x55a31a1f4860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 -- 172.21.15.83:0/282076569 >> 172.21.15.83:0/282076569 conn(0x55a319e41460 msgr2=0x55a31a1f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 -- 172.21.15.83:0/282076569 shutdown_connections 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb278503740 1 -- 172.21.15.83:0/282076569 wait complete. 2024-09-21T03:25:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.555+0000 7fb268ff9640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 0x7fb2540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.556+0000 7fb278503740 1 -- start start 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.556+0000 7fb26affd640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 0x7fb2540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.556+0000 7fb268ff9640 1 -- 172.21.15.83:0/3762587492 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb25c0e7390 con 0x7fb26c102870 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.556+0000 7fb278503740 1 Processor -- start 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.556+0000 7fb26affd640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 0x7fb2540b8aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb260009230 tx=0x7fb260031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e241b0 0x55a31a204330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 0x55a31a210730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a319fbb700 con 0x55a319e2c390 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a31a0f5160 con 0x55a319e4d070 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a31a2073a0 con 0x55a319e241b0 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34798/0 (socket says 172.21.15.83:34798) 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26affd640 1 -- 172.21.15.83:0/2200930031 learned_addr learned my addr 172.21.15.83:0/2200930031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb278503740 10 client.0 _ll_register_callbacks cb 0x55a31a10e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 0x55a31a210730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 -- 172.21.15.83:0/2200930031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e241b0 msgr2=0x55a31a204330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e241b0 0x55a31a204330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 -- 172.21.15.83:0/2200930031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 msgr2=0x55a31a204870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.557+0000 7fb26bfff640 1 -- 172.21.15.83:0/2200930031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31a122c90 con 0x55a319e4d070 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb26affd640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb26bfff640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 0x55a31a210730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb26401b040 tx=0x7fb26400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e914636ed09281 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb24affd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264029070 con 0x55a319e4d070 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb26400f9f0 con 0x55a319e4d070 2024-09-21T03:25:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb264004e50 con 0x55a319e4d070 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb264035050 con 0x55a319e4d070 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb278503740 10 client.26402 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.558+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a31a213120 con 0x55a319e4d070 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb26400f9f0 con 0x55a319e4d070 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fb264023870 con 0x55a319e4d070 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 10 client.26402 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 20 client.26402 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e0b9f0 0x55a31a2164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 4 client.26402 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a31a16cdf0 con 0x55a319e0b9f0 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e0b9f0 0x55a31a2164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.559+0000 7fb24affd640 10 client.26402 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.560+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e0b9f0 0x55a31a2164e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120fd00cd0091ef0 server_cookie=d61a0190b2551305 in_seq=0 out_seq=0 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.560+0000 7fb24affd640 1 -- 172.21.15.83:0/2200930031 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a31a16cdf0 con 0x55a319e0b9f0 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.560+0000 7fb24affd640 10 client.26402 handle_command_reply: tid=0 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.560+0000 7fb278503740 1 client.26402 shutdown 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 msgr2=0x55a31a210730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 0x55a31a210730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb26401b040 tx=0x7fb26400f7f0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 shutdown_connections 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e0b9f0 0x55a31a2164e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e4d070 0x55a31a210730 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb24affd640 0 client.26402 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e2c390 0x55a31a204870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 --2- 172.21.15.83:0/2200930031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e241b0 0x55a31a204330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 >> 172.21.15.83:0/2200930031 conn(0x55a319e41460 msgr2=0x55a31a1f0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 shutdown_connections 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 1 -- 172.21.15.83:0/2200930031 wait complete. 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.561+0000 7fb278503740 20 client.26402 trim_cache size 0 max 16384 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.576+0000 7fb278503740 1 Processor -- start 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.576+0000 7fb278503740 1 -- start start 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a201540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e2c390 0x55a31a201a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1ad360 con 0x55a319e0b9f0 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1aeea0 con 0x55a319e241b0 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb278503740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1ada30 con 0x55a319e2c390 2024-09-21T03:25:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34804/0 (socket says 172.21.15.83:34804) 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 -- 172.21.15.83:0/4156805732 learned_addr learned my addr 172.21.15.83:0/4156805732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26affd640 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a201540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26bfff640 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e2c390 0x55a31a201a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 -- 172.21.15.83:0/4156805732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e2c390 msgr2=0x55a31a201a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e2c390 0x55a31a201a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 -- 172.21.15.83:0/4156805732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 msgr2=0x55a31a201540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a201540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 -- 172.21.15.83:0/4156805732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31a1f6d70 con 0x55a319e0b9f0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0aa4c0 tx=0x7fb25c0a99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cb76a4828a4335 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.577+0000 7fb2517fa640 1 -- 172.21.15.83:0/4156805732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0d4030 con 0x55a319e0b9f0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.578+0000 7fb2517fa640 1 -- 172.21.15.83:0/4156805732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c0b1c80 con 0x55a319e0b9f0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.578+0000 7fb2517fa640 1 -- 172.21.15.83:0/4156805732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0d0d80 con 0x55a319e0b9f0 2024-09-21T03:25:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.578+0000 7fb278503740 1 -- 172.21.15.83:0/4156805732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 msgr2=0x55a31a1f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0aa4c0 tx=0x7fb25c0a99c0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 -- 172.21.15.83:0/4156805732 shutdown_connections 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e2c390 0x55a31a201a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e241b0 0x55a31a201540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 --2- 172.21.15.83:0/4156805732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e0b9f0 0x55a31a1f5fa0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 -- 172.21.15.83:0/4156805732 >> 172.21.15.83:0/4156805732 conn(0x55a319e41460 msgr2=0x55a31a1f5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 -- 172.21.15.83:0/4156805732 shutdown_connections 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.579+0000 7fb278503740 1 -- 172.21.15.83:0/4156805732 wait complete. 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 -- start start 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 Processor -- start 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 0x55a31a20a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e2c390 0x55a31a20a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1ae7d0 con 0x55a319e241b0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1ae100 con 0x55a319e2c390 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb26affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 0x55a31a20a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb278503740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1ab360 con 0x55a319e0b9f0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb26b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39324/0 (socket says 172.21.15.83:39324) 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.580+0000 7fb26affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 0x55a31a20a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34810/0 (socket says 172.21.15.83:34810) 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 -- 172.21.15.83:0/2353515621 learned_addr learned my addr 172.21.15.83:0/2353515621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb278503740 10 client.0 _ll_register_callbacks cb 0x7fb2640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26bfff640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e2c390 0x55a31a20a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 -- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e2c390 msgr2=0x55a31a20a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e2c390 0x55a31a20a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 -- 172.21.15.83:0/2353515621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 msgr2=0x55a31a20a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 0x55a31a20a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 -- 172.21.15.83:0/2353515621 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31a20aec0 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb26b7fe640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ad090 tx=0x7fb25c0e4f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beeeed91f4a5333e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb24affd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0c3020 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7fb25c0ac6b0 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25c0b1c80 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb25c0d4070 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb278503740 10 client.26383 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.581+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a31a1fab90 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c0b1c80 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fb25c0abf20 con 0x55a319e0b9f0 2024-09-21T03:25:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 10 client.26383 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 10 client.26383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 20 client.26383 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e4d070 0x55a31a1fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 4 client.26383 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a31a1688a0 con 0x55a319e4d070 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.582+0000 7fb26affd640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e4d070 0x55a31a1fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb26affd640 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e4d070 0x55a31a1fd370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7c151fc6aeef13 server_cookie=391cdba246bc625b in_seq=0 out_seq=0 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb24affd640 10 client.26383 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb24affd640 1 -- 172.21.15.83:0/2353515621 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a31a1688a0 con 0x55a319e4d070 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb24affd640 10 client.26383 handle_command_reply: tid=0 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 client.26383 shutdown 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 msgr2=0x55a31a20cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ad090 tx=0x7fb25c0e4f70 comp rx=0 tx=0).stop 2024-09-21T03:25:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 shutdown_connections 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55a319e4d070 0x55a31a1fd370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a319e2c390 0x55a31a20a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a319e241b0 0x55a31a20a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb24affd640 0 client.26383 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 --2- 172.21.15.83:0/2353515621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a319e0b9f0 0x55a31a20cdc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 >> 172.21.15.83:0/2353515621 conn(0x55a319e41460 msgr2=0x55a31a1f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 shutdown_connections 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 1 -- 172.21.15.83:0/2353515621 wait complete. 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.583+0000 7fb278503740 20 client.26383 trim_cache size 0 max 16384 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 msgr2=0x7fb2540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 0x7fb2540b8aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb260009230 tx=0x7fb260031400 comp rx=0 tx=0).stop 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 msgr2=0x7fb26c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009f50 tx=0x7fb25c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 shutdown_connections 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb2540b6590 0x7fb2540b8aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb26c108b00 0x7fb26c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26c1081b0 0x7fb26c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 --2- 172.21.15.83:0/3762587492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb26c102870 0x7fb26c199240 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 >> 172.21.15.83:0/3762587492 conn(0x7fb26c0fe2c0 msgr2=0x7fb26c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 shutdown_connections 2024-09-21T03:25:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:13.584+0000 7fb270d2d640 1 -- 172.21.15.83:0/3762587492 wait complete. 2024-09-21T03:25:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:14 smithi083 ceph-mon[31219]: pgmap v587: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T03:25:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:14 smithi083 ceph-mon[31219]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:25:15.229 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:14 smithi142 ceph-mon[34512]: pgmap v587: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T03:25:15.229 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:14 smithi142 ceph-mon[34512]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:25:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:14 smithi193 ceph-mon[35224]: pgmap v587: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T03:25:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:14 smithi193 ceph-mon[35224]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:25:15.894 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:25:15.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:25:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 475.0 s, 1719.8 tps, lat 57.734 ms stddev 62.003 2024-09-21T03:25:16.221 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 -- 172.21.15.83:0/3742814747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa810de10 msgr2=0x7f2aa8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 --2- 172.21.15.83:0/3742814747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa810de10 0x7f2aa8110300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ca0 tx=0x7f2aa002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 -- 172.21.15.83:0/3742814747 shutdown_connections 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 --2- 172.21.15.83:0/3742814747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa810de10 0x7f2aa8110300 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 --2- 172.21.15.83:0/3742814747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa8105300 0x7f2aa810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.807+0000 7f2aaea39640 1 --2- 172.21.15.83:0/3742814747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.808+0000 7f2aaea39640 1 -- 172.21.15.83:0/3742814747 >> 172.21.15.83:0/3742814747 conn(0x7f2aa8079730 msgr2=0x7f2aa8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.808+0000 7f2aaea39640 1 -- 172.21.15.83:0/3742814747 shutdown_connections 2024-09-21T03:25:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.808+0000 7f2aaea39640 1 -- 172.21.15.83:0/3742814747 wait complete. 2024-09-21T03:25:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.808+0000 7f2aaea39640 1 Processor -- start 2024-09-21T03:25:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.808+0000 7f2aaea39640 1 -- start start 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa810de10 0x7f2aa819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8113ac0 con 0x7f2aa8105300 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8113940 con 0x7f2aa81049b0 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aaea39640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8113c40 con 0x7f2aa810de10 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34830/0 (socket says 172.21.15.83:34830) 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 -- 172.21.15.83:0/1671631762 learned_addr learned my addr 172.21.15.83:0/1671631762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa810de10 0x7f2aa819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 -- 172.21.15.83:0/1671631762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa810de10 msgr2=0x7f2aa819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa810de10 0x7f2aa819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 -- 172.21.15.83:0/1671631762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 msgr2=0x7f2aa819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa77fe640 1 -- 172.21.15.83:0/1671631762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81a0420 con 0x7f2aa8105300 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.809+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e9f0 tx=0x7f2a9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aa57fa640 1 -- 172.21.15.83:0/1671631762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00ccc0 con 0x7f2aa8105300 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81a0710 con 0x7f2aa8105300 2024-09-21T03:25:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aa57fa640 1 -- 172.21.15.83:0/1671631762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0108e0 con 0x7f2aa8105300 2024-09-21T03:25:16.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aa57fa640 1 -- 172.21.15.83:0/1671631762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c018860 con 0x7f2aa8105300 2024-09-21T03:25:16.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.810+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81025d0 con 0x7f2aa8105300 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 Processor -- start 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 -- start start 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2aa57fa640 1 -- 172.21.15.83:0/1671631762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2a9c018a00 con 0x7f2aa8105300 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f28259d490 0x55f28296d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 0x55f28296deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f2828e3e60 con 0x55f2825c6350 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f2828d3290 con 0x55f28259d490 2024-09-21T03:25:16.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f2828d70a0 con 0x55f2825a5670 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2aacfd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2aacfd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34838/0 (socket says 172.21.15.83:34838) 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.813+0000 7f2aacfd6640 1 -- 172.21.15.83:0/1779363210 learned_addr learned my addr 172.21.15.83:0/1779363210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f28259d490 0x55f28296d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 0x55f28296deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 -- 172.21.15.83:0/1779363210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 msgr2=0x55f28296deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 0x55f28296deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 -- 172.21.15.83:0/1779363210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f28259d490 msgr2=0x55f28296d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f28259d490 0x55f28296d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 -- 172.21.15.83:0/1779363210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f282631930 con 0x55f2825c6350 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 0x55f28296deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa57fa640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ca0 tx=0x7f2aa0009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d1c87e883dfe48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.814+0000 7f2aa57fa640 1 -- 172.21.15.83:0/1671631762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2a9c0e7940 con 0x7f2aa8105300 2024-09-21T03:25:16.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.815+0000 7f2a8e7fc640 1 -- 172.21.15.83:0/1779363210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0031cf0 con 0x55f2825c6350 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.815+0000 7f2a8e7fc640 1 -- 172.21.15.83:0/1779363210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa0036030 con 0x55f2825c6350 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.815+0000 7f2a8e7fc640 1 -- 172.21.15.83:0/1779363210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00042a0 con 0x55f2825c6350 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 -- 172.21.15.83:0/1779363210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 msgr2=0x55f28296d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ca0 tx=0x7f2aa0009430 comp rx=0 tx=0).stop 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 0x7f2a900b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f28296c0b0 tx=0x7f2a98005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 -- 172.21.15.83:0/1779363210 shutdown_connections 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825a5670 0x55f28296deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f28259d490 0x55f28296d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/1779363210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825c6350 0x55f28296d430 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 -- 172.21.15.83:0/1779363210 >> 172.21.15.83:0/1779363210 conn(0x55f2825ba740 msgr2=0x55f28296b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 -- 172.21.15.83:0/1779363210 shutdown_connections 2024-09-21T03:25:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.816+0000 7f2ab4fad740 1 -- 172.21.15.83:0/1779363210 wait complete. 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.817+0000 7f2ab4fad740 1 -- start start 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 Processor -- start 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282979030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282979570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f282734890 con 0x55f28259d490 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f28286dd40 con 0x55f2825a5670 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.818+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f28297ba30 con 0x55f2825c6350 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2ab4fad740 10 client.0 _ll_register_callbacks cb 0x55f2828d1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39346/0 (socket says 172.21.15.83:39346) 2024-09-21T03:25:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 -- 172.21.15.83:0/129722276 learned_addr learned my addr 172.21.15.83:0/129722276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:16.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282979030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 -- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 msgr2=0x55f282979570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282979570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 -- 172.21.15.83:0/129722276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 msgr2=0x55f282979030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282979030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.819+0000 7f2aacfd6640 1 -- 172.21.15.83:0/129722276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28298db50 con 0x55f2825c6350 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.820+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0031f10 tx=0x7f2aa00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40eeb269ea4cd832 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.820+0000 7f2a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.820+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa004a040 con 0x55f2825c6350 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.820+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2aa0031650 con 0x55f2825c6350 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.820+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa0036030 con 0x55f2825c6350 2024-09-21T03:25:16.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2aa0003970 con 0x55f2825c6350 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 10 client.26395 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f282990460 con 0x55f2825c6350 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa0036030 con 0x55f2825c6350 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa004d030 con 0x55f2825c6350 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 10 client.26395 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 10 client.26395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 20 client.26395 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f282584cd0 0x55f2829937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 4 client.26395 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2828e5db0 con 0x55f282584cd0 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.821+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f282584cd0 0x55f2829937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.822+0000 7f2a877fe640 10 client.26395 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:16.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.822+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f282584cd0 0x55f2829937c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a434b80873261a server_cookie=ca1020ada9659a68 in_seq=0 out_seq=0 2024-09-21T03:25:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2a877fe640 1 -- 172.21.15.83:0/129722276 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2828e5db0 con 0x55f282584cd0 2024-09-21T03:25:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2a877fe640 10 client.26395 handle_command_reply: tid=0 2024-09-21T03:25:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 client.26395 shutdown 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 msgr2=0x55f28298d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0031f10 tx=0x7f2aa00094a0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 shutdown_connections 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f282584cd0 0x55f2829937c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f28298d590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2a877fe640 0 client.26395 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282979570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/129722276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282979030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.823+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 >> 172.21.15.83:0/129722276 conn(0x55f2825ba740 msgr2=0x55f282970610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.824+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 shutdown_connections 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.824+0000 7f2ab4fad740 1 -- 172.21.15.83:0/129722276 wait complete. 2024-09-21T03:25:16.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.824+0000 7f2ab4fad740 20 client.26395 trim_cache size 0 max 16384 2024-09-21T03:25:16.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.849+0000 7f2ab4fad740 1 Processor -- start 2024-09-21T03:25:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.850+0000 7f2ab4fad740 1 -- start start 2024-09-21T03:25:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 0x55f28298d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 0x55f28296b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825c6350 0x55f2829726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 0x55f28298d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 0x55f28298d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39348/0 (socket says 172.21.15.83:39348) 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ad470 con 0x55f2825a5670 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2aa7fff640 1 -- 172.21.15.83:0/3627883403 learned_addr learned my addr 172.21.15.83:0/3627883403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81aefb0 con 0x55f2825c6350 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.851+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81adb40 con 0x55f28259d490 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825c6350 0x55f2829726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 0x55f28296b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 -- 172.21.15.83:0/3627883403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 msgr2=0x55f28298d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 0x55f28298d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 -- 172.21.15.83:0/3627883403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825c6350 msgr2=0x55f2829726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825c6350 0x55f2829726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.852+0000 7f2aa77fe640 1 -- 172.21.15.83:0/3627883403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28296aa70 con 0x55f2825a5670 2024-09-21T03:25:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.853+0000 7f2aa77fe640 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 0x55f28296b370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0e50e0 tx=0x7f2a9c0ed4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40b6a4522b170f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.854+0000 7f2a8dffb640 1 -- 172.21.15.83:0/3627883403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0a9730 con 0x55f2825a5670 2024-09-21T03:25:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.854+0000 7f2a8dffb640 1 -- 172.21.15.83:0/3627883403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0ad3a0 con 0x55f2825a5670 2024-09-21T03:25:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.854+0000 7f2a8dffb640 1 -- 172.21.15.83:0/3627883403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0ad000 con 0x55f2825a5670 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 msgr2=0x55f28296b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 0x55f28296b370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0e50e0 tx=0x7f2a9c0ed4f0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 shutdown_connections 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825c6350 0x55f2829726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f2825a5670 0x55f28296b370 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/3627883403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f28259d490 0x55f28298d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.856+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 >> 172.21.15.83:0/3627883403 conn(0x55f2825ba740 msgr2=0x55f28296ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.858+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 shutdown_connections 2024-09-21T03:25:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.858+0000 7f2ab4fad740 1 -- 172.21.15.83:0/3627883403 wait complete. 2024-09-21T03:25:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.859+0000 7f2ab4fad740 1 -- start start 2024-09-21T03:25:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.859+0000 7f2ab4fad740 1 Processor -- start 2024-09-21T03:25:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282989f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282984590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282989f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2ab4fad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f282984ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282989f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34874/0 (socket says 172.21.15.83:34874) 2024-09-21T03:25:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282984590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.860+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282984590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38200/0 (socket says 172.21.15.83:38200) 2024-09-21T03:25:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.861+0000 7f2aa7fff640 1 -- 172.21.15.83:0/2884097627 learned_addr learned my addr 172.21.15.83:0/2884097627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.861+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f282984ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.861+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ae8e0 con 0x55f28259d490 2024-09-21T03:25:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.861+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ae210 con 0x55f2825a5670 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.861+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ab490 con 0x55f2825c6350 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2ab4fad740 10 client.0 _ll_register_callbacks cb 0x7f2a98009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2aacfd6640 1 -- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 msgr2=0x55f282984590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282984590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2aacfd6640 1 -- 172.21.15.83:0/2884097627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 msgr2=0x55f282989f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282989f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.862+0000 7f2aacfd6640 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2829fe0a0 con 0x55f2825c6350 2024-09-21T03:25:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.863+0000 7f2aacfd6640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f282984ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0030660 tx=0x7f2aa0009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d9a8fe686a8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.863+0000 7f2a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.863+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa002fa20 con 0x55f2825c6350 2024-09-21T03:25:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.863+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2aa002fbc0 con 0x55f2825c6350 2024-09-21T03:25:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.863+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9c0ad3a0 con 0x55f2825c6350 2024-09-21T03:25:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.866+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2aa0009b20 con 0x55f2825c6350 2024-09-21T03:25:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.866+0000 7f2ab4fad740 10 client.26401 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.866+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f2829ff1f0 con 0x55f2825c6350 2024-09-21T03:25:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.868+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0ad3a0 con 0x55f2825c6350 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.868+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa004d040 con 0x55f2825c6350 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 10 client.26401 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 20 client.26401 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f28255a080 0x55f282a019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 4 client.26401 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:25:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f2828e1860 con 0x55f28255a080 2024-09-21T03:25:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.869+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f28255a080 0x55f282a019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.870+0000 7f2aa7fff640 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f28255a080 0x55f282a019d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b66e02c30740089 server_cookie=7b61c15db036c5fa in_seq=0 out_seq=0 2024-09-21T03:25:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.870+0000 7f2a877fe640 10 client.26401 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.873+0000 7f2a877fe640 1 -- 172.21.15.83:0/2884097627 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2828e1860 con 0x55f28255a080 2024-09-21T03:25:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.873+0000 7f2a877fe640 10 client.26401 handle_command_reply: tid=0 2024-09-21T03:25:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.873+0000 7f2ab4fad740 1 client.26401 shutdown 2024-09-21T03:25:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.884+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 msgr2=0x55f282984ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.884+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f282984ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0030660 tx=0x7f2aa0009430 comp rx=0 tx=0).stop 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 shutdown_connections 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f28255a080 0x55f282a019d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2825c6350 0x55f282984ca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f2825a5670 0x55f282984590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 --2- 172.21.15.83:0/2884097627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28259d490 0x55f282989f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 >> 172.21.15.83:0/2884097627 conn(0x55f2825ba740 msgr2=0x55f282979010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2a877fe640 0 client.26401 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 shutdown_connections 2024-09-21T03:25:16.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 1 -- 172.21.15.83:0/2884097627 wait complete. 2024-09-21T03:25:16.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.885+0000 7f2ab4fad740 20 client.26401 trim_cache size 0 max 16384 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e1d42f2c-77b3-4186-9c20-c7446db13fce", 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.887+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 msgr2=0x7f2a900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.887+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 0x7f2a900b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f28296c0b0 tx=0x7f2a98005f90 comp rx=0 tx=0).stop 2024-09-21T03:25:16.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.887+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 msgr2=0x7f2aa819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.887+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e9f0 tx=0x7f2a9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 shutdown_connections 2024-09-21T03:25:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2aa810de10 0x7f2aa819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2aa8105300 0x7f2aa819aa50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 --2- 172.21.15.83:0/1671631762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa81049b0 0x7f2aa819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.889+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 >> 172.21.15.83:0/1671631762 conn(0x7f2aa8079730 msgr2=0x7f2aa8106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.891+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 shutdown_connections 2024-09-21T03:25:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:16.891+0000 7f2aaea39640 1 -- 172.21.15.83:0/1671631762 wait complete. 2024-09-21T03:25:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:16 smithi193 ceph-mon[35224]: pgmap v589: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T03:25:17.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:16 smithi083 ceph-mon[31219]: pgmap v589: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T03:25:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:16 smithi142 ceph-mon[34512]: pgmap v589: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T03:25:17.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:25:18.061 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:18.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:25:18.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:25:18.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:25:18.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.618+0000 7f3269f98640 1 -- 172.21.15.83:0/3435040932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410a1b0 msgr2=0x7f326410a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.618+0000 7f3269f98640 1 --2- 172.21.15.83:0/3435040932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410a1b0 0x7f326410a5c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3254009900 tx=0x7f325403b420 comp rx=0 tx=0).stop 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 -- 172.21.15.83:0/3435040932 shutdown_connections 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 --2- 172.21.15.83:0/3435040932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f3264116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 --2- 172.21.15.83:0/3435040932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410faf0 0x7f326410ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 --2- 172.21.15.83:0/3435040932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410a1b0 0x7f326410a5c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 -- 172.21.15.83:0/3435040932 >> 172.21.15.83:0/3435040932 conn(0x7f32640780e0 msgr2=0x7f32641081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 -- 172.21.15.83:0/3435040932 shutdown_connections 2024-09-21T03:25:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.619+0000 7f3269f98640 1 -- 172.21.15.83:0/3435040932 wait complete. 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.620+0000 7f3269f98640 1 Processor -- start 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.620+0000 7f3269f98640 1 -- start start 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 0x7f32641169d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f32641131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f326406f620 con 0x7f3264110440 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f326406f4a0 con 0x7f326410faf0 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3269f98640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f326406f7a0 con 0x7f326410a1b0 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3263fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f32641131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 0x7f32641169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38212/0 (socket says 172.21.15.83:38212) 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 0x7f32641169d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39378/0 (socket says 172.21.15.83:39378) 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3263fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f32641131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34892/0 (socket says 172.21.15.83:34892) 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 -- 172.21.15.83:0/283778471 learned_addr learned my addr 172.21.15.83:0/283778471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 -- 172.21.15.83:0/283778471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 msgr2=0x7f32641169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 0x7f32641169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 -- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 msgr2=0x7f32641131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.621+0000 7f3262ffd640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f32641131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.622+0000 7f3262ffd640 1 -- 172.21.15.83:0/283778471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32641139f0 con 0x7f326410faf0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.622+0000 7f3262ffd640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.622+0000 7f3260ff9640 1 -- 172.21.15.83:0/283778471 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325800ccc0 con 0x7f326410faf0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.622+0000 7f3260ff9640 1 -- 172.21.15.83:0/283778471 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32580108e0 con 0x7f326410faf0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.622+0000 7f3260ff9640 1 -- 172.21.15.83:0/283778471 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580188c0 con 0x7f326410faf0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.623+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32641a4680 con 0x7f326410faf0 2024-09-21T03:25:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.623+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32641a4b40 con 0x7f326410faf0 2024-09-21T03:25:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 Processor -- start 2024-09-21T03:25:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 -- start start 2024-09-21T03:25:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 0x55c1ac3ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac022390 0x55c1ac3eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac360e50 con 0x55c1ac043070 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac350280 con 0x55c1ac022390 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f327050c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac354090 con 0x55c1ac01a1b0 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f3263fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 0x55c1ac3ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f3263fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 0x55c1ac3ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34896/0 (socket says 172.21.15.83:34896) 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f3263fff640 1 -- 172.21.15.83:0/1110628553 learned_addr learned my addr 172.21.15.83:0/1110628553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.626+0000 7f32637fe640 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f3260ff9640 1 -- 172.21.15.83:0/283778471 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3258010430 con 0x7f326410faf0 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 -- 172.21.15.83:0/1110628553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac022390 msgr2=0x55c1ac3eb0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac022390 0x55c1ac3eb0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 -- 172.21.15.83:0/1110628553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 msgr2=0x55c1ac3ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 0x55c1ac3ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 -- 172.21.15.83:0/1110628553 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ac0ae650 con 0x55c1ac01a1b0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f32637fe640 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3eab90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3264116720 tx=0x7f3254043bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d92fb5ae31ceb5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f3249ffb640 1 -- 172.21.15.83:0/1110628553 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254051070 con 0x55c1ac01a1b0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f3249ffb640 1 -- 172.21.15.83:0/1110628553 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3254004030 con 0x55c1ac01a1b0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f3249ffb640 1 -- 172.21.15.83:0/1110628553 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254046040 con 0x55c1ac01a1b0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.627+0000 7f3260ff9640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 0x7f324c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.628+0000 7f3263fff640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 0x7f324c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.628+0000 7f3260ff9640 1 -- 172.21.15.83:0/283778471 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f32580e7920 con 0x7f326410faf0 2024-09-21T03:25:18.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.628+0000 7f327050c740 1 -- 172.21.15.83:0/1110628553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 msgr2=0x55c1ac3eab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.628+0000 7f327050c740 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3eab90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3264116720 tx=0x7f3254043bd0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.628+0000 7f3263fff640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 0x7f324c0b8c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f325c003b60 tx=0x7f325c0096a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 -- 172.21.15.83:0/1110628553 shutdown_connections 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac022390 0x55c1ac3eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3eab90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 --2- 172.21.15.83:0/1110628553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac043070 0x55c1ac3ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 -- 172.21.15.83:0/1110628553 >> 172.21.15.83:0/1110628553 conn(0x55c1ac037460 msgr2=0x55c1ac3e8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.629+0000 7f327050c740 1 -- 172.21.15.83:0/1110628553 shutdown_connections 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.630+0000 7f327050c740 1 -- 172.21.15.83:0/1110628553 wait complete. 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.630+0000 7f327050c740 1 -- start start 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.630+0000 7f327050c740 1 Processor -- start 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac4020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac1b1700 con 0x55c1ac022390 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac2eb110 con 0x55c1ac043070 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac3f8dd0 con 0x55c1ac01a1b0 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f327050c740 10 client.0 _ll_register_callbacks cb 0x55c1ac304990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39396/0 (socket says 172.21.15.83:39396) 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 -- 172.21.15.83:0/632409690 learned_addr learned my addr 172.21.15.83:0/632409690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f3263fff640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac4020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f3262ffd640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 -- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 msgr2=0x55c1ac4020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac4020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 -- 172.21.15.83:0/632409690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 msgr2=0x55c1ac3f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.631+0000 7f32637fe640 1 -- 172.21.15.83:0/632409690 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ac318c40 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f32637fe640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3254009430 tx=0x7f3254005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83034fa110b6b978 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f3242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325405d070 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3254004030 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3254005460 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3254069050 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f327050c740 10 client.26413 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c1ac404ac0 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.632+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3254004030 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3254004c70 con 0x55c1ac01a1b0 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 10 client.26413 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 20 client.26413 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:18.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac0019f0 0x55c1ac408090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 4 client.26413 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:18.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1ac362da0 con 0x55c1ac0019f0 2024-09-21T03:25:18.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f3262ffd640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac0019f0 0x55c1ac408090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f3242ffd640 10 client.26413 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:18.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.633+0000 7f3262ffd640 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac0019f0 0x55c1ac408090 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a89e8841df0a65 server_cookie=cff9e340fd8a604 in_seq=0 out_seq=0 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f3242ffd640 1 -- 172.21.15.83:0/632409690 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1ac362da0 con 0x55c1ac0019f0 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f3242ffd640 10 client.26413 handle_command_reply: tid=0 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 client.26413 shutdown 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 msgr2=0x55c1ac3f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3254009430 tx=0x7f3254005ec0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 shutdown_connections 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac0019f0 0x55c1ac408090 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac4020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 --2- 172.21.15.83:0/632409690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac01a1b0 0x55c1ac3f5cf0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3254009430 tx=0x7f3254005ec0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 >> 172.21.15.83:0/632409690 conn(0x55c1ac037460 msgr2=0x55c1ac3ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f3242ffd640 0 client.26413 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 shutdown_connections 2024-09-21T03:25:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.635+0000 7f327050c740 1 -- 172.21.15.83:0/632409690 wait complete. 2024-09-21T03:25:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.636+0000 7f327050c740 20 client.26413 trim_cache size 0 max 16384 2024-09-21T03:25:18.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.662+0000 7f327050c740 1 Processor -- start 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.662+0000 7f327050c740 1 -- start start 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac401fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1abfd6da0 0x55c1ac3e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32641ad480 con 0x55c1abfd6da0 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32641aefc0 con 0x55c1ac043070 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f327050c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32641adb50 con 0x55c1ac022390 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39408/0 (socket says 172.21.15.83:39408) 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 -- 172.21.15.83:0/2132959299 learned_addr learned my addr 172.21.15.83:0/2132959299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f3263fff640 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1abfd6da0 0x55c1ac3e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f3262ffd640 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac401fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 -- 172.21.15.83:0/2132959299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 msgr2=0x55c1ac401fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac401fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 -- 172.21.15.83:0/2132959299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1abfd6da0 msgr2=0x55c1ac3e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1abfd6da0 0x55c1ac3e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.663+0000 7f32637fe640 1 -- 172.21.15.83:0/2132959299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ac3f6b60 con 0x55c1ac022390 2024-09-21T03:25:18.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.664+0000 7f32637fe640 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3254009400 tx=0x7f3254005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f30b010ce567ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.664+0000 7f32497fa640 1 -- 172.21.15.83:0/2132959299 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325405d040 con 0x55c1ac022390 2024-09-21T03:25:18.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.664+0000 7f32497fa640 1 -- 172.21.15.83:0/2132959299 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3254066020 con 0x55c1ac022390 2024-09-21T03:25:18.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.664+0000 7f32497fa640 1 -- 172.21.15.83:0/2132959299 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254051050 con 0x55c1ac022390 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.665+0000 7f327050c740 1 -- 172.21.15.83:0/2132959299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 msgr2=0x55c1ac3f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.665+0000 7f327050c740 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3254009400 tx=0x7f3254005f30 comp rx=0 tx=0).stop 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 -- 172.21.15.83:0/2132959299 shutdown_connections 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1abfd6da0 0x55c1ac3e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac401fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 --2- 172.21.15.83:0/2132959299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1ac022390 0x55c1ac3f61b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 -- 172.21.15.83:0/2132959299 >> 172.21.15.83:0/2132959299 conn(0x55c1ac037460 msgr2=0x55c1ac3f5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 -- 172.21.15.83:0/2132959299 shutdown_connections 2024-09-21T03:25:18.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.666+0000 7f327050c740 1 -- 172.21.15.83:0/2132959299 wait complete. 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.667+0000 7f327050c740 1 -- start start 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.667+0000 7f327050c740 1 Processor -- start 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac470a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32641ae8f0 con 0x55c1ac022390 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32641ae220 con 0x55c1ac043070 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32641ab480 con 0x55c1abfd6da0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f327050c740 10 client.0 _ll_register_callbacks cb 0x7f325c009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f3262ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39412/0 (socket says 172.21.15.83:39412) 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 -- 172.21.15.83:0/178179402 learned_addr learned my addr 172.21.15.83:0/178179402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f3263fff640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac470a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 -- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 msgr2=0x55c1ac470a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac470a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 -- 172.21.15.83:0/178179402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 msgr2=0x55c1ac3f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f32637fe640 1 -- 172.21.15.83:0/178179402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ac470fb0 con 0x55c1abfd6da0 2024-09-21T03:25:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.668+0000 7f3262ffd640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:18.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.669+0000 7f32637fe640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f325403b940 tx=0x7f3254004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfb8f0b04ecb9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:18.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.669+0000 7f3242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254051070 con 0x55c1abfd6da0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3254066020 con 0x55c1abfd6da0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f32540053f0 con 0x55c1abfd6da0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f325405d070 con 0x55c1abfd6da0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 10 client.26419 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c1ac472240 con 0x55c1abfd6da0 2024-09-21T03:25:18.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3254066020 con 0x55c1abfd6da0 2024-09-21T03:25:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f32540046c0 con 0x55c1abfd6da0 2024-09-21T03:25:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 10 client.26419 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 20 client.26419 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac4727a0 0x55c1ac474bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 4 client.26419 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.670+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1ac35e850 con 0x55c1ac4727a0 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.671+0000 7f3262ffd640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac4727a0 0x55c1ac474bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.671+0000 7f3242ffd640 10 client.26419 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.671+0000 7f3262ffd640 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac4727a0 0x55c1ac474bc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51edcd005ecfab0 server_cookie=d4523e22f5954965 in_seq=0 out_seq=0 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f3242ffd640 1 -- 172.21.15.83:0/178179402 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55c1ac35e850 con 0x55c1ac4727a0 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f3242ffd640 10 client.26419 handle_command_reply: tid=0 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 client.26419 shutdown 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 msgr2=0x55c1ac3f97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f325403b940 tx=0x7f3254004610 comp rx=0 tx=0).stop 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 shutdown_connections 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c1ac4727a0 0x55c1ac474bc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1ac043070 0x55c1ac470a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c1ac022390 0x55c1ac3f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 --2- 172.21.15.83:0/178179402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1abfd6da0 0x55c1ac3f97c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 >> 172.21.15.83:0/178179402 conn(0x55c1ac037460 msgr2=0x55c1ac3eb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.672+0000 7f3242ffd640 0 client.26419 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 shutdown_connections 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f327050c740 1 -- 172.21.15.83:0/178179402 wait complete. 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f327050c740 20 client.26419 trim_cache size 0 max 16384 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:25:18.677 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e1d42f2c-77b3-4186-9c20-c7446db13fce", 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:25:18.681 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:25:18.684 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 msgr2=0x7f324c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 0x7f324c0b8c10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f325c003b60 tx=0x7f325c0096a0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 msgr2=0x7f3264112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.673+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 shutdown_connections 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f324c0b6700 0x7f324c0b8c10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3264110440 0x7f32641131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f326410faf0 0x7f3264112cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 --2- 172.21.15.83:0/283778471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326410a1b0 0x7f32641169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 >> 172.21.15.83:0/283778471 conn(0x7f32640780e0 msgr2=0x7f3264078a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 shutdown_connections 2024-09-21T03:25:18.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:18.674+0000 7f3269f98640 1 -- 172.21.15.83:0/283778471 wait complete. 2024-09-21T03:25:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:18 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:18 smithi193 ceph-mon[35224]: pgmap v590: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 665 op/s 2024-09-21T03:25:19.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:18 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:19.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:18 smithi083 ceph-mon[31219]: pgmap v590: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 665 op/s 2024-09-21T03:25:19.360 INFO:tasks.cephfs.filesystem:scrub status for tag:e1d42f2c-77b3-4186-9c20-c7446db13fce - {'path': '/', 'tag': 'e1d42f2c-77b3-4186-9c20-c7446db13fce', 'options': 'recursive,force'} 2024-09-21T03:25:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:18 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:25:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:18 smithi142 ceph-mon[34512]: pgmap v590: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 665 op/s 2024-09-21T03:25:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 480.0 s, 2117.2 tps, lat 47.377 ms stddev 59.584 2024-09-21T03:25:21.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:20 smithi083 ceph-mon[31219]: pgmap v591: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:25:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:20 smithi193 ceph-mon[35224]: pgmap v591: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:25:21.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:20 smithi142 ceph-mon[34512]: pgmap v591: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 676 op/s 2024-09-21T03:25:23.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:22 smithi193 ceph-mon[35224]: pgmap v592: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:23.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:22 smithi083 ceph-mon[31219]: pgmap v592: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:22 smithi142 ceph-mon[34512]: pgmap v592: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:25:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:25:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:25:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:25:24.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:25:24.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:25:24.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:25:24.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:25:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:25:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:25:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:25:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:25:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:24 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:25:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:24 smithi193 ceph-mon[35224]: pgmap v593: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:24 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:25:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:24 smithi083 ceph-mon[31219]: pgmap v593: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:25.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:24 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:25:25.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:24 smithi142 ceph-mon[34512]: pgmap v593: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:25:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 485.0 s, 2410.8 tps, lat 41.709 ms stddev 49.750 2024-09-21T03:25:27.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:26 smithi193 ceph-mon[35224]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.6 MiB/s wr, 571 op/s 2024-09-21T03:25:27.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:26 smithi083 ceph-mon[31219]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.6 MiB/s wr, 571 op/s 2024-09-21T03:25:27.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:26 smithi142 ceph-mon[34512]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.6 MiB/s wr, 571 op/s 2024-09-21T03:25:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:28 smithi193 ceph-mon[35224]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-09-21T03:25:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:28 smithi083 ceph-mon[31219]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-09-21T03:25:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:28 smithi142 ceph-mon[34512]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-09-21T03:25:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 490.0 s, 2091.6 tps, lat 47.348 ms stddev 52.896 2024-09-21T03:25:31.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:30 smithi193 ceph-mon[35224]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-09-21T03:25:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:30 smithi083 ceph-mon[31219]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-09-21T03:25:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:30 smithi142 ceph-mon[34512]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-09-21T03:25:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:32 smithi193 ceph-mon[35224]: pgmap v597: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 631 op/s 2024-09-21T03:25:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:32 smithi083 ceph-mon[31219]: pgmap v597: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 631 op/s 2024-09-21T03:25:33.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:32 smithi142 ceph-mon[34512]: pgmap v597: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 631 op/s 2024-09-21T03:25:35.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:34 smithi193 ceph-mon[35224]: pgmap v598: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:25:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:34 smithi083 ceph-mon[31219]: pgmap v598: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:25:35.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:34 smithi142 ceph-mon[34512]: pgmap v598: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 727 op/s 2024-09-21T03:25:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 495.0 s, 2143.2 tps, lat 46.845 ms stddev 50.725 2024-09-21T03:25:37.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:36 smithi193 ceph-mon[35224]: pgmap v599: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-21T03:25:37.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:36 smithi083 ceph-mon[31219]: pgmap v599: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-21T03:25:37.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:36 smithi142 ceph-mon[34512]: pgmap v599: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-21T03:25:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:38 smithi193 ceph-mon[35224]: pgmap v600: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-09-21T03:25:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:38 smithi083 ceph-mon[31219]: pgmap v600: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-09-21T03:25:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:38 smithi142 ceph-mon[34512]: pgmap v600: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-09-21T03:25:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 500.0 s, 2113.6 tps, lat 47.402 ms stddev 55.050 2024-09-21T03:25:41.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:40 smithi193 ceph-mon[35224]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T03:25:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:40 smithi083 ceph-mon[31219]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T03:25:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:40 smithi142 ceph-mon[34512]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 778 op/s 2024-09-21T03:25:43.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:42 smithi193 ceph-mon[35224]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 664 op/s 2024-09-21T03:25:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:42 smithi083 ceph-mon[31219]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 664 op/s 2024-09-21T03:25:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:42 smithi142 ceph-mon[34512]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 664 op/s 2024-09-21T03:25:45.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:44 smithi193 ceph-mon[35224]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 726 op/s 2024-09-21T03:25:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:44 smithi083 ceph-mon[31219]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 726 op/s 2024-09-21T03:25:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:44 smithi142 ceph-mon[34512]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 726 op/s 2024-09-21T03:25:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 505.0 s, 2226.4 tps, lat 45.111 ms stddev 52.223 2024-09-21T03:25:47.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:47 smithi193 ceph-mon[35224]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T03:25:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:47 smithi083 ceph-mon[31219]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T03:25:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:47 smithi142 ceph-mon[34512]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T03:25:49.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:49 smithi193 ceph-mon[35224]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 701 op/s 2024-09-21T03:25:49.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:25:49.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:49 smithi083 ceph-mon[31219]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 701 op/s 2024-09-21T03:25:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:49 smithi142 ceph-mon[34512]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 701 op/s 2024-09-21T03:25:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 -- 172.21.15.83:0/3202893190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f711810ded0 msgr2=0x7f7118110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 --2- 172.21.15.83:0/3202893190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f711810ded0 0x7f7118110330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ca0 tx=0x7f711002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 -- 172.21.15.83:0/3202893190 shutdown_connections 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 --2- 172.21.15.83:0/3202893190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f711810ded0 0x7f7118110330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 --2- 172.21.15.83:0/3202893190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f711810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 --2- 172.21.15.83:0/3202893190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7118104920 0x7f7118104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 -- 172.21.15.83:0/3202893190 >> 172.21.15.83:0/3202893190 conn(0x7f7118079760 msgr2=0x7f7118079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 -- 172.21.15.83:0/3202893190 shutdown_connections 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.561+0000 7f712051a640 1 -- 172.21.15.83:0/3202893190 wait complete. 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 Processor -- start 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 -- start start 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 0x7f7118102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810ded0 0x7f71181032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7118113af0 con 0x7f7118105270 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7118113970 con 0x7f7118104920 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f712051a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7118113c70 con 0x7f711810ded0 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f711dab5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f711dab5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56762/0 (socket says 172.21.15.83:56762) 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f711dab5640 1 -- 172.21.15.83:0/3214762676 learned_addr learned my addr 172.21.15.83:0/3214762676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.562+0000 7f711e2b6640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 0x7f7118102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.563+0000 7f711eab7640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810ded0 0x7f71181032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.563+0000 7f711dab5640 1 -- 172.21.15.83:0/3214762676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810ded0 msgr2=0x7f71181032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.563+0000 7f711dab5640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810ded0 0x7f71181032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.563+0000 7f711dab5640 1 -- 172.21.15.83:0/3214762676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 msgr2=0x7f7118102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.563+0000 7f711dab5640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 0x7f7118102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.564+0000 7f711dab5640 1 -- 172.21.15.83:0/3214762676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a87a0 con 0x7f7118105270 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.564+0000 7f711e2b6640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 0x7f7118102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.564+0000 7f711dab5640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9c0 tx=0x7f710800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.565+0000 7f710f7fe640 1 -- 172.21.15.83:0/3214762676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710800cd80 con 0x7f7118105270 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.565+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a8a90 con 0x7f7118105270 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.565+0000 7f710f7fe640 1 -- 172.21.15.83:0/3214762676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71080054d0 con 0x7f7118105270 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.565+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a9020 con 0x7f7118105270 2024-09-21T03:25:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.566+0000 7f710f7fe640 1 -- 172.21.15.83:0/3214762676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108010430 con 0x7f7118105270 2024-09-21T03:25:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.568+0000 7f7126a8e740 1 Processor -- start 2024-09-21T03:25:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.568+0000 7f7126a8e740 1 -- start start 2024-09-21T03:25:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f710f7fe640 1 -- 172.21.15.83:0/3214762676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f71080105d0 con 0x7f7118105270 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a09cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f711eab7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f711eab7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35256/0 (socket says 172.21.15.83:35256) 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.569+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a097b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56776/0 (socket says 172.21.15.83:56776) 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f710f7fe640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 0x7f71040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 -- 172.21.15.83:0/990228897 learned_addr learned my addr 172.21.15.83:0/990228897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56184697fe50 con 0x5618466391b0 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56184696f280 con 0x561846641390 2024-09-21T03:25:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561846973090 con 0x561846662070 2024-09-21T03:25:50.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f710f7fe640 1 -- 172.21.15.83:0/3214762676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f71080e78b0 con 0x7f7118105270 2024-09-21T03:25:50.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711dab5640 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a09cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 0x7f71040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 -- 172.21.15.83:0/990228897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 msgr2=0x561846a09cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a09cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 -- 172.21.15.83:0/990228897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 msgr2=0x561846a097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.570+0000 7f711eab7640 1 -- 172.21.15.83:0/990228897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618466cd650 con 0x561846662070 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f711dab5640 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a09cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:50.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f711eab7640 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ca0 tx=0x7f7110031e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7280ab0d70bb1819 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f711eab7640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 0x7f71040b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7110035000 tx=0x7f711004a000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f710cff9640 1 -- 172.21.15.83:0/990228897 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110003b70 con 0x561846662070 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f710cff9640 1 -- 172.21.15.83:0/990228897 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110036070 con 0x561846662070 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.571+0000 7f710cff9640 1 -- 172.21.15.83:0/990228897 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100309e0 con 0x561846662070 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 msgr2=0x561846a09270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ca0 tx=0x7f7110031e10 comp rx=0 tx=0).stop 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 shutdown_connections 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a09cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 --2- 172.21.15.83:0/990228897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a09270 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.572+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 >> 172.21.15.83:0/990228897 conn(0x561846656460 msgr2=0x561846a07160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.573+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 shutdown_connections 2024-09-21T03:25:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.573+0000 7f7126a8e740 1 -- 172.21.15.83:0/990228897 wait complete. 2024-09-21T03:25:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.573+0000 7f7126a8e740 1 -- start start 2024-09-21T03:25:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.573+0000 7f7126a8e740 1 Processor -- start 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a14df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618467d0700 con 0x5618466391b0 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56184690a110 con 0x561846641390 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.574+0000 7f7126a8e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561846a17810 con 0x561846662070 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.575+0000 7f711eab7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.575+0000 7f711eab7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35266/0 (socket says 172.21.15.83:35266) 2024-09-21T03:25:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.575+0000 7f711eab7640 1 -- 172.21.15.83:0/2166622141 learned_addr learned my addr 172.21.15.83:0/2166622141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.575+0000 7f7126a8e740 10 client.0 _ll_register_callbacks cb 0x561846923990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.575+0000 7f711e2b6640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711dab5640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a14df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711e2b6640 1 -- 172.21.15.83:0/2166622141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 msgr2=0x561846a15330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711e2b6640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711e2b6640 1 -- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 msgr2=0x561846a14df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711e2b6640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a14df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711e2b6640 1 -- 172.21.15.83:0/2166622141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561846937c40 con 0x5618466391b0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711eab7640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.576+0000 7f711dab5640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.577+0000 7f711e2b6640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a148b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7118104280 tx=0x7f711400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb49caed666b003 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.577+0000 7f7101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:25:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.577+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140120d0 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.577+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110036070 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.577+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f7114004030 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7114012270 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7126a8e740 10 client.16914 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561846a235b0 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110036070 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7114026480 con 0x5618466391b0 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7126a8e740 10 client.16914 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7126a8e740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.578+0000 7f7126a8e740 20 client.16914 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618466209f0 0x561846a269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f7126a8e740 4 client.16914 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561846981da0 con 0x5618466209f0 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f711dab5640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618466209f0 0x561846a269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f7101ffb640 10 client.16914 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.579+0000 7f711dab5640 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618466209f0 0x561846a269f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d10acb64f35ac7f server_cookie=4a7feacffe23a324 in_seq=0 out_seq=0 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.580+0000 7f7101ffb640 1 -- 172.21.15.83:0/2166622141 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561846981da0 con 0x5618466209f0 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.580+0000 7f7101ffb640 10 client.16914 handle_command_reply: tid=0 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.580+0000 7f7126a8e740 1 client.16914 shutdown 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 msgr2=0x561846a148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a148b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7118104280 tx=0x7f711400f790 comp rx=0 tx=0).stop 2024-09-21T03:25:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 shutdown_connections 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618466209f0 0x561846a269f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a15330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2166622141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a148b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 >> 172.21.15.83:0/2166622141 conn(0x561846656460 msgr2=0x561846a0c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7101ffb640 0 client.16914 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 shutdown_connections 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 1 -- 172.21.15.83:0/2166622141 wait complete. 2024-09-21T03:25:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.581+0000 7f7126a8e740 20 client.16914 trim_cache size 0 max 16384 2024-09-21T03:25:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.606+0000 7f7126a8e740 1 Processor -- start 2024-09-21T03:25:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.606+0000 7f7126a8e740 1 -- start start 2024-09-21T03:25:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc00e6a0 con 0x5618466391b0 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc0101e0 con 0x561846641390 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7126a8e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc00ed70 con 0x561846662070 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a20a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56800/0 (socket says 172.21.15.83:56800) 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711e2b6640 1 -- 172.21.15.83:0/2400603373 learned_addr learned my addr 172.21.15.83:0/2400603373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711eab7640 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 -- 172.21.15.83:0/2400603373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 msgr2=0x561846a11aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a11aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 -- 172.21.15.83:0/2400603373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 msgr2=0x561846a20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 -- 172.21.15.83:0/2400603373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561846a13810 con 0x561846641390 2024-09-21T03:25:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f711dab5640 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a164e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71080e5050 tx=0x7f71080a96d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5c88fa62d97e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.607+0000 7f7103fff640 1 -- 172.21.15.83:0/2400603373 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71080ade50 con 0x561846641390 2024-09-21T03:25:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.608+0000 7f7103fff640 1 -- 172.21.15.83:0/2400603373 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71080a7e50 con 0x561846641390 2024-09-21T03:25:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.608+0000 7f7103fff640 1 -- 172.21.15.83:0/2400603373 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71080b6f40 con 0x561846641390 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.609+0000 7f7126a8e740 1 -- 172.21.15.83:0/2400603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 msgr2=0x561846a164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.609+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a164e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71080e5050 tx=0x7f71080a96d0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 -- 172.21.15.83:0/2400603373 shutdown_connections 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a11aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561846641390 0x561846a164e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 --2- 172.21.15.83:0/2400603373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618466391b0 0x561846a20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 -- 172.21.15.83:0/2400603373 >> 172.21.15.83:0/2400603373 conn(0x561846656460 msgr2=0x561846a15dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 -- 172.21.15.83:0/2400603373 shutdown_connections 2024-09-21T03:25:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.611+0000 7f7126a8e740 1 -- 172.21.15.83:0/2400603373 wait complete. 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.612+0000 7f7126a8e740 1 -- start start 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.612+0000 7f7126a8e740 1 Processor -- start 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.612+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 0x561846a1d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.612+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 0x561846a18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a18c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 0x561846a1d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711e2b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 0x561846a1d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44014/0 (socket says 172.21.15.83:44014) 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 0x561846a18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711e2b6640 1 -- 172.21.15.83:0/3879407709 learned_addr learned my addr 172.21.15.83:0/3879407709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc00fb10 con 0x561846641390 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc00f440 con 0x5618466391b0 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc00c6a0 con 0x561846662070 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711eab7640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a18c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 -- 172.21.15.83:0/3879407709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 msgr2=0x561846a18c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 10 client.0 _ll_register_callbacks cb 0x7f7114009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a18c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 -- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 msgr2=0x561846a1d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 0x561846a1d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561846a193c0 con 0x561846641390 2024-09-21T03:25:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71080a7e50 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f711dab5640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 0x561846a18740 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71080ad450 tx=0x7f71080e5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66627573fa9200cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7101ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7101ffb640 1 -- 172.21.15.83:0/3879407709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71080d3070 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7101ffb640 1 -- 172.21.15.83:0/3879407709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f71080ac9b0 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.613+0000 7f7101ffb640 1 -- 172.21.15.83:0/3879407709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f71080acc10 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 10 client.16920 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=44}) -- 0x561846a911f0 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7101ffb640 1 -- 172.21.15.83:0/3879407709 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f71080046c0 con 0x561846641390 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 10 client.16920 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 20 client.16920 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618465f5da0 0x561846a93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 4 client.16920 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56184697d850 con 0x5618465f5da0 2024-09-21T03:25:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.614+0000 7f711e2b6640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618465f5da0 0x561846a93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7101ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f711e2b6640 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618465f5da0 0x561846a93940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2512cf52d787ceac server_cookie=6c9bfb98b6d77b3b in_seq=0 out_seq=0 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7101ffb640 1 -- 172.21.15.83:0/3879407709 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x56184697d850 con 0x5618465f5da0 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7101ffb640 10 client.16920 handle_command_reply: tid=0 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 client.16920 shutdown 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 msgr2=0x561846a18740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 0x561846a18740 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71080ad450 tx=0x7f71080e5590 comp rx=0 tx=0).stop 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 shutdown_connections 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5618465f5da0 0x561846a93940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561846662070 0x561846a18c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561846641390 0x561846a18740 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 --2- 172.21.15.83:0/3879407709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618466391b0 0x561846a1d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7101ffb640 0 client.16920 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:25:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 >> 172.21.15.83:0/3879407709 conn(0x561846656460 msgr2=0x561846a15c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 shutdown_connections 2024-09-21T03:25:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.615+0000 7f7126a8e740 1 -- 172.21.15.83:0/3879407709 wait complete. 2024-09-21T03:25:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.618+0000 7f7126a8e740 20 client.16920 trim_cache size 0 max 16384 2024-09-21T03:25:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:25:50.623 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:25:50.624 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.625 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.626 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.627 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:25:50.628 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:25:50.629 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:25:50.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:25:50.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:25:50.629 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 msgr2=0x7f71040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 0x7f71040b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7110035000 tx=0x7f711004a000 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 msgr2=0x7f7118102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9c0 tx=0x7f710800ee80 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 shutdown_connections 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f71040b6680 0x7f71040b8b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810ded0 0x7f71181032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7118105270 0x7f7118102d70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 --2- 172.21.15.83:0/3214762676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118104920 0x7f7118102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 >> 172.21.15.83:0/3214762676 conn(0x7f7118079760 msgr2=0x7f7118107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 shutdown_connections 2024-09-21T03:25:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:50.619+0000 7f712051a640 1 -- 172.21.15.83:0/3214762676 wait complete. 2024-09-21T03:25:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 510.0 s, 2348.2 tps, lat 42.017 ms stddev 47.124 2024-09-21T03:25:51.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:51 smithi193 ceph-mon[35224]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T03:25:51.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:51 smithi083 ceph-mon[31219]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T03:25:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:51 smithi142 ceph-mon[34512]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.1 MiB/s wr, 713 op/s 2024-09-21T03:25:51.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:25:51.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:25:51.873 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 -- 172.21.15.83:0/1061942245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 msgr2=0x7f3494104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 --2- 172.21.15.83:0/1061942245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f3494104dd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3488009900 tx=0x7f348802a410 comp rx=0 tx=0).stop 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 -- 172.21.15.83:0/1061942245 shutdown_connections 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 --2- 172.21.15.83:0/1061942245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 0x7f3494110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 --2- 172.21.15.83:0/1061942245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 --2- 172.21.15.83:0/1061942245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f3494104dd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 -- 172.21.15.83:0/1061942245 >> 172.21.15.83:0/1061942245 conn(0x7f3494079730 msgr2=0x7f3494079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:52.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 -- 172.21.15.83:0/1061942245 shutdown_connections 2024-09-21T03:25:52.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.473+0000 7f349af9a640 1 -- 172.21.15.83:0/1061942245 wait complete. 2024-09-21T03:25:52.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.474+0000 7f349af9a640 1 Processor -- start 2024-09-21T03:25:52.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.474+0000 7f349af9a640 1 -- start start 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.474+0000 7f349af9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.474+0000 7f349af9a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.474+0000 7f349af9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 0x7f349419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f349af9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113aa0 con 0x7f34941049c0 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f349af9a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113920 con 0x7f3494105310 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f349af9a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113c20 con 0x7f349410de20 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56826/0 (socket says 172.21.15.83:56826) 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 -- 172.21.15.83:0/783878425 learned_addr learned my addr 172.21.15.83:0/783878425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:52.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3493fff640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:52.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3499537640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 0x7f349419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:52.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 -- 172.21.15.83:0/783878425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 msgr2=0x7f349419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 0x7f349419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 -- 172.21.15.83:0/783878425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 msgr2=0x7f349419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 -- 172.21.15.83:0/783878425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34941a0400 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3493fff640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3498d36640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3488032fd0 tx=0x7f3488031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348803f030 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3488004490 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34941a0690 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.475+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488035070 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.476+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34941a6e50 con 0x7f34941049c0 2024-09-21T03:25:52.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.477+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349406c1e0 con 0x7f34941049c0 2024-09-21T03:25:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.479+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3488042030 con 0x7f34941049c0 2024-09-21T03:25:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.480+0000 7f3491ffb640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 0x7f34840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.480+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3488107f60 con 0x7f34941049c0 2024-09-21T03:25:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.480+0000 7f3493fff640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 0x7f34840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.481+0000 7f3493fff640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 0x7f34840b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3494102380 tx=0x7f3480005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.483+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3488139710 con 0x7f34941049c0 2024-09-21T03:25:52.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.806+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3494108f50 con 0x7f34941049c0 2024-09-21T03:25:52.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.808+0000 7f3491ffb640 1 -- 172.21.15.83:0/783878425 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11982 (secure 0 0 0) 0x7f34880ce4a0 con 0x7f34941049c0 2024-09-21T03:25:52.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:25:52.811 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"btime":"2024-09-21T03:23:29:976813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:23:29.048942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:25:52.811 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-09-21T03:25:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 msgr2=0x7f34840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 0x7f34840b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3494102380 tx=0x7f3480005f90 comp rx=0 tx=0).stop 2024-09-21T03:25:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 msgr2=0x7f349419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3488032fd0 tx=0x7f3488031a20 comp rx=0 tx=0).stop 2024-09-21T03:25:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 shutdown_connections 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f34840b6700 0x7f34840b8c10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f349410de20 0x7f349419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494105310 0x7f349419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 --2- 172.21.15.83:0/783878425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34941049c0 0x7f349419a490 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 >> 172.21.15.83:0/783878425 conn(0x7f3494079730 msgr2=0x7f34941007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 shutdown_connections 2024-09-21T03:25:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:52.814+0000 7f349af9a640 1 -- 172.21.15.83:0/783878425 wait complete. 2024-09-21T03:25:53.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:53 smithi083 ceph-mon[31219]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 609 op/s 2024-09-21T03:25:53.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:53 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/783878425' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:53.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:53 smithi193 ceph-mon[35224]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 609 op/s 2024-09-21T03:25:53.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:53 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/783878425' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:53 smithi142 ceph-mon[34512]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 609 op/s 2024-09-21T03:25:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:53 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/783878425' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:25:53.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:25:53.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:54.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 -- 172.21.15.83:0/183326247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a24102a60 msgr2=0x7f0a24102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/183326247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a24102a60 0x7f0a24102e70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009900 tx=0x7f0a1402a410 comp rx=0 tx=0).stop 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 -- 172.21.15.83:0/183326247 shutdown_connections 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/183326247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24108cf0 0x7f0a2410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/183326247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a241083a0 0x7f0a241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/183326247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a24102a60 0x7f0a24102e70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 -- 172.21.15.83:0/183326247 >> 172.21.15.83:0/183326247 conn(0x7f0a24079730 msgr2=0x7f0a24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 -- 172.21.15.83:0/183326247 shutdown_connections 2024-09-21T03:25:54.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.540+0000 7f0a28ee4640 1 -- 172.21.15.83:0/183326247 wait complete. 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 Processor -- start 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 -- start start 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 0x7f0a24104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 0x7f0a241049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 0x7f0a24104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112960 con 0x7f0a241083a0 2024-09-21T03:25:54.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241127e0 con 0x7f0a24108cf0 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.541+0000 7f0a28ee4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112ae0 con 0x7f0a24102a60 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 0x7f0a24104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 0x7f0a24104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35318/0 (socket says 172.21.15.83:35318) 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a2259c640 1 -- 172.21.15.83:0/3651988477 learned_addr learned my addr 172.21.15.83:0/3651988477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 0x7f0a241049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 0x7f0a24104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 -- 172.21.15.83:0/3651988477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 msgr2=0x7f0a24104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 0x7f0a24104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 -- 172.21.15.83:0/3651988477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 msgr2=0x7f0a24104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 0x7f0a24104f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a1bfff640 1 -- 172.21.15.83:0/3651988477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241a86e0 con 0x7f0a241083a0 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.542+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 0x7f0a24104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 0x7f0a241049c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005fd0 tx=0x7f0a0c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a1b7fe640 1 -- 172.21.15.83:0/3651988477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c009c00 con 0x7f0a241083a0 2024-09-21T03:25:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241a89d0 con 0x7f0a241083a0 2024-09-21T03:25:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a1b7fe640 1 -- 172.21.15.83:0/3651988477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a0c009da0 con 0x7f0a241083a0 2024-09-21T03:25:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a8f60 con 0x7f0a241083a0 2024-09-21T03:25:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.543+0000 7f0a1b7fe640 1 -- 172.21.15.83:0/3651988477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003e10 con 0x7f0a241083a0 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.545+0000 7f0a2f458740 1 Processor -- start 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 -- start start 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce37826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 0x55dce3782be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce3783120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dce36f8ea0 con 0x55dce33b21b0 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dce36e82d0 con 0x55dce33ba390 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2f458740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dce36ec0e0 con 0x55dce33db070 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 0x55dce3782be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce37826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 0x55dce3782be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56842/0 (socket says 172.21.15.83:56842) 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a2259c640 1 -- 172.21.15.83:0/3515588140 learned_addr learned my addr 172.21.15.83:0/3515588140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 -- 172.21.15.83:0/3515588140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 msgr2=0x55dce3783120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce3783120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 -- 172.21.15.83:0/3515588140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 msgr2=0x55dce3782be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 0x55dce3782be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.546+0000 7f0a22d9d640 1 -- 172.21.15.83:0/3515588140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce3446650 con 0x55dce33db070 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.547+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce37826a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c005fd0 tx=0x7f0a1c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c947d2e6efb28aba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.547+0000 7f0a18ff9640 1 -- 172.21.15.83:0/3515588140 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c004d20 con 0x55dce33db070 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.547+0000 7f0a18ff9640 1 -- 172.21.15.83:0/3515588140 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c00f9a0 con 0x55dce33db070 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.547+0000 7f0a18ff9640 1 -- 172.21.15.83:0/3515588140 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c004470 con 0x55dce33db070 2024-09-21T03:25:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.547+0000 7f0a1b7fe640 1 -- 172.21.15.83:0/3651988477 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a0c004030 con 0x7f0a241083a0 2024-09-21T03:25:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a1b7fe640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 0x7f0a100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 -- 172.21.15.83:0/3515588140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 msgr2=0x55dce37826a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2259c640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 0x7f0a100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce37826a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c005fd0 tx=0x7f0a1c011bf0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a1b7fe640 1 -- 172.21.15.83:0/3651988477 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0a0c0e6b30 con 0x7f0a241083a0 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 -- 172.21.15.83:0/3515588140 shutdown_connections 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce3783120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33b21b0 0x55dce3782be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 --2- 172.21.15.83:0/3515588140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce37826a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 -- 172.21.15.83:0/3515588140 >> 172.21.15.83:0/3515588140 conn(0x55dce33cf460 msgr2=0x55dce3780590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 -- 172.21.15.83:0/3515588140 shutdown_connections 2024-09-21T03:25:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.548+0000 7f0a2f458740 1 -- 172.21.15.83:0/3515588140 wait complete. 2024-09-21T03:25:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.549+0000 7f0a2f458740 1 -- start start 2024-09-21T03:25:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.549+0000 7f0a2f458740 1 Processor -- start 2024-09-21T03:25:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.549+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.549+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.549+0000 7f0a2259c640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 0x7f0a100b8c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005f70 tx=0x7f0a140058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2f458740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dce3549700 con 0x55dce33ba390 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2f458740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dce3683160 con 0x55dce33b21b0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2f458740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dce3790e80 con 0x55dce33db070 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2f458740 10 client.0 _ll_register_callbacks cb 0x55dce369c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56856/0 (socket says 172.21.15.83:56856) 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a1bfff640 1 -- 172.21.15.83:0/4167192726 learned_addr learned my addr 172.21.15.83:0/4167192726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a22d9d640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.550+0000 7f0a2259c640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 -- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 msgr2=0x55dce378e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 -- 172.21.15.83:0/4167192726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 msgr2=0x55dce378dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 -- 172.21.15.83:0/4167192726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce36b0c90 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a8ff0 tx=0x7f0a0c0ac7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9ae32d67095604 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c020040 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.551+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0a0c0a8370 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.552+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a1c00f9a0 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.552+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0ac190 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.552+0000 7f0a2f458740 10 client.16938 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.552+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dce3793ab0 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.552+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a0c0ac530 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f0a0c0d1050 con 0x55dce33ba390 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 10 client.16938 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 10 client.16938 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 20 client.16938 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce33999f0 0x55dce3796fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 4 client.16938 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a1c00f9a0 con 0x55dce33999f0 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a2259c640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce33999f0 0x55dce3796fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.553+0000 7f0a09ffb640 10 client.16938 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2259c640 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce33999f0 0x55dce3796fb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6f748218d675ac3 server_cookie=20e21ceeb62b0d78 in_seq=0 out_seq=0 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a09ffb640 1 -- 172.21.15.83:0/4167192726 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0a1c00f9a0 con 0x55dce33999f0 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a09ffb640 10 client.16938 handle_command_reply: tid=0 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2f458740 1 client.16938 shutdown 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 msgr2=0x55dce378e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a8ff0 tx=0x7f0a0c0ac7f0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 shutdown_connections 2024-09-21T03:25:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.554+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce33999f0 0x55dce3796fb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce378e280 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 --2- 172.21.15.83:0/4167192726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 >> 172.21.15.83:0/4167192726 conn(0x55dce33cf460 msgr2=0x55dce3785830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a09ffb640 0 client.16938 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 shutdown_connections 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 1 -- 172.21.15.83:0/4167192726 wait complete. 2024-09-21T03:25:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.555+0000 7f0a2f458740 20 client.16938 trim_cache size 0 max 16384 2024-09-21T03:25:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.571+0000 7f0a2f458740 1 Processor -- start 2024-09-21T03:25:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.571+0000 7f0a2f458740 1 -- start start 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0400e770 con 0x55dce33ba390 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a040102b0 con 0x55dce33b21b0 2024-09-21T03:25:54.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.572+0000 7f0a2f458740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0400ee40 con 0x55dce33db070 2024-09-21T03:25:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56862/0 (socket says 172.21.15.83:56862) 2024-09-21T03:25:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a1bfff640 1 -- 172.21.15.83:0/2290012740 learned_addr learned my addr 172.21.15.83:0/2290012740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 -- 172.21.15.83:0/2290012740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 msgr2=0x55dce378e200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 -- 172.21.15.83:0/2290012740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 msgr2=0x55dce37836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 -- 172.21.15.83:0/2290012740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce378f000 con 0x55dce33b21b0 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.574+0000 7f0a2259c640 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378fc20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a14031440 tx=0x7f0a1403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1718e6876b2a5303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.575+0000 7f0a0bfff640 1 -- 172.21.15.83:0/2290012740 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14035040 con 0x55dce33b21b0 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.575+0000 7f0a0bfff640 1 -- 172.21.15.83:0/2290012740 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a14055030 con 0x55dce33b21b0 2024-09-21T03:25:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.575+0000 7f0a0bfff640 1 -- 172.21.15.83:0/2290012740 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1403c7c0 con 0x55dce33b21b0 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- 172.21.15.83:0/2290012740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 msgr2=0x55dce378fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378fc20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a14031440 tx=0x7f0a1403e000 comp rx=0 tx=0).stop 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- 172.21.15.83:0/2290012740 shutdown_connections 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33db070 0x55dce378e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33ba390 0x55dce37836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2290012740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33b21b0 0x55dce378fc20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- 172.21.15.83:0/2290012740 >> 172.21.15.83:0/2290012740 conn(0x55dce33cf460 msgr2=0x55dce378daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- 172.21.15.83:0/2290012740 shutdown_connections 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- 172.21.15.83:0/2290012740 wait complete. 2024-09-21T03:25:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.576+0000 7f0a2f458740 1 -- start start 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 Processor -- start 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33b21b0 0x55dce379a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33db070 0x55dce3801d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0400fbe0 con 0x55dce33db070 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0400f510 con 0x55dce33ba390 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0400c770 con 0x55dce33b21b0 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2f458740 10 client.0 _ll_register_callbacks cb 0x7f0a0c00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33db070 0x55dce3801d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33b21b0 0x55dce379a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44058/0 (socket says 172.21.15.83:44058) 2024-09-21T03:25:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 -- 172.21.15.83:0/2466912616 learned_addr learned my addr 172.21.15.83:0/2466912616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 -- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33b21b0 msgr2=0x55dce379a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33b21b0 0x55dce379a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 -- 172.21.15.83:0/2466912616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33db070 msgr2=0x55dce3801d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33db070 0x55dce3801d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.577+0000 7f0a1bfff640 1 -- 172.21.15.83:0/2466912616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce3802370 con 0x55dce33ba390 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a1bfff640 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0ac9b0 tx=0x7f0a0c0abab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72c05eed9c681b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c020070 con 0x55dce33ba390 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a14055030 con 0x55dce33ba390 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0a0c0b26d0 con 0x55dce33ba390 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0a8190 con 0x55dce33ba390 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a2f458740 10 client.26477 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.578+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dce380a880 con 0x55dce33ba390 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a0c0ac530 con 0x55dce33ba390 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f0a0c0d1020 con 0x55dce33ba390 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 10 client.26477 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 10 client.26477 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 20 client.26477 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce336eda0 0x55dce380ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 4 client.26477 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.579+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0a14055030 con 0x55dce336eda0 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2259c640 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce336eda0 0x55dce380ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2259c640 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce336eda0 0x55dce380ceb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c029950cd35d755 server_cookie=a328b65eff1a0903 in_seq=0 out_seq=0 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a09ffb640 10 client.26477 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a09ffb640 1 -- 172.21.15.83:0/2466912616 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0a14055030 con 0x55dce336eda0 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a09ffb640 10 client.26477 handle_command_reply: tid=0 2024-09-21T03:25:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2f458740 1 client.26477 shutdown 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 msgr2=0x55dce379a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0ac9b0 tx=0x7f0a0c0abab0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.580+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 shutdown_connections 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55dce336eda0 0x55dce380ceb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dce33db070 0x55dce3801d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce33ba390 0x55dce379a8a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 --2- 172.21.15.83:0/2466912616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dce33b21b0 0x55dce379a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 >> 172.21.15.83:0/2466912616 conn(0x55dce33cf460 msgr2=0x55dce378f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a09ffb640 0 client.26477 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 shutdown_connections 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 1 -- 172.21.15.83:0/2466912616 wait complete. 2024-09-21T03:25:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a2f458740 20 client.26477 trim_cache size 0 max 16384 2024-09-21T03:25:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 msgr2=0x7f0a100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 0x7f0a100b8c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005f70 tx=0x7f0a140058d0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 msgr2=0x7f0a241049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.581+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 0x7f0a241049c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005fd0 tx=0x7f0a0c00dde0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 shutdown_connections 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0a100b6700 0x7f0a100b8c10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108cf0 0x7f0a24104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a241083a0 0x7f0a241049c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 --2- 172.21.15.83:0/3651988477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a24102a60 0x7f0a24104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 >> 172.21.15.83:0/3651988477 conn(0x7f0a24079730 msgr2=0x7f0a241067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 shutdown_connections 2024-09-21T03:25:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:54.582+0000 7f0a28ee4640 1 -- 172.21.15.83:0/3651988477 wait complete. 2024-09-21T03:25:55.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:25:55.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:55 smithi083 ceph-mon[31219]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-09-21T03:25:55.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:55 smithi193 ceph-mon[35224]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-09-21T03:25:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:55 smithi142 ceph-mon[34512]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-09-21T03:25:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 515.0 s, 1350.2 tps, lat 73.955 ms stddev 80.486 2024-09-21T03:25:57.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 -- 172.21.15.83:0/2768892933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109320 msgr2=0x7f18cc109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 --2- 172.21.15.83:0/2768892933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109320 0x7f18cc109730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002a410 comp rx=0 tx=0).stop 2024-09-21T03:25:57.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 -- 172.21.15.83:0/2768892933 shutdown_connections 2024-09-21T03:25:57.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 --2- 172.21.15.83:0/2768892933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109c70 0x7f18cc1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 --2- 172.21.15.83:0/2768892933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109320 0x7f18cc109730 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 --2- 172.21.15.83:0/2768892933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 0x7f18cc103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 -- 172.21.15.83:0/2768892933 >> 172.21.15.83:0/2768892933 conn(0x7f18cc0ff430 msgr2=0x7f18cc101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 -- 172.21.15.83:0/2768892933 shutdown_connections 2024-09-21T03:25:57.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.051+0000 7f18cbfff640 1 -- 172.21.15.83:0/2768892933 wait complete. 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.052+0000 7f18cbfff640 1 Processor -- start 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.053+0000 7f18cbfff640 1 -- start start 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.053+0000 7f18cbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 0x7f18cc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.053+0000 7f18cbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 0x7f18cc075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cbfff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc113980 con 0x7f18cc109c70 2024-09-21T03:25:57.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cbfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc113800 con 0x7f18cc1039e0 2024-09-21T03:25:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cbfff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc113b00 con 0x7f18cc109320 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56894/0 (socket says 172.21.15.83:56894) 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18cb7fe640 1 -- 172.21.15.83:0/921132441 learned_addr learned my addr 172.21.15.83:0/921132441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 0x7f18cc075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.054+0000 7f18caffd640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 0x7f18cc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 -- 172.21.15.83:0/921132441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 msgr2=0x7f18cc075770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 0x7f18cc075770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 -- 172.21.15.83:0/921132441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 msgr2=0x7f18cc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 0x7f18cc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 -- 172.21.15.83:0/921132441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc076530 con 0x7f18cc109c70 2024-09-21T03:25:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 0x7f18cc075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18cb7fe640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb10 tx=0x7f18c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.055+0000 7f18d0986640 1 -- 172.21.15.83:0/921132441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4004b20 con 0x7f18cc109c70 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.056+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc1accf0 con 0x7f18cc109c70 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.056+0000 7f18d0986640 1 -- 172.21.15.83:0/921132441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c4004cc0 con 0x7f18cc109c70 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.056+0000 7f18d0986640 1 -- 172.21.15.83:0/921132441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4010690 con 0x7f18cc109c70 2024-09-21T03:25:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.056+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1ad1b0 con 0x7f18cc109c70 2024-09-21T03:25:57.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.058+0000 7f18d7efc740 1 Processor -- start 2024-09-21T03:25:57.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d0986640 1 -- 172.21.15.83:0/921132441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f18c40108f0 con 0x7f18cc109c70 2024-09-21T03:25:57.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 -- start start 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee7cbc1ea0 con 0x55ee7c8a4070 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ee7cbb12d0 con 0x55ee7c883390 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18d7efc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee7cbb50e0 con 0x55ee7c87b1b0 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc4b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc4b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56910/0 (socket says 172.21.15.83:56910) 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18ca7fc640 1 -- 172.21.15.83:0/1547539342 learned_addr learned my addr 172.21.15.83:0/1547539342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.059+0000 7f18caffd640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18d0986640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 0x7f18b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18caffd640 1 -- 172.21.15.83:0/1547539342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 msgr2=0x55ee7cc4c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18d0986640 1 -- 172.21.15.83:0/921132441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f18c40e7c50 con 0x7f18cc109c70 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18caffd640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18caffd640 1 -- 172.21.15.83:0/1547539342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 msgr2=0x55ee7cc4b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18caffd640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc4b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18caffd640 1 -- 172.21.15.83:0/1547539342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee7c90f650 con 0x55ee7c87b1b0 2024-09-21T03:25:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.060+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.061+0000 7f18caffd640 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc4bda0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009bf0 tx=0x7f18bc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2cad0e177983c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.061+0000 7f18b1ffb640 1 -- 172.21.15.83:0/1547539342 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc009da0 con 0x55ee7c87b1b0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.061+0000 7f18b1ffb640 1 -- 172.21.15.83:0/1547539342 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18bc005020 con 0x55ee7c87b1b0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.061+0000 7f18b1ffb640 1 -- 172.21.15.83:0/1547539342 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0049e0 con 0x55ee7c87b1b0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.061+0000 7f18cb7fe640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 0x7f18b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 -- 172.21.15.83:0/1547539342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 msgr2=0x55ee7cc4bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18cb7fe640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 0x7f18b40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f18c40aa0c0 tx=0x7f18c4014000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc4bda0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009bf0 tx=0x7f18bc011810 comp rx=0 tx=0).stop 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 -- 172.21.15.83:0/1547539342 shutdown_connections 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc4bda0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1547539342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.062+0000 7f18d7efc740 1 -- 172.21.15.83:0/1547539342 >> 172.21.15.83:0/1547539342 conn(0x55ee7c898460 msgr2=0x55ee7cc49750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 -- 172.21.15.83:0/1547539342 shutdown_connections 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 -- 172.21.15.83:0/1547539342 wait complete. 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 -- start start 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 Processor -- start 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc56f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.063+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18d7efc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee7ca12700 con 0x55ee7c8a4070 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18d7efc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ee7cb4c160 con 0x55ee7c87b1b0 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18d7efc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee7cc5a040 con 0x55ee7c883390 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35418/0 (socket says 172.21.15.83:35418) 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 -- 172.21.15.83:0/1787055291 learned_addr learned my addr 172.21.15.83:0/1787055291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18d7efc740 10 client.0 _ll_register_callbacks cb 0x55ee7cb659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 -- 172.21.15.83:0/1787055291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 msgr2=0x55ee7cc56f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc56f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc56f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 -- 172.21.15.83:0/1787055291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 msgr2=0x55ee7cc63360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 -- 172.21.15.83:0/1787055291 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7cb79c90 con 0x55ee7c883390 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc56f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18caffd640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18bc000c00 tx=0x7f18bc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51ca0ec60a1fec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.064+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc029020 con 0x55ee7c883390 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18bc005020 con 0x55ee7c883390 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f18bc004970 con 0x55ee7c883390 2024-09-21T03:25:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f18bc034050 con 0x55ee7c883390 2024-09-21T03:25:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18d7efc740 10 client.26470 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ee7cc65cb0 con 0x55ee7c883390 2024-09-21T03:25:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18bc005020 con 0x55ee7c883390 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.065+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f18bc023420 con 0x55ee7c883390 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 10 client.26470 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 10 client.26470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 20 client.26470 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7c8629f0 0x55ee7cc69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 4 client.26470 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee7cbc3df0 con 0x55ee7c8629f0 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.066+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7c8629f0 0x55ee7cc69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.069+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7c8629f0 0x55ee7cc69120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d7598f23142ad78 server_cookie=19bc3e16c43013b3 in_seq=0 out_seq=0 2024-09-21T03:25:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.069+0000 7f18aaffd640 10 client.26470 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18aaffd640 1 -- 172.21.15.83:0/1787055291 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee7cbc3df0 con 0x55ee7c8629f0 2024-09-21T03:25:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18aaffd640 10 client.26470 handle_command_reply: tid=0 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 client.26470 shutdown 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 msgr2=0x55ee7cc57480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18bc000c00 tx=0x7f18bc0098c0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 shutdown_connections 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7c8629f0 0x55ee7cc69120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c883390 0x55ee7cc57480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 --2- 172.21.15.83:0/1787055291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc56f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18aaffd640 0 client.26470 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 >> 172.21.15.83:0/1787055291 conn(0x55ee7c898460 msgr2=0x55ee7cc4ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 shutdown_connections 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 1 -- 172.21.15.83:0/1787055291 wait complete. 2024-09-21T03:25:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.070+0000 7f18d7efc740 20 client.26470 trim_cache size 0 max 16384 2024-09-21T03:25:57.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.086+0000 7f18d7efc740 1 Processor -- start 2024-09-21T03:25:57.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.086+0000 7f18d7efc740 1 -- start start 2024-09-21T03:25:57.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b5b50 con 0x55ee7c8a4070 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b7690 con 0x55ee7c883390 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18d7efc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b6220 con 0x55ee7c87b1b0 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18ca7fc640 1 -- 172.21.15.83:0/3440943191 learned_addr learned my addr 172.21.15.83:0/3440943191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.087+0000 7f18caffd640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18ca7fc640 1 -- 172.21.15.83:0/3440943191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 msgr2=0x55ee7cc58e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc58e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18ca7fc640 1 -- 172.21.15.83:0/3440943191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 msgr2=0x55ee7cc4cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18ca7fc640 1 -- 172.21.15.83:0/3440943191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee7cc4db50 con 0x55ee7c8a4070 2024-09-21T03:25:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.088+0000 7f18caffd640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.089+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f18c002a930 tx=0x7f18c003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d843b923ab545ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.089+0000 7f18b17fa640 1 -- 172.21.15.83:0/3440943191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0035040 con 0x55ee7c8a4070 2024-09-21T03:25:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.089+0000 7f18b17fa640 1 -- 172.21.15.83:0/3440943191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c0003e50 con 0x55ee7c8a4070 2024-09-21T03:25:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.089+0000 7f18b17fa640 1 -- 172.21.15.83:0/3440943191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0004a40 con 0x55ee7c8a4070 2024-09-21T03:25:57.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.090+0000 7f18d7efc740 1 -- 172.21.15.83:0/3440943191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 msgr2=0x55ee7cc63260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.090+0000 7f18d7efc740 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f18c002a930 tx=0x7f18c003f040 comp rx=0 tx=0).stop 2024-09-21T03:25:57.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 -- 172.21.15.83:0/3440943191 shutdown_connections 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c8a4070 0x55ee7cc63260 secure :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f18c002a930 tx=0x7f18c003f040 comp rx=0 tx=0).stop 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c883390 0x55ee7cc4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 --2- 172.21.15.83:0/3440943191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc58e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 -- 172.21.15.83:0/3440943191 >> 172.21.15.83:0/3440943191 conn(0x55ee7c898460 msgr2=0x55ee7cc4c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 -- 172.21.15.83:0/3440943191 shutdown_connections 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 -- 172.21.15.83:0/3440943191 wait complete. 2024-09-21T03:25:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.091+0000 7f18d7efc740 1 -- start start 2024-09-21T03:25:57.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.092+0000 7f18d7efc740 1 Processor -- start 2024-09-21T03:25:57.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.092+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.092+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 0x55ee7cc5aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.092+0000 7f18d7efc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c837da0 0x55ee7cc5b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.092+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18d7efc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b6fc0 con 0x55ee7c883390 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18d7efc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b68f0 con 0x55ee7c87b1b0 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18ca7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44098/0 (socket says 172.21.15.83:44098) 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18ca7fc640 1 -- 172.21.15.83:0/335610731 learned_addr learned my addr 172.21.15.83:0/335610731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1b3b50 con 0x55ee7c837da0 2024-09-21T03:25:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18d7efc740 10 client.0 _ll_register_callbacks cb 0x7f18bc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:57.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.093+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c837da0 0x55ee7cc5b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 0x55ee7cc5aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 -- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c837da0 msgr2=0x55ee7cc5b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c837da0 0x55ee7cc5b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 -- 172.21.15.83:0/335610731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 msgr2=0x55ee7cc5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 -- 172.21.15.83:0/335610731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7cc5baf0 con 0x55ee7c883390 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18caffd640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 0x55ee7cc5aec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0059f0 tx=0x7f18bc009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5ca73a1a5ecc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.094+0000 7f18aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc029050 con 0x55ee7c883390 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f18bc023c60 con 0x55ee7c883390 2024-09-21T03:25:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c0003e50 con 0x55ee7c883390 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f18bc046050 con 0x55ee7c883390 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18d7efc740 10 client.16962 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ee7ccd3800 con 0x55ee7c883390 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c0003e50 con 0x55ee7c883390 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f18bc0035f0 con 0x55ee7c883390 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.095+0000 7f18d7efc740 10 client.16962 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18d7efc740 10 client.16962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18d7efc740 20 client.16962 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:57.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7ccd3ee0 0x55ee7ccd6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:57.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18d7efc740 4 client.16962 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:57.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee7cbbf8a0 con 0x55ee7ccd3ee0 2024-09-21T03:25:57.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7ccd3ee0 0x55ee7ccd6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:57.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18aaffd640 10 client.16962 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:57.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.096+0000 7f18ca7fc640 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7ccd3ee0 0x55ee7ccd6300 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e38b5d8817295cda server_cookie=1087085de050d7c7 in_seq=0 out_seq=0 2024-09-21T03:25:57.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18aaffd640 1 -- 172.21.15.83:0/335610731 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee7cbbf8a0 con 0x55ee7ccd3ee0 2024-09-21T03:25:57.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18aaffd640 10 client.16962 handle_command_reply: tid=0 2024-09-21T03:25:57.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 client.16962 shutdown 2024-09-21T03:25:57.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 msgr2=0x55ee7cc5aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 0x55ee7cc5aec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0059f0 tx=0x7f18bc009810 comp rx=0 tx=0).stop 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 shutdown_connections 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55ee7ccd3ee0 0x55ee7ccd6300 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee7c837da0 0x55ee7cc5b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee7c883390 0x55ee7cc5aec0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 --2- 172.21.15.83:0/335610731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee7c87b1b0 0x55ee7cc5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 >> 172.21.15.83:0/335610731 conn(0x55ee7c898460 msgr2=0x55ee7cc58b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.097+0000 7f18aaffd640 0 client.16962 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.098+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 shutdown_connections 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.098+0000 7f18d7efc740 1 -- 172.21.15.83:0/335610731 wait complete. 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.098+0000 7f18d7efc740 20 client.16962 trim_cache size 0 max 16384 2024-09-21T03:25:57.101 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:57.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 msgr2=0x7f18b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 0x7f18b40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f18c40aa0c0 tx=0x7f18c4014000 comp rx=0 tx=0).stop 2024-09-21T03:25:57.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 msgr2=0x7f18cc075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:57.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb10 tx=0x7f18c400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 shutdown_connections 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f18b40b6680 0x7f18b40b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc109c70 0x7f18cc075d10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc109320 0x7f18cc075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 --2- 172.21.15.83:0/921132441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18cc1039e0 0x7f18cc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 >> 172.21.15.83:0/921132441 conn(0x7f18cc0ff430 msgr2=0x7f18cc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 shutdown_connections 2024-09-21T03:25:57.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:57.099+0000 7f18cbfff640 1 -- 172.21.15.83:0/921132441 wait complete. 2024-09-21T03:25:57.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:57 smithi193 ceph-mon[35224]: pgmap v609: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-09-21T03:25:57.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:57 smithi083 ceph-mon[31219]: pgmap v609: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-09-21T03:25:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:57 smithi142 ceph-mon[34512]: pgmap v609: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-09-21T03:25:57.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:25:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.880+0000 7f905e929640 1 -- 172.21.15.83:0/4287710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 msgr2=0x7f905810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.880+0000 7f905e929640 1 --2- 172.21.15.83:0/4287710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 0x7f905810adf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f904401a910 tx=0x7f904403b470 comp rx=0 tx=0).stop 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.881+0000 7f905e929640 1 -- 172.21.15.83:0/4287710132 shutdown_connections 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.881+0000 7f905e929640 1 --2- 172.21.15.83:0/4287710132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f905811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.881+0000 7f905e929640 1 --2- 172.21.15.83:0/4287710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 0x7f905810adf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.881+0000 7f905e929640 1 --2- 172.21.15.83:0/4287710132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f905810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.881+0000 7f905e929640 1 -- 172.21.15.83:0/4287710132 >> 172.21.15.83:0/4287710132 conn(0x7f9058070ec0 msgr2=0x7f90580712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.882+0000 7f905e929640 1 -- 172.21.15.83:0/4287710132 shutdown_connections 2024-09-21T03:25:58.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.882+0000 7f905e929640 1 -- 172.21.15.83:0/4287710132 wait complete. 2024-09-21T03:25:58.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.883+0000 7f905e929640 1 Processor -- start 2024-09-21T03:25:58.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 -- start start 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 0x7f905811c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9058114c50 con 0x7f905810a9e0 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9058114ad0 con 0x7f9058111f40 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.886+0000 7f905e929640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9058114dd0 con 0x7f905811cc30 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f905cec6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44120/0 (socket says 172.21.15.83:44120) 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f905cec6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35472/0 (socket says 172.21.15.83:35472) 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 -- 172.21.15.83:0/981636046 learned_addr learned my addr 172.21.15.83:0/981636046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 -- 172.21.15.83:0/981636046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 msgr2=0x7f9058118d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 -- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 msgr2=0x7f905811c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 0x7f905811c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f90577fe640 1 -- 172.21.15.83:0/981636046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9058119590 con 0x7f9058111f40 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.887+0000 7f905cec6640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:25:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.888+0000 7f90577fe640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f9044004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.888+0000 7f90557fa640 1 -- 172.21.15.83:0/981636046 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904404f070 con 0x7f9058111f40 2024-09-21T03:25:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.888+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581ace90 con 0x7f9058111f40 2024-09-21T03:25:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.888+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581ad360 con 0x7f9058111f40 2024-09-21T03:25:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.889+0000 7f90557fa640 1 -- 172.21.15.83:0/981636046 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90440409c0 con 0x7f9058111f40 2024-09-21T03:25:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.889+0000 7f90557fa640 1 -- 172.21.15.83:0/981636046 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044046030 con 0x7f9058111f40 2024-09-21T03:25:58.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 Processor -- start 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 -- start start 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e1141b0 0x55fe7e4e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e45aea0 con 0x55fe7e11c390 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e44a2d0 con 0x55fe7e13d070 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f9064e9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e44e0e0 con 0x55fe7e1141b0 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.893+0000 7f905cec6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.894+0000 7f905cec6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44134/0 (socket says 172.21.15.83:44134) 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.894+0000 7f905cec6640 1 -- 172.21.15.83:0/756170747 learned_addr learned my addr 172.21.15.83:0/756170747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.894+0000 7f9057fff640 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e1141b0 0x55fe7e4e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.894+0000 7f90557fa640 1 -- 172.21.15.83:0/981636046 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9044052070 con 0x7f9058111f40 2024-09-21T03:25:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.895+0000 7f905cec6640 1 -- 172.21.15.83:0/756170747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e1141b0 msgr2=0x55fe7e4e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.895+0000 7f905cec6640 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e1141b0 0x55fe7e4e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.895+0000 7f905cec6640 1 -- 172.21.15.83:0/756170747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 msgr2=0x55fe7e4e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.895+0000 7f905cec6640 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.895+0000 7f905cec6640 1 -- 172.21.15.83:0/756170747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7e1a8650 con 0x55fe7e13d070 2024-09-21T03:25:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.896+0000 7f90557fa640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 0x7f903c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.896+0000 7f90557fa640 1 -- 172.21.15.83:0/981636046 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9044118880 con 0x7f9058111f40 2024-09-21T03:25:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.897+0000 7f9057fff640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 0x7f903c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.898+0000 7f9057fff640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 0x7f903c0b8cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55fe7e4e2f40 tx=0x7f904800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.898+0000 7f905cec6640 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f905000a870 tx=0x7f9050009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c99b1bf4a9eb03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.898+0000 7f904e7fc640 1 -- 172.21.15.83:0/756170747 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050018020 con 0x55fe7e13d070 2024-09-21T03:25:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.899+0000 7f904e7fc640 1 -- 172.21.15.83:0/756170747 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9050014400 con 0x55fe7e13d070 2024-09-21T03:25:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.899+0000 7f904e7fc640 1 -- 172.21.15.83:0/756170747 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500135c0 con 0x55fe7e13d070 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 -- 172.21.15.83:0/756170747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 msgr2=0x55fe7e4e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f905000a870 tx=0x7f9050009650 comp rx=0 tx=0).stop 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 -- 172.21.15.83:0/756170747 shutdown_connections 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e1141b0 0x55fe7e4e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 --2- 172.21.15.83:0/756170747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4e42c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 -- 172.21.15.83:0/756170747 >> 172.21.15.83:0/756170747 conn(0x55fe7e131460 msgr2=0x55fe7e4e21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 -- 172.21.15.83:0/756170747 shutdown_connections 2024-09-21T03:25:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.900+0000 7f9064e9d740 1 -- 172.21.15.83:0/756170747 wait complete. 2024-09-21T03:25:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 -- start start 2024-09-21T03:25:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 Processor -- start 2024-09-21T03:25:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 0x55fe7e4eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e2ab700 con 0x55fe7e1141b0 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.901+0000 7f9064e9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e3e5160 con 0x55fe7e11c390 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9064e9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe7e4f2af0 con 0x55fe7e13d070 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9064e9d740 10 client.0 _ll_register_callbacks cb 0x55fe7e3fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56976/0 (socket says 172.21.15.83:56976) 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 -- 172.21.15.83:0/2455362688 learned_addr learned my addr 172.21.15.83:0/2455362688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f90577fe640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 0x55fe7e4eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f905cec6640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 -- 172.21.15.83:0/2455362688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 msgr2=0x55fe7e4fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 -- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 msgr2=0x55fe7e4eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 0x55fe7e4eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f9057fff640 1 -- 172.21.15.83:0/2455362688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7e412c90 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.902+0000 7f90577fe640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 0x55fe7e4eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f9057fff640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9048004150 tx=0x7f90480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c16b1df7bac1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f905cec6640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f903b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904801e620 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f904801e7c0 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9050014400 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f904801e9c0 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f9064e9d740 10 client.16968 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.903+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fe7e4fe8e0 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9050014400 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f9048030cd0 con 0x55fe7e1141b0 2024-09-21T03:25:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 10 client.16968 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 20 client.16968 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0fb9f0 0x55fe7e501d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 4 client.16968 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe7e45cdf0 con 0x55fe7e0fb9f0 2024-09-21T03:25:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.904+0000 7f90577fe640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0fb9f0 0x55fe7e501d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.905+0000 7f903b7fe640 10 client.16968 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.905+0000 7f90577fe640 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0fb9f0 0x55fe7e501d00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3110ca08463d7596 server_cookie=c111d7b8708ed9b7 in_seq=0 out_seq=0 2024-09-21T03:25:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f903b7fe640 1 -- 172.21.15.83:0/2455362688 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe7e45cdf0 con 0x55fe7e0fb9f0 2024-09-21T03:25:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f903b7fe640 10 client.16968 handle_command_reply: tid=0 2024-09-21T03:25:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 client.16968 shutdown 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 msgr2=0x55fe7e4efa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9048004150 tx=0x7f90480046c0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 shutdown_connections 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0fb9f0 0x55fe7e501d00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e11c390 0x55fe7e4eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2455362688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4efa20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f903b7fe640 0 client.16968 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 >> 172.21.15.83:0/2455362688 conn(0x55fe7e131460 msgr2=0x55fe7e4e7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 shutdown_connections 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.906+0000 7f9064e9d740 1 -- 172.21.15.83:0/2455362688 wait complete. 2024-09-21T03:25:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.907+0000 7f9064e9d740 20 client.16968 trim_cache size 0 max 16384 2024-09-21T03:25:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 Processor -- start 2024-09-21T03:25:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 -- start start 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 0x55fe7e4ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f904000e770 con 0x55fe7e11c390 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90400102b0 con 0x55fe7e1141b0 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9064e9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f904000ee40 con 0x55fe7e13d070 2024-09-21T03:25:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f90577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f90577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56984/0 (socket says 172.21.15.83:56984) 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f90577fe640 1 -- 172.21.15.83:0/2992364571 learned_addr learned my addr 172.21.15.83:0/2992364571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f9057fff640 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 0x55fe7e4ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.927+0000 7f905cec6640 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 -- 172.21.15.83:0/2992364571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 msgr2=0x55fe7e4ecc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4ecc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 -- 172.21.15.83:0/2992364571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 msgr2=0x55fe7e4e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 -- 172.21.15.83:0/2992364571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7e4edf20 con 0x55fe7e1141b0 2024-09-21T03:25:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f9057fff640 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 0x55fe7e4ee910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9048004120 tx=0x7f9048011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bddccce7deb0c301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f904dffb640 1 -- 172.21.15.83:0/2992364571 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904803f070 con 0x55fe7e1141b0 2024-09-21T03:25:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.928+0000 7f904dffb640 1 -- 172.21.15.83:0/2992364571 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9048026640 con 0x55fe7e1141b0 2024-09-21T03:25:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.929+0000 7f904dffb640 1 -- 172.21.15.83:0/2992364571 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048036040 con 0x55fe7e1141b0 2024-09-21T03:25:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 -- 172.21.15.83:0/2992364571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 msgr2=0x55fe7e4ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 0x55fe7e4ee910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9048004120 tx=0x7f9048011dd0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 -- 172.21.15.83:0/2992364571 shutdown_connections 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e13d070 0x55fe7e4ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e11c390 0x55fe7e4e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 --2- 172.21.15.83:0/2992364571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e1141b0 0x55fe7e4ee910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 -- 172.21.15.83:0/2992364571 >> 172.21.15.83:0/2992364571 conn(0x55fe7e131460 msgr2=0x55fe7e4e73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 -- 172.21.15.83:0/2992364571 shutdown_connections 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.930+0000 7f9064e9d740 1 -- 172.21.15.83:0/2992364571 wait complete. 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.931+0000 7f9064e9d740 1 -- start start 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.931+0000 7f9064e9d740 1 Processor -- start 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9064e9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f90577fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9057fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f90577fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:35514/0 (socket says 172.21.15.83:35514) 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9057fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56988/0 (socket says 172.21.15.83:56988) 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f90577fe640 1 -- 172.21.15.83:0/514116522 learned_addr learned my addr 172.21.15.83:0/514116522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.932+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f904000fbe0 con 0x55fe7e1141b0 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f905cec6640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f904000f510 con 0x55fe7e13d070 2024-09-21T03:25:58.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f904000c770 con 0x55fe7e11c390 2024-09-21T03:25:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f90577fe640 1 -- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 msgr2=0x55fe7e4f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f90577fe640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f90577fe640 1 -- 172.21.15.83:0/514116522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 msgr2=0x55fe7e4f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f90577fe640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.933+0000 7f90577fe640 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7e56b6e0 con 0x55fe7e11c390 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f9064e9d740 10 client.0 _ll_register_callbacks cb 0x7f9048004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:25:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9048026640 con 0x55fe7e11c390 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f90577fe640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90440db930 tx=0x7f90440da560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0fb05bd9c86c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f903b7fe640 10 client.26488 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f903b7fe640 1 -- 172.21.15.83:0/514116522 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440c9020 con 0x55fe7e11c390 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.934+0000 7f903b7fe640 1 -- 172.21.15.83:0/514116522 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9044040b70 con 0x55fe7e11c390 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f903b7fe640 1 -- 172.21.15.83:0/514116522 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9044059040 con 0x55fe7e11c390 2024-09-21T03:25:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f9064e9d740 10 client.26488 fetch_fsmap learned FSMap version 44 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55fe7e56c890 con 0x55fe7e11c390 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f903b7fe640 1 -- 172.21.15.83:0/514116522 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f90440da930 con 0x55fe7e11c390 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f9064e9d740 10 client.26488 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f9064e9d740 10 client.26488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.935+0000 7f9064e9d740 20 client.26488 populate_metadata read hostname 'smithi083' 2024-09-21T03:25:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.936+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0d0da0 0x55fe7e56eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.936+0000 7f9064e9d740 4 client.26488 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.936+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe7e4588a0 con 0x55fe7e0d0da0 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.936+0000 7f9057fff640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0d0da0 0x55fe7e56eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f903b7fe640 10 client.26488 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9057fff640 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0d0da0 0x55fe7e56eec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6024fc70b815088 server_cookie=4f10493b302fb3c0 in_seq=0 out_seq=0 2024-09-21T03:25:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f903b7fe640 1 -- 172.21.15.83:0/514116522 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe7e4588a0 con 0x55fe7e0d0da0 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f903b7fe640 10 client.26488 handle_command_reply: tid=0 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 client.26488 shutdown 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 msgr2=0x55fe7e4f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90440db930 tx=0x7f90440da560 comp rx=0 tx=0).stop 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 shutdown_connections 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55fe7e0d0da0 0x55fe7e56eec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f903b7fe640 0 client.26488 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe7e13d070 0x55fe7e4f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe7e11c390 0x55fe7e4f42f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 --2- 172.21.15.83:0/514116522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe7e1141b0 0x55fe7e4f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 >> 172.21.15.83:0/514116522 conn(0x55fe7e131460 msgr2=0x55fe7e4e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 shutdown_connections 2024-09-21T03:25:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 1 -- 172.21.15.83:0/514116522 wait complete. 2024-09-21T03:25:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.937+0000 7f9064e9d740 20 client.26488 trim_cache size 0 max 16384 2024-09-21T03:25:58.947 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:25:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 msgr2=0x7f903c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 0x7f903c0b8cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55fe7e4e2f40 tx=0x7f904800b390 comp rx=0 tx=0).stop 2024-09-21T03:25:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 msgr2=0x7f9058118780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f9044004a30 comp rx=0 tx=0).stop 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 shutdown_connections 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f903c0b67b0 0x7f903c0b8cc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905811cc30 0x7f9058118d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9058111f40 0x7f9058118780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 --2- 172.21.15.83:0/981636046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f905810a9e0 0x7f905811c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 >> 172.21.15.83:0/981636046 conn(0x7f9058070ec0 msgr2=0x7f905810ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 shutdown_connections 2024-09-21T03:25:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:25:58.938+0000 7f905e929640 1 -- 172.21.15.83:0/981636046 wait complete. 2024-09-21T03:25:59.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:25:59 smithi193 ceph-mon[35224]: pgmap v610: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-09-21T03:25:59.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:25:59 smithi083 ceph-mon[31219]: pgmap v610: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-09-21T03:25:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:25:59 smithi142 ceph-mon[34512]: pgmap v610: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-09-21T03:25:59.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:26:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 520.0 s, 1788.8 tps, lat 54.404 ms stddev 61.060 2024-09-21T03:26:01.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 -- 172.21.15.83:0/3932309710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 msgr2=0x7f78641098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 --2- 172.21.15.83:0/3932309710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f78641098c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7858009900 tx=0x7f785802a410 comp rx=0 tx=0).stop 2024-09-21T03:26:01.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 -- 172.21.15.83:0/3932309710 shutdown_connections 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 --2- 172.21.15.83:0/3932309710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864109e00 0x7f7864110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 --2- 172.21.15.83:0/3932309710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f78641098c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 --2- 172.21.15.83:0/3932309710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864101f70 0x7f7864102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 -- 172.21.15.83:0/3932309710 >> 172.21.15.83:0/3932309710 conn(0x7f7864077c60 msgr2=0x7f78640780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 -- 172.21.15.83:0/3932309710 shutdown_connections 2024-09-21T03:26:01.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.000+0000 7f78691c5640 1 -- 172.21.15.83:0/3932309710 wait complete. 2024-09-21T03:26:01.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.001+0000 7f78691c5640 1 Processor -- start 2024-09-21T03:26:01.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.001+0000 7f78691c5640 1 -- start start 2024-09-21T03:26:01.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 0x7f786419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864109e00 0x7f786419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7864113b10 con 0x7f78641094b0 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7864113990 con 0x7f7864109e00 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f78691c5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7864113c90 con 0x7f7864101f70 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57994/0 (socket says 172.21.15.83:57994) 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 -- 172.21.15.83:0/704253968 learned_addr learned my addr 172.21.15.83:0/704253968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f7862d9d640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 0x7f786419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786359e640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864109e00 0x7f786419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 -- 172.21.15.83:0/704253968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 msgr2=0x7f786419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 0x7f786419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 -- 172.21.15.83:0/704253968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864109e00 msgr2=0x7f786419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864109e00 0x7f786419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f786259c640 1 -- 172.21.15.83:0/704253968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78641a03f0 con 0x7f78641094b0 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.002+0000 7f7862d9d640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 0x7f786419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f786259c640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7858005ab0 tx=0x7f785803cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f784bfff640 1 -- 172.21.15.83:0/704253968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858035030 con 0x7f78641094b0 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78641a0680 con 0x7f78641094b0 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f784bfff640 1 -- 172.21.15.83:0/704253968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78580314a0 con 0x7f78641094b0 2024-09-21T03:26:01.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f784bfff640 1 -- 172.21.15.83:0/704253968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858044640 con 0x7f78641094b0 2024-09-21T03:26:01.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.003+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7864102df0 con 0x7f78641094b0 2024-09-21T03:26:01.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.006+0000 7f786f739740 1 Processor -- start 2024-09-21T03:26:01.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.006+0000 7f786f739740 1 -- start start 2024-09-21T03:26:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.006+0000 7f786f739740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e1086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.006+0000 7f786f739740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e108be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786f739740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786f739740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c17e07eea0 con 0x55c17dd40390 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786f739740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c17e06e2d0 con 0x55c17dd381b0 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786f739740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c17e0720e0 con 0x55c17dd61070 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e108be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e108be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36604/0 (socket says 172.21.15.83:36604) 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e1086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f7862d9d640 1 -- 172.21.15.83:0/741135547 learned_addr learned my addr 172.21.15.83:0/741135547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 -- 172.21.15.83:0/741135547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 msgr2=0x55c17e108be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e108be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 -- 172.21.15.83:0/741135547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 msgr2=0x55c17e109120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e109120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 -- 172.21.15.83:0/741135547 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17ddcc650 con 0x55c17dd61070 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f784bfff640 1 -- 172.21.15.83:0/704253968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7858003a00 con 0x7f78641094b0 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.007+0000 7f786359e640 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e1086a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f785c009f00 tx=0x7f785c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1da0f82fefb487d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.008+0000 7f78497fa640 1 -- 172.21.15.83:0/741135547 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c004d20 con 0x55c17dd61070 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.008+0000 7f78497fa640 1 -- 172.21.15.83:0/741135547 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c004030 con 0x55c17dd61070 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.008+0000 7f78497fa640 1 -- 172.21.15.83:0/741135547 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c005260 con 0x55c17dd61070 2024-09-21T03:26:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.008+0000 7f786f739740 1 -- 172.21.15.83:0/741135547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 msgr2=0x55c17e1086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.008+0000 7f786f739740 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e1086a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f785c009f00 tx=0x7f785c011810 comp rx=0 tx=0).stop 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f784bfff640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 0x7f784c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 -- 172.21.15.83:0/741135547 shutdown_connections 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 --2- 172.21.15.83:0/741135547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e1086a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 -- 172.21.15.83:0/741135547 >> 172.21.15.83:0/741135547 conn(0x55c17dd55460 msgr2=0x55c17e106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f784bfff640 1 -- 172.21.15.83:0/704253968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f78581079f0 con 0x7f78641094b0 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786359e640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 0x7f784c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 -- 172.21.15.83:0/741135547 shutdown_connections 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.009+0000 7f786f739740 1 -- 172.21.15.83:0/741135547 wait complete. 2024-09-21T03:26:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786359e640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 0x7f784c0b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785c0048b0 tx=0x7f785c004910 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786f739740 1 -- start start 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786f739740 1 Processor -- start 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786f739740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786f739740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd40390 0x55c17e11cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.010+0000 7f786f739740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd61070 0x55c17e128910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786f739740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c17decf700 con 0x55c17dd61070 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43576/0 (socket says 172.21.15.83:43576) 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 -- 172.21.15.83:0/2154142099 learned_addr learned my addr 172.21.15.83:0/2154142099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd61070 0x55c17e128910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786259c640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd40390 0x55c17e11cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786f739740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c17e009160 con 0x55c17dd40390 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c17e11f5b0 con 0x55c17dd381b0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f786f739740 10 client.0 _ll_register_callbacks cb 0x55c17e0229e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 -- 172.21.15.83:0/2154142099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd40390 msgr2=0x55c17e11cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd40390 0x55c17e11cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 -- 172.21.15.83:0/2154142099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd61070 msgr2=0x55c17e128910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd61070 0x55c17e128910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 -- 172.21.15.83:0/2154142099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17e036c90 con 0x55c17dd381b0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7854012d00 tx=0x7f785400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8cb8bf3f3781175 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.011+0000 7f7846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785400fe20 con 0x55c17dd381b0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f78540120d0 con 0x55c17dd381b0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c004030 con 0x55c17dd381b0 2024-09-21T03:26:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7854004030 con 0x55c17dd381b0 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 10 client.26500 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c17e12b330 con 0x55c17dd381b0 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c004030 con 0x55c17dd381b0 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7854026480 con 0x55c17dd381b0 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 10 client.26500 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 10 client.26500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 20 client.26500 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dd1f9f0 0x55c17e12e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 4 client.26500 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.012+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c17e080df0 con 0x55c17dd1f9f0 2024-09-21T03:26:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.013+0000 7f786259c640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dd1f9f0 0x55c17e12e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.013+0000 7f7846ff5640 10 client.26500 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.013+0000 7f786259c640 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dd1f9f0 0x55c17e12e7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d15dbea17da9266e server_cookie=261f89cba8a1a1ba in_seq=0 out_seq=0 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f7846ff5640 1 -- 172.21.15.83:0/2154142099 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c17e080df0 con 0x55c17dd1f9f0 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f7846ff5640 10 client.26500 handle_command_reply: tid=0 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 client.26500 shutdown 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 msgr2=0x55c17e11c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7854012d00 tx=0x7f785400f790 comp rx=0 tx=0).stop 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 shutdown_connections 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dd1f9f0 0x55c17e12e7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd61070 0x55c17e128910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd40390 0x55c17e11cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 --2- 172.21.15.83:0/2154142099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd381b0 0x55c17e11c5a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 >> 172.21.15.83:0/2154142099 conn(0x55c17dd55460 msgr2=0x55c17e104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.014+0000 7f7846ff5640 0 client.26500 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.015+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 shutdown_connections 2024-09-21T03:26:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.015+0000 7f786f739740 1 -- 172.21.15.83:0/2154142099 wait complete. 2024-09-21T03:26:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.015+0000 7f786f739740 20 client.26500 trim_cache size 0 max 16384 2024-09-21T03:26:01.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 Processor -- start 2024-09-21T03:26:01.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 -- start start 2024-09-21T03:26:01.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e119790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e119cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e119790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e119790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58014/0 (socket says 172.21.15.83:58014) 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36622/0 (socket says 172.21.15.83:36622) 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ad270 con 0x55c17dd40390 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.029+0000 7f786f739740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78641aedb0 con 0x55c17dd381b0 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f786259c640 1 -- 172.21.15.83:0/2652460629 learned_addr learned my addr 172.21.15.83:0/2652460629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f786359e640 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e119cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ad940 con 0x55c17dd61070 2024-09-21T03:26:01.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 -- 172.21.15.83:0/2652460629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 msgr2=0x55c17e119cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e119cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 -- 172.21.15.83:0/2652460629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 msgr2=0x55c17e119790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e119790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 -- 172.21.15.83:0/2652460629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17e10a320 con 0x55c17dd381b0 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.030+0000 7f7862d9d640 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f7854005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735688c99f04654c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.031+0000 7f7848ff9640 1 -- 172.21.15.83:0/2652460629 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854026540 con 0x55c17dd381b0 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.031+0000 7f7848ff9640 1 -- 172.21.15.83:0/2652460629 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78540120d0 con 0x55c17dd381b0 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.031+0000 7f7848ff9640 1 -- 172.21.15.83:0/2652460629 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78540035f0 con 0x55c17dd381b0 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 msgr2=0x55c17e109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f7854005970 comp rx=0 tx=0).stop 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 shutdown_connections 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd61070 0x55c17e119cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd40390 0x55c17e119790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 --2- 172.21.15.83:0/2652460629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd381b0 0x55c17e109de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 >> 172.21.15.83:0/2652460629 conn(0x55c17dd55460 msgr2=0x55c17e1096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 shutdown_connections 2024-09-21T03:26:01.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- 172.21.15.83:0/2652460629 wait complete. 2024-09-21T03:26:01.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 -- start start 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.032+0000 7f786f739740 1 Processor -- start 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd40390 0x55c17e1225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd61070 0x55c17e122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ae6e0 con 0x55c17dd381b0 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ae010 con 0x55c17dd61070 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78641ab250 con 0x55c17dd40390 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58024/0 (socket says 172.21.15.83:58024) 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786f739740 10 client.0 _ll_register_callbacks cb 0x7f7854009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 -- 172.21.15.83:0/4119667198 learned_addr learned my addr 172.21.15.83:0/4119667198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786359e640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd61070 0x55c17e122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f786259c640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd40390 0x55c17e1225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 -- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd40390 msgr2=0x55c17e1225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd40390 0x55c17e1225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 -- 172.21.15.83:0/4119667198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd61070 msgr2=0x55c17e122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd61070 0x55c17e122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 -- 172.21.15.83:0/4119667198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17e123340 con 0x55c17dd381b0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7862d9d640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f785400f7d0 tx=0x7f7854026770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a21b27d6c147f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.033+0000 7f7846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854005720 con 0x55c17dd381b0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f7854005040 con 0x55c17dd381b0 2024-09-21T03:26:01.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78540120d0 con 0x55c17dd381b0 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f785403f050 con 0x55c17dd381b0 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 10 client.16989 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c17e199180 con 0x55c17dd381b0 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78540120d0 con 0x55c17dd381b0 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f785402e030 con 0x55c17dd381b0 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 10 client.16989 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 10 client.16989 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 20 client.16989 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dcf4da0 0x55c17e19b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 4 client.16989 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:01.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.034+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c17e07c8a0 con 0x55c17dcf4da0 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786259c640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dcf4da0 0x55c17e19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f7846ff5640 10 client.16989 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786259c640 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dcf4da0 0x55c17e19b800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f822565c4c221ce6 server_cookie=f4f1043622d8f5e in_seq=0 out_seq=0 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f7846ff5640 1 -- 172.21.15.83:0/4119667198 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c17e07c8a0 con 0x55c17dcf4da0 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f7846ff5640 10 client.16989 handle_command_reply: tid=0 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786f739740 1 client.16989 shutdown 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 msgr2=0x55c17e125100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f785400f7d0 tx=0x7f7854026770 comp rx=0 tx=0).stop 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 shutdown_connections 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.035+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c17dcf4da0 0x55c17e19b800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c17dd61070 0x55c17e122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f7846ff5640 0 client.16989 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c17dd40390 0x55c17e1225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 --2- 172.21.15.83:0/4119667198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17dd381b0 0x55c17e125100 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 >> 172.21.15.83:0/4119667198 conn(0x55c17dd55460 msgr2=0x55c17e11d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 shutdown_connections 2024-09-21T03:26:01.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 1 -- 172.21.15.83:0/4119667198 wait complete. 2024-09-21T03:26:01.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f786f739740 20 client.16989 trim_cache size 0 max 16384 2024-09-21T03:26:01.042 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 msgr2=0x7f784c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 0x7f784c0b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785c0048b0 tx=0x7f785c004910 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 msgr2=0x7f786419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.036+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7858005ab0 tx=0x7f785803cc90 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 shutdown_connections 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f784c0b6680 0x7f784c0b8b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7864109e00 0x7f786419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78641094b0 0x7f786419aa20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 --2- 172.21.15.83:0/704253968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864101f70 0x7f786419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:01.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 >> 172.21.15.83:0/704253968 conn(0x7f7864077c60 msgr2=0x7f78641074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:01.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 shutdown_connections 2024-09-21T03:26:01.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:01.037+0000 7f78691c5640 1 -- 172.21.15.83:0/704253968 wait complete. 2024-09-21T03:26:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:01 smithi193 ceph-mon[35224]: pgmap v611: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-09-21T03:26:01.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:01.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:01 smithi083 ceph-mon[31219]: pgmap v611: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-09-21T03:26:01.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:01 smithi142 ceph-mon[34512]: pgmap v611: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-09-21T03:26:01.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:02.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:02.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:02.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:26:02.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:02.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:02.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:26:02.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:02.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:02.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:26:02.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:26:02.942 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:03.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:03 smithi083 ceph-mon[31219]: pgmap v612: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-09-21T03:26:03.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.548+0000 7f4176a79640 1 -- 172.21.15.83:0/1176188963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170102870 msgr2=0x7f4170102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.548+0000 7f4176a79640 1 --2- 172.21.15.83:0/1176188963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170102870 0x7f4170102c80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f416c014ce0 tx=0x7f416c032700 comp rx=0 tx=0).stop 2024-09-21T03:26:03.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.551+0000 7f4176a79640 1 -- 172.21.15.83:0/1176188963 shutdown_connections 2024-09-21T03:26:03.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.551+0000 7f4176a79640 1 --2- 172.21.15.83:0/1176188963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170108b00 0x7f417010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.551+0000 7f4176a79640 1 --2- 172.21.15.83:0/1176188963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 0x7f41701085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.551+0000 7f4176a79640 1 --2- 172.21.15.83:0/1176188963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170102870 0x7f4170102c80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.551+0000 7f4176a79640 1 -- 172.21.15.83:0/1176188963 >> 172.21.15.83:0/1176188963 conn(0x7f41700fe2c0 msgr2=0x7f4170100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.558+0000 7f4176a79640 1 -- 172.21.15.83:0/1176188963 shutdown_connections 2024-09-21T03:26:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.558+0000 7f4176a79640 1 -- 172.21.15.83:0/1176188963 wait complete. 2024-09-21T03:26:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.559+0000 7f4176a79640 1 Processor -- start 2024-09-21T03:26:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.559+0000 7f4176a79640 1 -- start start 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 0x7f41701a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 0x7f41701ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4170111700 con 0x7f4170108b00 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4170111580 con 0x7f4170102870 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176a79640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4170111880 con 0x7f41701081b0 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4175a77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4175a77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36648/0 (socket says 172.21.15.83:36648) 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4175a77640 1 -- 172.21.15.83:0/1704680667 learned_addr learned my addr 172.21.15.83:0/1704680667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.560+0000 7f4176278640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 0x7f41701ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4175276640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 0x7f41701a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4176278640 1 -- 172.21.15.83:0/1704680667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 msgr2=0x7f41701a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4176278640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 0x7f41701a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4176278640 1 -- 172.21.15.83:0/1704680667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 msgr2=0x7f41701a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4176278640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4176278640 1 -- 172.21.15.83:0/1704680667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41701af030 con 0x7f4170108b00 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.561+0000 7f4175a77640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.562+0000 7f4176278640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 0x7f41701ae900 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f416800bea0 tx=0x7f416800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.562+0000 7f4166ffd640 1 -- 172.21.15.83:0/1704680667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168009bd0 con 0x7f4170108b00 2024-09-21T03:26:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.562+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41701af320 con 0x7f4170108b00 2024-09-21T03:26:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.562+0000 7f4166ffd640 1 -- 172.21.15.83:0/1704680667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4168009d70 con 0x7f4170108b00 2024-09-21T03:26:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:03 smithi193 ceph-mon[35224]: pgmap v612: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-09-21T03:26:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.562+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41701b5a80 con 0x7f4170108b00 2024-09-21T03:26:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.563+0000 7f4166ffd640 1 -- 172.21.15.83:0/1704680667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168009bd0 con 0x7f4170108b00 2024-09-21T03:26:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.566+0000 7f417e24f740 1 Processor -- start 2024-09-21T03:26:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.568+0000 7f4166ffd640 1 -- 172.21.15.83:0/1704680667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4168003a00 con 0x7f4170108b00 2024-09-21T03:26:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.569+0000 7f4166ffd640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 0x7f415c0b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.569+0000 7f4175276640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 0x7f415c0b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f4166ffd640 1 -- 172.21.15.83:0/1704680667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f416800d400 con 0x7f4170108b00 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 -- start start 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe905460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe5351b0 0x55bbfe9059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe905ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe87bea0 con 0x55bbfe55e070 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe86b2d0 con 0x55bbfe53d390 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.571+0000 7f417e24f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe86f0e0 con 0x55bbfe5351b0 2024-09-21T03:26:03.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4176278640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe905ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4176278640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe905ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36652/0 (socket says 172.21.15.83:36652) 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4176278640 1 -- 172.21.15.83:0/2845693676 learned_addr learned my addr 172.21.15.83:0/2845693676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe905460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175276640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 0x7f415c0b8fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4160005fd0 tx=0x7f4160005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175276640 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe5351b0 0x55bbfe9059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 -- 172.21.15.83:0/2845693676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe5351b0 msgr2=0x55bbfe9059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe5351b0 0x55bbfe9059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 -- 172.21.15.83:0/2845693676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 msgr2=0x55bbfe905ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe905ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.572+0000 7f4175a77640 1 -- 172.21.15.83:0/2845693676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbfe5c9650 con 0x55bbfe55e070 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.573+0000 7f4175a77640 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe905460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f416c014cb0 tx=0x7f416c03aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f97a4617b08b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.573+0000 7f415bfff640 1 -- 172.21.15.83:0/2845693676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c012040 con 0x55bbfe55e070 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.573+0000 7f415bfff640 1 -- 172.21.15.83:0/2845693676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416c047070 con 0x55bbfe55e070 2024-09-21T03:26:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.573+0000 7f415bfff640 1 -- 172.21.15.83:0/2845693676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0046c0 con 0x55bbfe55e070 2024-09-21T03:26:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 -- 172.21.15.83:0/2845693676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 msgr2=0x55bbfe905460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe905460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f416c014cb0 tx=0x7f416c03aec0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 -- 172.21.15.83:0/2845693676 shutdown_connections 2024-09-21T03:26:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe905ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe5351b0 0x55bbfe9059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 --2- 172.21.15.83:0/2845693676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe905460 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 -- 172.21.15.83:0/2845693676 >> 172.21.15.83:0/2845693676 conn(0x55bbfe552460 msgr2=0x55bbfe903350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 -- 172.21.15.83:0/2845693676 shutdown_connections 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.574+0000 7f417e24f740 1 -- 172.21.15.83:0/2845693676 wait complete. 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 -- start start 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 Processor -- start 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe910b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe91cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe6cc700 con 0x55bbfe55e070 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe806160 con 0x55bbfe5351b0 2024-09-21T03:26:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbfe913be0 con 0x55bbfe53d390 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.575+0000 7f417e24f740 10 client.0 _ll_register_callbacks cb 0x55bbfe81f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:43630/0 (socket says 172.21.15.83:43630) 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4176278640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe91cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 -- 172.21.15.83:0/909160707 learned_addr learned my addr 172.21.15.83:0/909160707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175a77640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe910b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 -- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 msgr2=0x55bbfe910b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe910b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 -- 172.21.15.83:0/909160707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 msgr2=0x55bbfe91cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe91cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 -- 172.21.15.83:0/909160707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbfe833c90 con 0x55bbfe53d390 2024-09-21T03:26:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175a77640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe910b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f4175276640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f417006aa90 tx=0x7f41600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f7866aff55b89df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f41597fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160023070 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416c047070 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.576+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f41600109a0 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f416002f050 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 10 client.26515 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bbfe91f990 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416c047070 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f416000f040 con 0x55bbfe53d390 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 10 client.26515 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 10 client.26515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 20 client.26515 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe51c9f0 0x55bbfe922e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 4 client.26515 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.577+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbfe87ddf0 con 0x55bbfe51c9f0 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.578+0000 7f4175a77640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe51c9f0 0x55bbfe922e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.578+0000 7f4175a77640 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe51c9f0 0x55bbfe922e80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc925fdca98c549 server_cookie=eb2240bbff0aadab in_seq=0 out_seq=0 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.578+0000 7f41597fa640 10 client.26515 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f41597fa640 1 -- 172.21.15.83:0/909160707 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbfe87ddf0 con 0x55bbfe51c9f0 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f41597fa640 10 client.26515 handle_command_reply: tid=0 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 client.26515 shutdown 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 msgr2=0x55bbfe911080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f417006aa90 tx=0x7f41600046c0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 shutdown_connections 2024-09-21T03:26:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe51c9f0 0x55bbfe922e80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe55e070 0x55bbfe91cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe53d390 0x55bbfe911080 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 --2- 172.21.15.83:0/909160707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe910b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 >> 172.21.15.83:0/909160707 conn(0x55bbfe552460 msgr2=0x55bbfe908640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f41597fa640 0 client.26515 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 shutdown_connections 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 1 -- 172.21.15.83:0/909160707 wait complete. 2024-09-21T03:26:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.579+0000 7f417e24f740 20 client.26515 trim_cache size 0 max 16384 2024-09-21T03:26:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 Processor -- start 2024-09-21T03:26:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 -- start start 2024-09-21T03:26:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe90fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe902980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41701be1e0 con 0x55bbfe53d390 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41701bfd20 con 0x55bbfe5351b0 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f417e24f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41701be8b0 con 0x55bbfe55e070 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f4175276640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f4175276640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58078/0 (socket says 172.21.15.83:58078) 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f4175276640 1 -- 172.21.15.83:0/395553344 learned_addr learned my addr 172.21.15.83:0/395553344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.595+0000 7f4175a77640 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe90fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4176278640 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe902980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 -- 172.21.15.83:0/395553344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 msgr2=0x55bbfe902980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe902980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 -- 172.21.15.83:0/395553344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 msgr2=0x55bbfe90fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe90fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 -- 172.21.15.83:0/395553344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbfe903cc0 con 0x55bbfe53d390 2024-09-21T03:26:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f4175276640 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600275c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79c666afba1150c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f415b7fe640 1 -- 172.21.15.83:0/395553344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160038070 con 0x55bbfe53d390 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f415b7fe640 1 -- 172.21.15.83:0/395553344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160017040 con 0x55bbfe53d390 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.596+0000 7f415b7fe640 1 -- 172.21.15.83:0/395553344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160023070 con 0x55bbfe53d390 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- 172.21.15.83:0/395553344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 msgr2=0x55bbfe906ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600275c0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- 172.21.15.83:0/395553344 shutdown_connections 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe902980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe53d390 0x55bbfe906ba0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 --2- 172.21.15.83:0/395553344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe5351b0 0x55bbfe90fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- 172.21.15.83:0/395553344 >> 172.21.15.83:0/395553344 conn(0x55bbfe552460 msgr2=0x55bbfe906490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- 172.21.15.83:0/395553344 shutdown_connections 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- 172.21.15.83:0/395553344 wait complete. 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.597+0000 7f417e24f740 1 -- start start 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 Processor -- start 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe914c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe9152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41701bf650 con 0x55bbfe5351b0 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41701bef80 con 0x55bbfe53d390 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41701bc1e0 con 0x55bbfe55e070 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f417e24f740 10 client.0 _ll_register_callbacks cb 0x7f41600092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f4175a77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f4175a77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58084/0 (socket says 172.21.15.83:58084) 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f4175a77640 1 -- 172.21.15.83:0/2505570922 learned_addr learned my addr 172.21.15.83:0/2505570922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f4176278640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe9152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.598+0000 7f4175276640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe914c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 -- 172.21.15.83:0/2505570922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 msgr2=0x55bbfe9152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe9152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 -- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 msgr2=0x55bbfe914c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe914c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 -- 172.21.15.83:0/2505570922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbfe915a30 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f4175a77640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f416c014e00 tx=0x7f416c03ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644c65979613936b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f41597fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c012040 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f416c039a20 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4160017040 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f416c039610 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f417e24f740 10 client.17007 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bbfe98d8e0 con 0x55bbfe5351b0 2024-09-21T03:26:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.599+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4160017040 con 0x55bbfe5351b0 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f416c004d00 con 0x55bbfe5351b0 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 10 client.17007 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 20 client.17007 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe4f1da0 0x55bbfe98fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 4 client.17007 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbfe8fe7a0 con 0x55bbfe4f1da0 2024-09-21T03:26:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f4175276640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe4f1da0 0x55bbfe98fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.600+0000 7f41597fa640 10 client.17007 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f4175276640 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe4f1da0 0x55bbfe98fbd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b7bcecdacc437f server_cookie=6c40b67ce8186e6b in_seq=0 out_seq=0 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f41597fa640 1 -- 172.21.15.83:0/2505570922 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbfe8fe7a0 con 0x55bbfe4f1da0 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f41597fa640 10 client.17007 handle_command_reply: tid=0 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 client.17007 shutdown 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 msgr2=0x55bbfe919770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f416c014e00 tx=0x7f416c03ad70 comp rx=0 tx=0).stop 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 shutdown_connections 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55bbfe4f1da0 0x55bbfe98fbd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f41597fa640 0 client.17007 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfe55e070 0x55bbfe9152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbfe53d390 0x55bbfe914c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 --2- 172.21.15.83:0/2505570922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbfe5351b0 0x55bbfe919770 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 >> 172.21.15.83:0/2505570922 conn(0x55bbfe552460 msgr2=0x55bbfe906e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 shutdown_connections 2024-09-21T03:26:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 1 -- 172.21.15.83:0/2505570922 wait complete. 2024-09-21T03:26:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.601+0000 7f417e24f740 20 client.17007 trim_cache size 0 max 16384 2024-09-21T03:26:03.608 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 msgr2=0x7f415c0b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 0x7f415c0b8fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4160005fd0 tx=0x7f4160005850 comp rx=0 tx=0).stop 2024-09-21T03:26:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 msgr2=0x7f41701ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 0x7f41701ae900 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f416800bea0 tx=0x7f416800bf90 comp rx=0 tx=0).stop 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 shutdown_connections 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f415c0b6ad0 0x7f415c0b8fe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4170108b00 0x7f41701ae900 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41701081b0 0x7f41701a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 --2- 172.21.15.83:0/1704680667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4170102870 0x7f41701a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 >> 172.21.15.83:0/1704680667 conn(0x7f41700fe2c0 msgr2=0x7f41701006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 shutdown_connections 2024-09-21T03:26:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:03.602+0000 7f4176a79640 1 -- 172.21.15.83:0/1704680667 wait complete. 2024-09-21T03:26:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:03 smithi142 ceph-mon[34512]: pgmap v612: 97 pgs: 97 active+clean; 4.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-09-21T03:26:05.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:26:05.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:26:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:05 smithi193 ceph-mon[35224]: pgmap v613: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2024-09-21T03:26:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:05 smithi193 ceph-mon[35224]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:26:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:05 smithi083 ceph-mon[31219]: pgmap v613: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2024-09-21T03:26:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:05 smithi083 ceph-mon[31219]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:26:05.653 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:05 smithi142 ceph-mon[34512]: pgmap v613: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.21k op/s 2024-09-21T03:26:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:05 smithi142 ceph-mon[34512]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:26:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 525.0 s, 737.0 tps, lat 138.029 ms stddev 262.122 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 -- 172.21.15.83:0/1021861255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3044108cf0 msgr2=0x7f304410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 --2- 172.21.15.83:0/1021861255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3044108cf0 0x7f304410f520 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f303c009ca0 tx=0x7f303c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 -- 172.21.15.83:0/1021861255 shutdown_connections 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 --2- 172.21.15.83:0/1021861255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3044108cf0 0x7f304410f520 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 --2- 172.21.15.83:0/1021861255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30441083a0 0x7f30441087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 --2- 172.21.15.83:0/1021861255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f3044102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 -- 172.21.15.83:0/1021861255 >> 172.21.15.83:0/1021861255 conn(0x7f3044079730 msgr2=0x7f3044079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 -- 172.21.15.83:0/1021861255 shutdown_connections 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.208+0000 7f304ad0e640 1 -- 172.21.15.83:0/1021861255 wait complete. 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.209+0000 7f304ad0e640 1 Processor -- start 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.209+0000 7f304ad0e640 1 -- start start 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.210+0000 7f304ad0e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f304419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.210+0000 7f304ad0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.210+0000 7f304ad0e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3044108cf0 0x7f304419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58100/0 (socket says 172.21.15.83:58100) 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3048aaa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f304419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 -- 172.21.15.83:0/669339228 learned_addr learned my addr 172.21.15.83:0/669339228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f30492ab640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3044108cf0 0x7f304419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3044112960 con 0x7f30441083a0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30441127e0 con 0x7f3044102a60 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3044112ae0 con 0x7f3044108cf0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 -- 172.21.15.83:0/669339228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3044108cf0 msgr2=0x7f304419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3044108cf0 0x7f304419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 -- 172.21.15.83:0/669339228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 msgr2=0x7f304419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f304419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30441a0410 con 0x7f30441083a0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3048aaa640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f304419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.211+0000 7f3043fff640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9c0 tx=0x7f303000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.212+0000 7f3041ffb640 1 -- 172.21.15.83:0/669339228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303000cd80 con 0x7f30441083a0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.212+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30441a0700 con 0x7f30441083a0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.212+0000 7f3041ffb640 1 -- 172.21.15.83:0/669339228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30300054d0 con 0x7f30441083a0 2024-09-21T03:26:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.212+0000 7f3041ffb640 1 -- 172.21.15.83:0/669339228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030010430 con 0x7f30441083a0 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.212+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441a6e60 con 0x7f30441083a0 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.215+0000 7f3051282740 1 Processor -- start 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.215+0000 7f3051282740 1 -- start start 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3041ffb640 1 -- 172.21.15.83:0/669339228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3030004fd0 con 0x7f30441083a0 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2463970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2463eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ddb23d9e60 con 0x55ddb20bc350 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ddb23c9290 con 0x55ddb209b670 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3051282740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ddb23cd0a0 con 0x55ddb2093490 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f30492ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f30492ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58108/0 (socket says 172.21.15.83:58108) 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f30492ab640 1 -- 172.21.15.83:0/3049462003 learned_addr learned my addr 172.21.15.83:0/3049462003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3048aaa640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2463970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.216+0000 7f3043fff640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2463eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 -- 172.21.15.83:0/3049462003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 msgr2=0x55ddb2463970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2463970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 -- 172.21.15.83:0/3049462003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 msgr2=0x55ddb2463eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2463eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 -- 172.21.15.83:0/3049462003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb2127930 con 0x55ddb20bc350 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f3048aaa640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2463970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f3041ffb640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 0x7f30340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f30492ab640 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f303c02f410 tx=0x7f303c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c170027c3e0c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f3048aaa640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 0x7f30340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f3041ffb640 1 -- 172.21.15.83:0/669339228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7f30300e7aa0 con 0x7f30441083a0 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f302effd640 1 -- 172.21.15.83:0/3049462003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c031cf0 con 0x55ddb20bc350 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f302effd640 1 -- 172.21.15.83:0/3049462003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c036030 con 0x55ddb20bc350 2024-09-21T03:26:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.217+0000 7f302effd640 1 -- 172.21.15.83:0/3049462003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c0042a0 con 0x55ddb20bc350 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 -- 172.21.15.83:0/3049462003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 msgr2=0x55ddb2463430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3048aaa640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 0x7f30340b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3038009fd0 tx=0x7f3038005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f303c02f410 tx=0x7f303c02f440 comp rx=0 tx=0).stop 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 -- 172.21.15.83:0/3049462003 shutdown_connections 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2463eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2463970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 --2- 172.21.15.83:0/3049462003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb2463430 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 -- 172.21.15.83:0/3049462003 >> 172.21.15.83:0/3049462003 conn(0x55ddb20b0740 msgr2=0x55ddb2461320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 -- 172.21.15.83:0/3049462003 shutdown_connections 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.218+0000 7f3051282740 1 -- 172.21.15.83:0/3049462003 wait complete. 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 -- start start 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 Processor -- start 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb246b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 0x55ddb24834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ddb222a890 con 0x55ddb2093490 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2363d40 con 0x55ddb20bc350 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ddb24663b0 con 0x55ddb209b670 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3048aaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3048aaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58110/0 (socket says 172.21.15.83:58110) 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3048aaa640 1 -- 172.21.15.83:0/1642168830 learned_addr learned my addr 172.21.15.83:0/1642168830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f30492ab640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 0x55ddb24834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3043fff640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb246b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.219+0000 7f3051282740 10 client.0 _ll_register_callbacks cb 0x55ddb23c7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 -- 172.21.15.83:0/1642168830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 msgr2=0x55ddb246b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb246b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 -- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 msgr2=0x55ddb24834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 0x55ddb24834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 -- 172.21.15.83:0/1642168830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb2483ac0 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f30492ab640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 0x55ddb24834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3048aaa640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3038012e50 tx=0x7f3038004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20963966caccafda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3023fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303800fe20 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c036030 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7f3038011b50 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3038011d50 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3051282740 10 client.17019 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ddb2486440 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.220+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c036030 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f303802f070 con 0x55ddb2093490 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 10 client.17019 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 20 client.17019 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb207acd0 0x55ddb2489930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 4 client.17019 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb23dbdb0 con 0x55ddb207acd0 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.221+0000 7f3043fff640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb207acd0 0x55ddb2489930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.223+0000 7f3043fff640 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb207acd0 0x55ddb2489930 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a58b19000826d7d server_cookie=d468bba07e4cc843 in_seq=0 out_seq=0 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.223+0000 7f3023fff640 10 client.17019 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.223+0000 7f3023fff640 1 -- 172.21.15.83:0/1642168830 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ddb23dbdb0 con 0x55ddb207acd0 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.223+0000 7f3023fff640 10 client.17019 handle_command_reply: tid=0 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 client.17019 shutdown 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 msgr2=0x55ddb246afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3038012e50 tx=0x7f3038004730 comp rx=0 tx=0).stop 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 shutdown_connections 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb207acd0 0x55ddb2489930 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb20bc350 0x55ddb24834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3023fff640 0 client.17019 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb246b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 --2- 172.21.15.83:0/1642168830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb2093490 0x55ddb246afe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 >> 172.21.15.83:0/1642168830 conn(0x55ddb20b0740 msgr2=0x55ddb245f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 shutdown_connections 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 1 -- 172.21.15.83:0/1642168830 wait complete. 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.224+0000 7f3051282740 20 client.17019 trim_cache size 0 max 16384 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 Processor -- start 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 -- start start 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2469660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2469ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30441ad4f0 con 0x55ddb20bc350 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30441af030 con 0x55ddb209b670 2024-09-21T03:26:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.250+0000 7f3051282740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30441adbc0 con 0x55ddb2093490 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.255+0000 7f30492ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.255+0000 7f30492ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58112/0 (socket says 172.21.15.83:58112) 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.255+0000 7f30492ab640 1 -- 172.21.15.83:0/1291826295 learned_addr learned my addr 172.21.15.83:0/1291826295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.255+0000 7f3043fff640 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2469ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.258+0000 7f30492ab640 1 -- 172.21.15.83:0/1291826295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 msgr2=0x55ddb2469660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.258+0000 7f30492ab640 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2469660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.258+0000 7f30492ab640 1 -- 172.21.15.83:0/1291826295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 msgr2=0x55ddb2469ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.258+0000 7f30492ab640 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2469ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.258+0000 7f30492ab640 1 -- 172.21.15.83:0/1291826295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb2478d50 con 0x55ddb20bc350 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.265+0000 7f30492ab640 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f303c003c90 tx=0x7f303c003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acee6302f995e671 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.268+0000 7f302e7fc640 1 -- 172.21.15.83:0/1291826295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c055020 con 0x55ddb20bc350 2024-09-21T03:26:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.268+0000 7f302e7fc640 1 -- 172.21.15.83:0/1291826295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c036040 con 0x55ddb20bc350 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.268+0000 7f302e7fc640 1 -- 172.21.15.83:0/1291826295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c04a040 con 0x55ddb20bc350 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.271+0000 7f3051282740 1 -- 172.21.15.83:0/1291826295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 msgr2=0x55ddb245f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.271+0000 7f3051282740 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f303c003c90 tx=0x7f303c003cc0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.272+0000 7f3051282740 1 -- 172.21.15.83:0/1291826295 shutdown_connections 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.272+0000 7f3051282740 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb245f850 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.272+0000 7f3051282740 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb209b670 0x55ddb2469ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.272+0000 7f3051282740 1 --2- 172.21.15.83:0/1291826295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb2093490 0x55ddb2469660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.272+0000 7f3051282740 1 -- 172.21.15.83:0/1291826295 >> 172.21.15.83:0/1291826295 conn(0x55ddb20b0740 msgr2=0x55ddb246afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.277+0000 7f3051282740 1 -- 172.21.15.83:0/1291826295 shutdown_connections 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.277+0000 7f3051282740 1 -- 172.21.15.83:0/1291826295 wait complete. 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.278+0000 7f3051282740 1 -- start start 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.278+0000 7f3051282740 1 Processor -- start 2024-09-21T03:26:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb247d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb247d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30441ae960 con 0x55ddb20bc350 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e5cdcd58-df20-47cb-bced-27da9f50e8be", 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:26:07.024 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30441ae290 con 0x55ddb2093490 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.279+0000 7f3051282740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30441ab4d0 con 0x55ddb209b670 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.280+0000 7f3051282740 10 client.0 _ll_register_callbacks cb 0x7f3038009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3043fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb247d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36734/0 (socket says 172.21.15.83:36734) 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 -- 172.21.15.83:0/162941569 learned_addr learned my addr 172.21.15.83:0/162941569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f30492ab640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb247d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 -- 172.21.15.83:0/162941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 msgr2=0x55ddb247d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb247d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 -- 172.21.15.83:0/162941569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 msgr2=0x55ddb247d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb247d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.281+0000 7f3048aaa640 1 -- 172.21.15.83:0/162941569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb247df00 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3048aaa640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f303800f800 tx=0x7f3038029dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c6560c215f648a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3023fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038035020 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c036040 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7f3038031670 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3038040050 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3051282740 10 client.26561 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ddb246db70 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.282+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c036040 con 0x55ddb2093490 2024-09-21T03:26:07.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f303802f040 con 0x55ddb2093490 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 10 client.26561 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 20 client.26561 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb2050080 0x55ddb2470400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 4 client.26561 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ddb23d7860 con 0x55ddb2050080 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.283+0000 7f3043fff640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb2050080 0x55ddb2470400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.284+0000 7f3043fff640 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb2050080 0x55ddb2470400 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18e206cedaf97c4 server_cookie=3ea81a9622337693 in_seq=0 out_seq=0 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.284+0000 7f3023fff640 10 client.26561 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.288+0000 7f3023fff640 1 -- 172.21.15.83:0/162941569 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ddb23d7860 con 0x55ddb2050080 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.288+0000 7f3023fff640 10 client.26561 handle_command_reply: tid=0 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.288+0000 7f3051282740 1 client.26561 shutdown 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.290+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 msgr2=0x55ddb247fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.290+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f303800f800 tx=0x7f3038029dd0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.290+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 shutdown_connections 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ddb2050080 0x55ddb2470400 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddb20bc350 0x55ddb247d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ddb209b670 0x55ddb247d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3051282740 1 --2- 172.21.15.83:0/162941569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ddb2093490 0x55ddb247fa10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 >> 172.21.15.83:0/162941569 conn(0x55ddb20b0740 msgr2=0x55ddb245f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.291+0000 7f3023fff640 0 client.26561 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.292+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 shutdown_connections 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.292+0000 7f3051282740 1 -- 172.21.15.83:0/162941569 wait complete. 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.292+0000 7f3051282740 20 client.26561 trim_cache size 0 max 16384 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 msgr2=0x7f30340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 0x7f30340b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3038009fd0 tx=0x7f3038005f90 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 msgr2=0x7f304419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9c0 tx=0x7f303000ee80 comp rx=0 tx=0).stop 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 shutdown_connections 2024-09-21T03:26:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f30340b6680 0x7f30340b8b90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3044108cf0 0x7f304419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30441083a0 0x7f304419aa40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 --2- 172.21.15.83:0/669339228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3044102a60 0x7f304419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 >> 172.21.15.83:0/669339228 conn(0x7f3044079730 msgr2=0x7f304406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 shutdown_connections 2024-09-21T03:26:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:06.293+0000 7f304ad0e640 1 -- 172.21.15.83:0/669339228 wait complete. 2024-09-21T03:26:07.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: pgmap v615: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T03:26:07.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:26:07.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:07 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: pgmap v615: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T03:26:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:26:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:07 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: pgmap v615: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T03:26:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:07 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:08 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:08 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:08 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:08 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:08 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:08 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:08.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:26:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:09.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:09 smithi083 ceph-mon[31219]: pgmap v616: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-09-21T03:26:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:09 smithi193 ceph-mon[35224]: pgmap v616: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-09-21T03:26:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:09 smithi142 ceph-mon[34512]: pgmap v616: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-09-21T03:26:09.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 -- 172.21.15.83:0/2714463079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109570 msgr2=0x7feb90109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 --2- 172.21.15.83:0/2714463079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109570 0x7feb90109980 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feb80009900 tx=0x7feb8002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 -- 172.21.15.83:0/2714463079 shutdown_connections 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 --2- 172.21.15.83:0/2714463079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90109ec0 0x7feb901106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 --2- 172.21.15.83:0/2714463079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109570 0x7feb90109980 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 --2- 172.21.15.83:0/2714463079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 -- 172.21.15.83:0/2714463079 >> 172.21.15.83:0/2714463079 conn(0x7feb9007b8d0 msgr2=0x7feb9007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 -- 172.21.15.83:0/2714463079 shutdown_connections 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.677+0000 7feb97cfe640 1 -- 172.21.15.83:0/2714463079 wait complete. 2024-09-21T03:26:09.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 Processor -- start 2024-09-21T03:26:09.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 -- start start 2024-09-21T03:26:09.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90109570 0x7feb90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109ec0 0x7feb90075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb90113ae0 con 0x7feb90109ec0 2024-09-21T03:26:09.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb90113960 con 0x7feb90103c30 2024-09-21T03:26:09.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.678+0000 7feb97cfe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb90113c60 con 0x7feb90109570 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:37140/0 (socket says 172.21.15.83:37140) 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 -- 172.21.15.83:0/750490476 learned_addr learned my addr 172.21.15.83:0/750490476 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb9629b640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109ec0 0x7feb90075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 -- 172.21.15.83:0/750490476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90109570 msgr2=0x7feb90075710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90109570 0x7feb90075710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 -- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109ec0 msgr2=0x7feb90075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109ec0 0x7feb90075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.679+0000 7feb95a9a640 1 -- 172.21.15.83:0/750490476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90076470 con 0x7feb90103c30 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.680+0000 7feb95a9a640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9f0 tx=0x7feb8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.680+0000 7feb86ffd640 1 -- 172.21.15.83:0/750490476 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cd80 con 0x7feb90103c30 2024-09-21T03:26:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.680+0000 7feb86ffd640 1 -- 172.21.15.83:0/750490476 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c0108e0 con 0x7feb90103c30 2024-09-21T03:26:09.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.680+0000 7feb86ffd640 1 -- 172.21.15.83:0/750490476 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c018980 con 0x7feb90103c30 2024-09-21T03:26:09.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.680+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb901ace30 con 0x7feb90103c30 2024-09-21T03:26:09.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.681+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901ad290 con 0x7feb90103c30 2024-09-21T03:26:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.684+0000 7feb86ffd640 1 -- 172.21.15.83:0/750490476 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7feb8c010430 con 0x7feb90103c30 2024-09-21T03:26:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.684+0000 7feb9e272740 1 Processor -- start 2024-09-21T03:26:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 -- start start 2024-09-21T03:26:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc9623b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc9623bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dc961b1e50 con 0x55dc95e73390 2024-09-21T03:26:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dc961a1280 con 0x55dc95e6b1b0 2024-09-21T03:26:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.685+0000 7feb9e272740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc961a5090 con 0x55dc95e94070 2024-09-21T03:26:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39046/0 (socket says 172.21.15.83:39046) 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 -- 172.21.15.83:0/2908987770 learned_addr learned my addr 172.21.15.83:0/2908987770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 -- 172.21.15.83:0/2908987770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 msgr2=0x55dc9623b7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc9623b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 -- 172.21.15.83:0/2908987770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 msgr2=0x55dc9623bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc9623bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.686+0000 7feb9629b640 1 -- 172.21.15.83:0/2908987770 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc95eff650 con 0x55dc95e94070 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb9629b640 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb88009f90 tx=0x7feb880118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533afeaebf62878f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb86ffd640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 0x7feb7c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb7bfff640 1 -- 172.21.15.83:0/2908987770 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004b20 con 0x55dc95e94070 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb7bfff640 1 -- 172.21.15.83:0/2908987770 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb88004cc0 con 0x55dc95e94070 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb86ffd640 1 -- 172.21.15.83:0/750490476 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7feb8c0e7a50 con 0x7feb90103c30 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb7bfff640 1 -- 172.21.15.83:0/2908987770 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88005020 con 0x55dc95e94070 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.687+0000 7feb95a9a640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 0x7feb7c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb95a9a640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 0x7feb7c0b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55dc96239ef0 tx=0x7feb8c0a9fa0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 -- 172.21.15.83:0/2908987770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 msgr2=0x55dc9623b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb88009f90 tx=0x7feb880118d0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 -- 172.21.15.83:0/2908987770 shutdown_connections 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc9623bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc9623b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 --2- 172.21.15.83:0/2908987770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e94070 0x55dc9623b270 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 -- 172.21.15.83:0/2908987770 >> 172.21.15.83:0/2908987770 conn(0x55dc95e88460 msgr2=0x55dc96239160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 -- 172.21.15.83:0/2908987770 shutdown_connections 2024-09-21T03:26:09.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.688+0000 7feb9e272740 1 -- 172.21.15.83:0/2908987770 wait complete. 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.689+0000 7feb9e272740 1 -- start start 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.689+0000 7feb9e272740 1 Processor -- start 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc96246e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e94070 0x55dc96252d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dc96002700 con 0x55dc95e73390 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dc9613c110 con 0x55dc95e94070 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc96249a50 con 0x55dc95e6b1b0 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9e272740 10 client.0 _ll_register_callbacks cb 0x55dc96155990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39056/0 (socket says 172.21.15.83:39056) 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 -- 172.21.15.83:0/923300699 learned_addr learned my addr 172.21.15.83:0/923300699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e94070 0x55dc96252d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb95299640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc96246e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 -- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e94070 msgr2=0x55dc96252d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e94070 0x55dc96252d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 -- 172.21.15.83:0/923300699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 msgr2=0x55dc96246e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc96246e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb9629b640 1 -- 172.21.15.83:0/923300699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc96169c40 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.690+0000 7feb95299640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc96246e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb9629b640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb88009da0 tx=0x7feb88004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa21a9dd87d5d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88003fe0 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7feb88016040 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb88004cc0 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7feb88004180 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb9e272740 10 client.26542 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc96255700 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.691+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb88004cc0 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7feb88025040 con 0x55dc95e6b1b0 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 10 client.26542 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 20 client.26542 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e529f0 0x55dc96258c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 4 client.26542 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc961b3da0 con 0x55dc95e529f0 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb95299640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e529f0 0x55dc96258c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.692+0000 7feb797fa640 10 client.26542 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.693+0000 7feb95299640 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e529f0 0x55dc96258c90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28180f7c66e1dbea server_cookie=5000d14d0b583feb in_seq=0 out_seq=0 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.694+0000 7feb797fa640 1 -- 172.21.15.83:0/923300699 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc961b3da0 con 0x55dc95e529f0 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.694+0000 7feb797fa640 10 client.26542 handle_command_reply: tid=0 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.694+0000 7feb9e272740 1 client.26542 shutdown 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.694+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 msgr2=0x55dc962468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.694+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb88009da0 tx=0x7feb88004430 comp rx=0 tx=0).stop 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 shutdown_connections 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e529f0 0x55dc96258c90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e94070 0x55dc96252d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e73390 0x55dc96246e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 --2- 172.21.15.83:0/923300699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc962468c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 >> 172.21.15.83:0/923300699 conn(0x55dc95e88460 msgr2=0x55dc9623e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb797fa640 0 client.26542 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 shutdown_connections 2024-09-21T03:26:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.695+0000 7feb9e272740 1 -- 172.21.15.83:0/923300699 wait complete. 2024-09-21T03:26:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.696+0000 7feb9e272740 20 client.26542 trim_cache size 0 max 16384 2024-09-21T03:26:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.717+0000 7feb9e272740 1 Processor -- start 2024-09-21T03:26:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.717+0000 7feb9e272740 1 -- start start 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc96252cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9623e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b5d50 con 0x55dc95e94070 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b7890 con 0x55dc95e6b1b0 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9e272740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b6420 con 0x55dc95e73390 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9629b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc96252cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb95299640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb95299640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39070/0 (socket says 172.21.15.83:39070) 2024-09-21T03:26:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb95299640 1 -- 172.21.15.83:0/2899847027 learned_addr learned my addr 172.21.15.83:0/2899847027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.718+0000 7feb9629b640 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9623e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 -- 172.21.15.83:0/2899847027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 msgr2=0x55dc96252cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc96252cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 -- 172.21.15.83:0/2899847027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 msgr2=0x55dc9623e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9623e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 -- 172.21.15.83:0/2899847027 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc96234580 con 0x55dc95e73390 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb9629b640 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9623e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb95299640 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb80006010 tx=0x7feb80000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb562109ba09249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb7b7fe640 1 -- 172.21.15.83:0/2899847027 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80035040 con 0x55dc95e73390 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb7b7fe640 1 -- 172.21.15.83:0/2899847027 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb80047070 con 0x55dc95e73390 2024-09-21T03:26:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.719+0000 7feb7b7fe640 1 -- 172.21.15.83:0/2899847027 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80004ab0 con 0x55dc95e73390 2024-09-21T03:26:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.721+0000 7feb9e272740 1 -- 172.21.15.83:0/2899847027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 msgr2=0x55dc9623c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.721+0000 7feb9e272740 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb80006010 tx=0x7feb80000bc0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.726+0000 7feb9e272740 1 -- 172.21.15.83:0/2899847027 shutdown_connections 2024-09-21T03:26:09.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.726+0000 7feb9e272740 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9623e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.726+0000 7feb9e272740 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e73390 0x55dc9623c9b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.726+0000 7feb9e272740 1 --2- 172.21.15.83:0/2899847027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e6b1b0 0x55dc96252cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.726+0000 7feb9e272740 1 -- 172.21.15.83:0/2899847027 >> 172.21.15.83:0/2899847027 conn(0x55dc95e88460 msgr2=0x55dc9623c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.736+0000 7feb9e272740 1 -- 172.21.15.83:0/2899847027 shutdown_connections 2024-09-21T03:26:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.740+0000 7feb9e272740 1 -- 172.21.15.83:0/2899847027 wait complete. 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.740+0000 7feb9e272740 1 -- start start 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.740+0000 7feb9e272740 1 Processor -- start 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e73390 0x55dc9625bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9625c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b71c0 con 0x55dc95e94070 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b6af0 con 0x55dc95e73390 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b3d50 con 0x55dc95e6b1b0 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9629b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:39076/0 (socket says 172.21.15.83:39076) 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9629b640 1 -- 172.21.15.83:0/1757853633 learned_addr learned my addr 172.21.15.83:0/1757853633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.741+0000 7feb9e272740 10 client.0 _ll_register_callbacks cb 0x7feb88009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:09.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.743+0000 7feb95299640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e73390 0x55dc9625bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9625c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 -- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e73390 msgr2=0x55dc9625bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e73390 0x55dc9625bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 -- 172.21.15.83:0/1757853633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 msgr2=0x55dc9625c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9625c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.745+0000 7feb9629b640 1 -- 172.21.15.83:0/1757853633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9625c920 con 0x55dc95e6b1b0 2024-09-21T03:26:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.750+0000 7feb9629b640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb88009270 tx=0x7feb88004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd12cd9ac6a88ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.751+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.752+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004c70 con 0x55dc95e6b1b0 2024-09-21T03:26:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.752+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb80047070 con 0x55dc95e6b1b0 2024-09-21T03:26:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.752+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8485+0+0 (secure 0 0 0) 0x7feb8800f4e0 con 0x55dc95e6b1b0 2024-09-21T03:26:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.753+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7feb8800f6e0 con 0x55dc95e6b1b0 2024-09-21T03:26:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.756+0000 7feb9e272740 10 client.26548 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.756+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc962b9670 con 0x55dc95e6b1b0 2024-09-21T03:26:09.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.756+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8800fd30 con 0x55dc95e6b1b0 2024-09-21T03:26:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7feb88065020 con 0x55dc95e6b1b0 2024-09-21T03:26:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 10 client.26548 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 20 client.26548 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e27da0 0x55dc962bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 4 client.26548 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:26:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.762+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7feb80047070 con 0x55dc95e27da0 2024-09-21T03:26:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.766+0000 7feb95299640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e27da0 0x55dc962bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.769+0000 7feb95299640 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e27da0 0x55dc962bbd00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad2e2f06b187140 server_cookie=d93eea1ffe017c94 in_seq=0 out_seq=0 2024-09-21T03:26:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.770+0000 7feb797fa640 10 client.26548 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.770+0000 7feb797fa640 1 -- 172.21.15.83:0/1757853633 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x7feb80047070 con 0x55dc95e27da0 2024-09-21T03:26:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.770+0000 7feb797fa640 10 client.26548 handle_command_reply: tid=0 2024-09-21T03:26:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.770+0000 7feb9e272740 1 client.26548 shutdown 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.778+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 msgr2=0x55dc9625e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.778+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb88009270 tx=0x7feb88004430 comp rx=0 tx=0).stop 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 shutdown_connections 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55dc95e27da0 0x55dc962bbd00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc95e94070 0x55dc9625c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dc95e73390 0x55dc9625bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 --2- 172.21.15.83:0/1757853633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc95e6b1b0 0x55dc9625e920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 >> 172.21.15.83:0/1757853633 conn(0x55dc95e88460 msgr2=0x55dc96247d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb797fa640 0 client.26548 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 shutdown_connections 2024-09-21T03:26:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.779+0000 7feb9e272740 1 -- 172.21.15.83:0/1757853633 wait complete. 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb9e272740 20 client.26548 trim_cache size 0 max 16384 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:26:09.782 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e5cdcd58-df20-47cb-bced-27da9f50e8be", 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.784 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:26:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.786 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.788 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:26:09.789 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:26:09.790 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 msgr2=0x7feb7c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 0x7feb7c0b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55dc96239ef0 tx=0x7feb8c0a9fa0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 msgr2=0x7feb90077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9f0 tx=0x7feb8c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 shutdown_connections 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7feb7c0b6700 0x7feb7c0b8c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb90109ec0 0x7feb90075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90109570 0x7feb90075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 --2- 172.21.15.83:0/750490476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb90103c30 0x7feb90077120 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.780+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 >> 172.21.15.83:0/750490476 conn(0x7feb9007b8d0 msgr2=0x7feb90105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.781+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 shutdown_connections 2024-09-21T03:26:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:09.781+0000 7feb97cfe640 1 -- 172.21.15.83:0/750490476 wait complete. 2024-09-21T03:26:10.686 INFO:tasks.cephfs.filesystem:scrub status for tag:e5cdcd58-df20-47cb-bced-27da9f50e8be - {'path': '/', 'tag': 'e5cdcd58-df20-47cb-bced-27da9f50e8be', 'options': 'recursive,force'} 2024-09-21T03:26:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 530.0 s, 1795.6 tps, lat 56.705 ms stddev 81.757 2024-09-21T03:26:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:11 smithi193 ceph-mon[35224]: pgmap v617: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 35 MiB/s wr, 1.97k op/s 2024-09-21T03:26:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:11 smithi083 ceph-mon[31219]: pgmap v617: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 35 MiB/s wr, 1.97k op/s 2024-09-21T03:26:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:11 smithi142 ceph-mon[34512]: pgmap v617: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 35 MiB/s wr, 1.97k op/s 2024-09-21T03:26:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:13 smithi193 ceph-mon[35224]: pgmap v618: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 35 MiB/s wr, 1.99k op/s 2024-09-21T03:26:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:13 smithi083 ceph-mon[31219]: pgmap v618: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 35 MiB/s wr, 1.99k op/s 2024-09-21T03:26:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:13 smithi142 ceph-mon[34512]: pgmap v618: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 35 MiB/s wr, 1.99k op/s 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:26:14.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:14 smithi193 ceph-mon[35224]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:26:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:26:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:26:14.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:26:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:26:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:26:14.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:14 smithi083 ceph-mon[31219]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:26:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:26:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:26:14.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:26:14.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:26:14.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:26:14.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:14 smithi142 ceph-mon[34512]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:26:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:15 smithi193 ceph-mon[35224]: pgmap v619: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T03:26:15.578 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:15 smithi142 ceph-mon[34512]: pgmap v619: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T03:26:15.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:15 smithi083 ceph-mon[31219]: pgmap v619: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.35k op/s 2024-09-21T03:26:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 535.0 s, 1825.8 tps, lat 54.810 ms stddev 73.497 2024-09-21T03:26:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:17 smithi193 ceph-mon[35224]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T03:26:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:17 smithi083 ceph-mon[31219]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T03:26:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:17 smithi142 ceph-mon[34512]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-21T03:26:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:19 smithi193 ceph-mon[35224]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 947 op/s 2024-09-21T03:26:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:19 smithi083 ceph-mon[31219]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 947 op/s 2024-09-21T03:26:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:19 smithi142 ceph-mon[34512]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 947 op/s 2024-09-21T03:26:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 540.0 s, 1576.8 tps, lat 62.489 ms stddev 68.631 2024-09-21T03:26:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:21 smithi193 ceph-mon[35224]: pgmap v623: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-21T03:26:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:21 smithi083 ceph-mon[31219]: pgmap v623: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-21T03:26:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:21 smithi142 ceph-mon[34512]: pgmap v623: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-21T03:26:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:23 smithi083 ceph-mon[31219]: pgmap v624: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:26:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:23 smithi142 ceph-mon[34512]: pgmap v624: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:26:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:23 smithi193 ceph-mon[35224]: pgmap v624: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:26:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:25 smithi083 ceph-mon[31219]: pgmap v625: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-09-21T03:26:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:25 smithi142 ceph-mon[34512]: pgmap v625: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-09-21T03:26:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:25 smithi193 ceph-mon[35224]: pgmap v625: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 545.0 s, 1456.0 tps, lat 69.445 ms stddev 77.756 2024-09-21T03:26:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:27 smithi083 ceph-mon[31219]: pgmap v626: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T03:26:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:27 smithi142 ceph-mon[34512]: pgmap v626: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T03:26:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:27 smithi193 ceph-mon[35224]: pgmap v626: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T03:26:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:29 smithi083 ceph-mon[31219]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:26:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:29 smithi142 ceph-mon[34512]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:26:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:29 smithi193 ceph-mon[35224]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:26:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 550.0 s, 1956.4 tps, lat 51.312 ms stddev 56.319 2024-09-21T03:26:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:31 smithi083 ceph-mon[31219]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:26:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:31 smithi142 ceph-mon[34512]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:26:31.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:31 smithi193 ceph-mon[35224]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:26:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:33 smithi083 ceph-mon[31219]: pgmap v629: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2024-09-21T03:26:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:33 smithi142 ceph-mon[34512]: pgmap v629: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2024-09-21T03:26:33.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:33 smithi193 ceph-mon[35224]: pgmap v629: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2024-09-21T03:26:35.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:35 smithi083 ceph-mon[31219]: pgmap v630: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-09-21T03:26:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:35 smithi142 ceph-mon[34512]: pgmap v630: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-09-21T03:26:35.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:35 smithi193 ceph-mon[35224]: pgmap v630: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 686 op/s 2024-09-21T03:26:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 555.0 s, 1912.0 tps, lat 52.381 ms stddev 55.839 2024-09-21T03:26:37.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:37 smithi083 ceph-mon[31219]: pgmap v631: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-21T03:26:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:37 smithi142 ceph-mon[34512]: pgmap v631: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-21T03:26:37.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:37 smithi193 ceph-mon[35224]: pgmap v631: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-21T03:26:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:39 smithi083 ceph-mon[31219]: pgmap v632: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T03:26:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:39 smithi142 ceph-mon[34512]: pgmap v632: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T03:26:39.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:39 smithi193 ceph-mon[35224]: pgmap v632: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T03:26:40.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:26:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 560.0 s, 2109.8 tps, lat 47.185 ms stddev 52.069 2024-09-21T03:26:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:41.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:41 smithi083 ceph-mon[31219]: pgmap v633: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 -- 172.21.15.83:0/1031222887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19681031c0 msgr2=0x7f19681035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 --2- 172.21.15.83:0/1031222887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19681031c0 0x7f19681035d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1964009900 tx=0x7f196402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 -- 172.21.15.83:0/1031222887 shutdown_connections 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 --2- 172.21.15.83:0/1031222887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1968105ae0 0x7f196810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 --2- 172.21.15.83:0/1031222887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 --2- 172.21.15.83:0/1031222887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19681031c0 0x7f19681035d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 -- 172.21.15.83:0/1031222887 >> 172.21.15.83:0/1031222887 conn(0x7f1968077c60 msgr2=0x7f19680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 -- 172.21.15.83:0/1031222887 shutdown_connections 2024-09-21T03:26:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.620+0000 7f1970424640 1 -- 172.21.15.83:0/1031222887 wait complete. 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.621+0000 7f1970424640 1 Processor -- start 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.621+0000 7f1970424640 1 -- start start 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19681031c0 0x7f1968199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 0x7f196819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53572/0 (socket says 172.21.15.83:53572) 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f196d9bf640 1 -- 172.21.15.83:0/1583162463 learned_addr learned my addr 172.21.15.83:0/1583162463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112980 con 0x7f1968105ae0 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112800 con 0x7f1968105190 2024-09-21T03:26:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.622+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112b00 con 0x7f19681031c0 2024-09-21T03:26:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 0x7f196819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196d9bf640 1 -- 172.21.15.83:0/1583162463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19681031c0 msgr2=0x7f1968199330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:26:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19681031c0 0x7f1968199330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196d9bf640 1 -- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 msgr2=0x7f196819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 0x7f196819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196d9bf640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196819f2a0 con 0x7f1968105190 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.623+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 0x7f196819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.624+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f195800e9c0 tx=0x7f195800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.624+0000 7f195f7fe640 1 -- 172.21.15.83:0/1583162463 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800cd80 con 0x7f1968105190 2024-09-21T03:26:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.624+0000 7f195f7fe640 1 -- 172.21.15.83:0/1583162463 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19580054d0 con 0x7f1968105190 2024-09-21T03:26:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.625+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196819f590 con 0x7f1968105190 2024-09-21T03:26:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.625+0000 7f195f7fe640 1 -- 172.21.15.83:0/1583162463 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958010430 con 0x7f1968105190 2024-09-21T03:26:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.625+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a5cf0 con 0x7f1968105190 2024-09-21T03:26:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 Processor -- start 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 -- start start 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c92412be50 con 0x55c923ded390 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c92411b280 con 0x55c923e0e070 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.628+0000 7f1976998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c92411f090 con 0x55c923de51b0 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f195f7fe640 1 -- 172.21.15.83:0/1583162463 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f19580105d0 con 0x7f1968105190 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e9c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51188/0 (socket says 172.21.15.83:51188) 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 -- 172.21.15.83:0/3309379092 learned_addr learned my addr 172.21.15.83:0/3309379092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196d9bf640 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 -- 172.21.15.83:0/3309379092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 msgr2=0x55c9241b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 -- 172.21.15.83:0/3309379092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 msgr2=0x55c9241b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 -- 172.21.15.83:0/3309379092 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c923e79650 con 0x55c923de51b0 2024-09-21T03:26:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196d9bf640 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.629+0000 7f196e1c0640 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1964006010 tx=0x7f1964005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafe275fbba0a0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.630+0000 7f195f7fe640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 0x7f19540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.630+0000 7f195f7fe640 1 -- 172.21.15.83:0/1583162463 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f19580e7c90 con 0x7f1968105190 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.630+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 0x7f19540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.630+0000 7f195cd9a640 1 -- 172.21.15.83:0/3309379092 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964040070 con 0x55c923de51b0 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.631+0000 7f195cd9a640 1 -- 172.21.15.83:0/3309379092 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1964032e20 con 0x55c923de51b0 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.631+0000 7f195cd9a640 1 -- 172.21.15.83:0/3309379092 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964035040 con 0x55c923de51b0 2024-09-21T03:26:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- 172.21.15.83:0/3309379092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 msgr2=0x55c9241c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 0x7f19540b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1960005fd0 tx=0x7f1960009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1964006010 tx=0x7f1964005e60 comp rx=0 tx=0).stop 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- 172.21.15.83:0/3309379092 shutdown_connections 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c15e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 --2- 172.21.15.83:0/3309379092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- 172.21.15.83:0/3309379092 >> 172.21.15.83:0/3309379092 conn(0x55c923e02460 msgr2=0x55c9241b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- 172.21.15.83:0/3309379092 shutdown_connections 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- 172.21.15.83:0/3309379092 wait complete. 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 -- start start 2024-09-21T03:26:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.633+0000 7f1976998740 1 Processor -- start 2024-09-21T03:26:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923de51b0 0x55c9241bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 0x55c9241bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c923f7c700 con 0x55c923de51b0 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c9240b6110 con 0x55c923ded390 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9241b6010 con 0x55c923e0e070 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f1976998740 10 client.0 _ll_register_callbacks cb 0x55c9240cf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53580/0 (socket says 172.21.15.83:53580) 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 -- 172.21.15.83:0/1709678938 learned_addr learned my addr 172.21.15.83:0/1709678938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 0x55c9241bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923de51b0 0x55c9241bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 -- 172.21.15.83:0/1709678938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 msgr2=0x55c9241bf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 0x55c9241bf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 -- 172.21.15.83:0/1709678938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923de51b0 msgr2=0x55c9241bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923de51b0 0x55c9241bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196d9bf640 1 -- 172.21.15.83:0/1709678938 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9240e3c40 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.634+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 0x55c9241bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19580ad830 tx=0x7f19580b7480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1abd0c02787c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580aee60 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f19580b23b0 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1964032e20 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f19580a99d0 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1976998740 10 client.26588 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c9241d7eb0 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.635+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1964032e20 con 0x55c923ded390 2024-09-21T03:26:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f1958097020 con 0x55c923ded390 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 10 client.26588 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 20 client.26588 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923dcc9f0 0x55c9241db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 4 client.26588 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c92412dda0 con 0x55c923dcc9f0 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923dcc9f0 0x55c9241db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f1951ffb640 10 client.26588 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.636+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923dcc9f0 0x55c9241db3c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ce36159b4b5e15 server_cookie=d03f42431134d677 in_seq=0 out_seq=0 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.638+0000 7f1951ffb640 1 -- 172.21.15.83:0/1709678938 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c92412dda0 con 0x55c923dcc9f0 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.638+0000 7f1951ffb640 10 client.26588 handle_command_reply: tid=0 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.638+0000 7f1976998740 1 client.26588 shutdown 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 msgr2=0x55c9241bf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19580ad830 tx=0x7f19580b7480 comp rx=0 tx=0).stop 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 shutdown_connections 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923dcc9f0 0x55c9241db3c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923e0e070 0x55c9241bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241bf2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 --2- 172.21.15.83:0/1709678938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923de51b0 0x55c9241bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 >> 172.21.15.83:0/1709678938 conn(0x55c923e02460 msgr2=0x55c9241b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1951ffb640 0 client.26588 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 shutdown_connections 2024-09-21T03:26:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.639+0000 7f1976998740 1 -- 172.21.15.83:0/1709678938 wait complete. 2024-09-21T03:26:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.640+0000 7f1976998740 20 client.26588 trim_cache size 0 max 16384 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 Processor -- start 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 -- start start 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923e0e070 0x55c9241b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19681ad3e0 con 0x55c923e0e070 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19681aef20 con 0x55c923ded390 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f1976998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19681adab0 con 0x55c923de51b0 2024-09-21T03:26:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f196e1c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53588/0 (socket says 172.21.15.83:53588) 2024-09-21T03:26:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f196d9bf640 1 -- 172.21.15.83:0/566764550 learned_addr learned my addr 172.21.15.83:0/566764550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.662+0000 7f196e9c1640 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923e0e070 0x55c9241b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 -- 172.21.15.83:0/566764550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 msgr2=0x55c9241b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 -- 172.21.15.83:0/566764550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923e0e070 msgr2=0x55c9241b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923e0e070 0x55c9241b69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 -- 172.21.15.83:0/566764550 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9241bd2b0 con 0x55c923de51b0 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196d9bf640 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f196e1c0640 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c0c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f196402a930 tx=0x7f196403c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63967752488f2219 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f1953fff640 1 -- 172.21.15.83:0/566764550 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964035040 con 0x55c923de51b0 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f1953fff640 1 -- 172.21.15.83:0/566764550 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640044d0 con 0x55c923de51b0 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.663+0000 7f1953fff640 1 -- 172.21.15.83:0/566764550 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964040040 con 0x55c923de51b0 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 -- 172.21.15.83:0/566764550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 msgr2=0x55c9241c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c0c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f196402a930 tx=0x7f196403c040 comp rx=0 tx=0).stop 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 -- 172.21.15.83:0/566764550 shutdown_connections 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923e0e070 0x55c9241b69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923ded390 0x55c9241b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 --2- 172.21.15.83:0/566764550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241c0c90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 -- 172.21.15.83:0/566764550 >> 172.21.15.83:0/566764550 conn(0x55c923e02460 msgr2=0x55c9241b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 -- 172.21.15.83:0/566764550 shutdown_connections 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.664+0000 7f1976998740 1 -- 172.21.15.83:0/566764550 wait complete. 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 -- start start 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 Processor -- start 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19681ae850 con 0x55c923ded390 2024-09-21T03:26:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19681ae180 con 0x55c923e0e070 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19681ab3e0 con 0x55c923de51b0 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f1976998740 10 client.0 _ll_register_callbacks cb 0x7f195800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196d9bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37290/0 (socket says 172.21.15.83:37290) 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196d9bf640 1 -- 172.21.15.83:0/1788850682 learned_addr learned my addr 172.21.15.83:0/1788850682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196e1c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.665+0000 7f196d9bf640 1 -- 172.21.15.83:0/1788850682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 msgr2=0x55c9241cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196d9bf640 1 -- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 msgr2=0x55c9241cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196d9bf640 1 -- 172.21.15.83:0/1788850682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c924244d80 con 0x55c923ded390 2024-09-21T03:26:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196e9c1640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.666+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.667+0000 7f196d9bf640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f195800cfd0 tx=0x7f19580d0d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5d6c338d605024 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.667+0000 7f1951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580c7020 con 0x55c923ded390 2024-09-21T03:26:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f19580051b0 con 0x55c923ded390 2024-09-21T03:26:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19640044d0 con 0x55c923ded390 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f19580aee60 con 0x55c923ded390 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1976998740 10 client.17055 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c924245fe0 con 0x55c923ded390 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.668+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640044d0 con 0x55c923ded390 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f19580cd020 con 0x55c923ded390 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 10 client.17055 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 20 client.17055 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923da1da0 0x55c9242487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 4 client.17055 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:26:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c924129850 con 0x55c923da1da0 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923da1da0 0x55c9242487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.669+0000 7f1951ffb640 10 client.17055 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f196e1c0640 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923da1da0 0x55c9242487a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef46e5f67252449 server_cookie=84f9d461c7e79ca0 in_seq=0 out_seq=0 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1951ffb640 1 -- 172.21.15.83:0/1788850682 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55c924129850 con 0x55c923da1da0 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1951ffb640 10 client.17055 handle_command_reply: tid=0 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 client.17055 shutdown 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 msgr2=0x55c9241cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f195800cfd0 tx=0x7f19580d0d80 comp rx=0 tx=0).stop 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 shutdown_connections 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55c923da1da0 0x55c9242487a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1951ffb640 0 client.17055 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c923e0e070 0x55c9241cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923ded390 0x55c9241cb9a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 --2- 172.21.15.83:0/1788850682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c923de51b0 0x55c9241cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 >> 172.21.15.83:0/1788850682 conn(0x55c923e02460 msgr2=0x55c9241c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 shutdown_connections 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 1 -- 172.21.15.83:0/1788850682 wait complete. 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.670+0000 7f1976998740 20 client.17055 trim_cache size 0 max 16384 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:26:41.676 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.678 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:26:41.679 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.682 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:26:41.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:26:41.684 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:26:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 msgr2=0x7f19540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 0x7f19540b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1960005fd0 tx=0x7f1960009500 comp rx=0 tx=0).stop 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 msgr2=0x7f19681998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f195800e9c0 tx=0x7f195800ee80 comp rx=0 tx=0).stop 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 shutdown_connections 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f19540b6680 0x7f19540b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1968105ae0 0x7f196819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968105190 0x7f19681998d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 --2- 172.21.15.83:0/1583162463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19681031c0 0x7f1968199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 >> 172.21.15.83:0/1583162463 conn(0x7f1968077c60 msgr2=0x7f19681070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 shutdown_connections 2024-09-21T03:26:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:41.671+0000 7f1970424640 1 -- 172.21.15.83:0/1583162463 wait complete. 2024-09-21T03:26:41.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:41 smithi142 ceph-mon[34512]: pgmap v633: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-21T03:26:41.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:41 smithi193 ceph-mon[35224]: pgmap v633: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-21T03:26:42.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:26:42.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:42.808 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 -- 172.21.15.83:0/731322699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 msgr2=0x7f78a010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 --2- 172.21.15.83:0/731322699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a010f560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7898009ca0 tx=0x7f789802a8d0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 -- 172.21.15.83:0/731322699 shutdown_connections 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 --2- 172.21.15.83:0/731322699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a010f560 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 --2- 172.21.15.83:0/731322699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 --2- 172.21.15.83:0/731322699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 0x7f78a0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 -- 172.21.15.83:0/731322699 >> 172.21.15.83:0/731322699 conn(0x7f78a00797e0 msgr2=0x7f78a0079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 -- 172.21.15.83:0/731322699 shutdown_connections 2024-09-21T03:26:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.412+0000 7f78a7034640 1 -- 172.21.15.83:0/731322699 wait complete. 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 Processor -- start 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 -- start start 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 0x7f78a0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01129a0 con 0x7f78a01083e0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112820 con 0x7f78a0102aa0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a7034640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112b20 con 0x7f78a0108d30 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f789ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f789ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37318/0 (socket says 172.21.15.83:37318) 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f789ffff640 1 -- 172.21.15.83:0/2876812770 learned_addr learned my addr 172.21.15.83:0/2876812770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a55d1640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.413+0000 7f78a4dd0640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 0x7f78a0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 -- 172.21.15.83:0/2876812770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 msgr2=0x7f78a019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 -- 172.21.15.83:0/2876812770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 msgr2=0x7f78a0199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 0x7f78a0199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 -- 172.21.15.83:0/2876812770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a019f280 con 0x7f78a01083e0 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f78a55d1640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789ffff640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e760 tx=0x7f788c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c01b070 con 0x7f78a01083e0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788c007c50 con 0x7f78a01083e0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00cd50 con 0x7f78a01083e0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a019f570 con 0x7f78a01083e0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.414+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a006b820 con 0x7f78a01083e0 2024-09-21T03:26:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.416+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a0103d20 con 0x7f78a01083e0 2024-09-21T03:26:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.417+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f788c01e070 con 0x7f78a01083e0 2024-09-21T03:26:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.418+0000 7f789dffb640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 0x7f78900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.419+0000 7f78a4dd0640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 0x7f78900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.419+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f788c0e6a00 con 0x7f78a01083e0 2024-09-21T03:26:43.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.419+0000 7f78a4dd0640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 0x7f78900b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7894009bd0 tx=0x7f7894005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.425+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f788c0b1320 con 0x7f78a01083e0 2024-09-21T03:26:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:43 smithi083 ceph-mon[31219]: pgmap v634: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:26:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:43 smithi142 ceph-mon[34512]: pgmap v634: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:26:43.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.773+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78a006c2a0 con 0x7f78a01083e0 2024-09-21T03:26:43.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.774+0000 7f789dffb640 1 -- 172.21.15.83:0/2876812770 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11982 (secure 0 0 0) 0x7f788c0b6260 con 0x7f78a01083e0 2024-09-21T03:26:43.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:26:43.776 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"btime":"2024-09-21T03:23:29:976813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:23:29.048942+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:26:43.776 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 44 2024-09-21T03:26:43.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.776+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 msgr2=0x7f78900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:43.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.776+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 0x7f78900b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7894009bd0 tx=0x7f7894005f90 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.776+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 msgr2=0x7f78a01998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e760 tx=0x7f788c00ec20 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 shutdown_connections 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f78900b6700 0x7f78900b8c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a0108d30 0x7f78a019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a01083e0 0x7f78a01998b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 --2- 172.21.15.83:0/2876812770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78a0102aa0 0x7f78a0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 >> 172.21.15.83:0/2876812770 conn(0x7f78a00797e0 msgr2=0x7f78a01043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 shutdown_connections 2024-09-21T03:26:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:43.777+0000 7f78a7034640 1 -- 172.21.15.83:0/2876812770 wait complete. 2024-09-21T03:26:43.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:43 smithi193 ceph-mon[35224]: pgmap v634: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 654 op/s 2024-09-21T03:26:44.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:26:44.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:44 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2876812770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:44.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:44 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2876812770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:44.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:44 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2876812770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.480+0000 7f881df2f640 1 -- 172.21.15.83:0/2072289324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 msgr2=0x7f881810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.480+0000 7f881df2f640 1 --2- 172.21.15.83:0/2072289324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f881810f520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8810009ca0 tx=0x7f881002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.481+0000 7f881df2f640 1 -- 172.21.15.83:0/2072289324 shutdown_connections 2024-09-21T03:26:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.481+0000 7f881df2f640 1 --2- 172.21.15.83:0/2072289324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f881810f520 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.481+0000 7f881df2f640 1 --2- 172.21.15.83:0/2072289324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 0x7f88181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.481+0000 7f881df2f640 1 --2- 172.21.15.83:0/2072289324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f8818102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.481+0000 7f881df2f640 1 -- 172.21.15.83:0/2072289324 >> 172.21.15.83:0/2072289324 conn(0x7f8818079730 msgr2=0x7f8818079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.482+0000 7f881df2f640 1 -- 172.21.15.83:0/2072289324 shutdown_connections 2024-09-21T03:26:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.482+0000 7f881df2f640 1 -- 172.21.15.83:0/2072289324 wait complete. 2024-09-21T03:26:45.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 Processor -- start 2024-09-21T03:26:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 -- start start 2024-09-21T03:26:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 0x7f881810c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f88181ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8818111850 con 0x7f8818108cf0 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88181116d0 con 0x7f8818102a60 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.483+0000 7f881df2f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88181119d0 con 0x7f88181083a0 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53642/0 (socket says 172.21.15.83:53642) 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 -- 172.21.15.83:0/1736703685 learned_addr learned my addr 172.21.15.83:0/1736703685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f8816ffd640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 0x7f881810c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f8817fff640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f88181ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 -- 172.21.15.83:0/1736703685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 msgr2=0x7f881810c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 0x7f881810c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 -- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 msgr2=0x7f88181ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f88181ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 -- 172.21.15.83:0/1736703685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881810c9e0 con 0x7f8818102a60 2024-09-21T03:26:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f8817fff640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f88181ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.484+0000 7f88177fe640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9f0 tx=0x7f880800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.485+0000 7f8814ff9640 1 -- 172.21.15.83:0/1736703685 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800cd80 con 0x7f8818102a60 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.485+0000 7f8814ff9640 1 -- 172.21.15.83:0/1736703685 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88080108e0 con 0x7f8818102a60 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.485+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88181acc40 con 0x7f8818102a60 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.485+0000 7f8814ff9640 1 -- 172.21.15.83:0/1736703685 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808018980 con 0x7f8818102a60 2024-09-21T03:26:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.486+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181ad180 con 0x7f8818102a60 2024-09-21T03:26:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.487+0000 7f8814ff9640 1 -- 172.21.15.83:0/1736703685 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8808010430 con 0x7f8818102a60 2024-09-21T03:26:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.488+0000 7f88244a3740 1 Processor -- start 2024-09-21T03:26:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.488+0000 7f88244a3740 1 -- start start 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d139b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 0x5559d139b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5559d1311ea0 con 0x5559d0fcb1b0 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559d13012d0 con 0x5559d0fd3390 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f88244a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5559d13050e0 con 0x5559d0ff4070 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53658/0 (socket says 172.21.15.83:53658) 2024-09-21T03:26:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.489+0000 7f8816ffd640 1 -- 172.21.15.83:0/4055619470 learned_addr learned my addr 172.21.15.83:0/4055619470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f88177fe640 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 0x5559d139b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 -- 172.21.15.83:0/4055619470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 msgr2=0x5559d139b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d139b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 -- 172.21.15.83:0/4055619470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 msgr2=0x5559d139b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 0x5559d139b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 -- 172.21.15.83:0/4055619470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d105f650 con 0x5559d0fd3390 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f88177fe640 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 0x5559d139b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8814ff9640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 0x7f88000b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.490+0000 7f8816ffd640 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f880c005fd0 tx=0x7f880c009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74825879e0f5c105 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.491+0000 7f8814ff9640 1 -- 172.21.15.83:0/1736703685 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f88080e7ae0 con 0x7f8818102a60 2024-09-21T03:26:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.491+0000 7f8817fff640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 0x7f88000b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.491+0000 7f87fdffb640 1 -- 172.21.15.83:0/4055619470 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c016070 con 0x5559d0fd3390 2024-09-21T03:26:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.491+0000 7f87fdffb640 1 -- 172.21.15.83:0/4055619470 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880c0054d0 con 0x5559d0fd3390 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.492+0000 7f87fdffb640 1 -- 172.21.15.83:0/4055619470 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c004dd0 con 0x5559d0fd3390 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.492+0000 7f88244a3740 1 -- 172.21.15.83:0/4055619470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 msgr2=0x5559d139bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.492+0000 7f8817fff640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 0x7f88000b8c40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f88100094e0 tx=0x7f8810005da0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.492+0000 7f88244a3740 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f880c005fd0 tx=0x7f880c009f20 comp rx=0 tx=0).stop 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 -- 172.21.15.83:0/4055619470 shutdown_connections 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fd3390 0x5559d139bd40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fcb1b0 0x5559d139b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 --2- 172.21.15.83:0/4055619470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d139b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 -- 172.21.15.83:0/4055619470 >> 172.21.15.83:0/4055619470 conn(0x5559d0fe8460 msgr2=0x5559d13991b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 -- 172.21.15.83:0/4055619470 shutdown_connections 2024-09-21T03:26:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 -- 172.21.15.83:0/4055619470 wait complete. 2024-09-21T03:26:45.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 -- start start 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.493+0000 7f88244a3740 1 Processor -- start 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5559d1162700 con 0x5559d0fd3390 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559d129c160 con 0x5559d0fcb1b0 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5559d13a9b40 con 0x5559d0ff4070 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88244a3740 10 client.0 _ll_register_callbacks cb 0x5559d12b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37368/0 (socket says 172.21.15.83:37368) 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 -- 172.21.15.83:0/563791839 learned_addr learned my addr 172.21.15.83:0/563791839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f88177fe640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 -- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 msgr2=0x5559d13a7340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13a7340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 -- 172.21.15.83:0/563791839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 msgr2=0x5559d13a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.494+0000 7f8816ffd640 1 -- 172.21.15.83:0/563791839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d12c9c90 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f8816ffd640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f880c00a010 tx=0x7f880c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9ce6dccd7f2b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f87f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c028060 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880c0054d0 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f880c004d60 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f880c016040 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f88244a3740 10 client.17070 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.495+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559d13b5700 con 0x5559d0fd3390 2024-09-21T03:26:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880c0054d0 con 0x5559d0fd3390 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f880c00f040 con 0x5559d0fd3390 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 10 client.17070 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 10 client.17070 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 20 client.17070 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0fb29f0 0x5559d13b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 4 client.17070 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559d1313df0 con 0x5559d0fb29f0 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.496+0000 7f88177fe640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0fb29f0 0x5559d13b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.497+0000 7f87f6ffd640 10 client.17070 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:26:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.497+0000 7f88177fe640 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0fb29f0 0x5559d13b8ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3beee712937853a server_cookie=641ac73b23b276fb in_seq=0 out_seq=0 2024-09-21T03:26:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f87f6ffd640 1 -- 172.21.15.83:0/563791839 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559d1313df0 con 0x5559d0fb29f0 2024-09-21T03:26:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f87f6ffd640 10 client.17070 handle_command_reply: tid=0 2024-09-21T03:26:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 client.17070 shutdown 2024-09-21T03:26:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 msgr2=0x5559d13a6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f880c00a010 tx=0x7f880c0048f0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 shutdown_connections 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0fb29f0 0x5559d13b8ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f87f6ffd640 0 client.17070 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13a6e00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 --2- 172.21.15.83:0/563791839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 >> 172.21.15.83:0/563791839 conn(0x5559d0fe8460 msgr2=0x5559d139e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 shutdown_connections 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 1 -- 172.21.15.83:0/563791839 wait complete. 2024-09-21T03:26:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.498+0000 7f88244a3740 20 client.17070 trim_cache size 0 max 16384 2024-09-21T03:26:45.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.516+0000 7f88244a3740 1 Processor -- start 2024-09-21T03:26:45.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.516+0000 7f88244a3740 1 -- start start 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.516+0000 7f88244a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.516+0000 7f88244a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d139c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88244a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0ff4070 0x5559d13a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88244a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b5cc0 con 0x5559d0fd3390 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88244a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b7800 con 0x5559d0ff4070 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88244a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b6390 con 0x5559d0fcb1b0 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d139c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51314/0 (socket says 172.21.15.83:51314) 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 -- 172.21.15.83:0/1245649595 learned_addr learned my addr 172.21.15.83:0/1245649595 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d139c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37378/0 (socket says 172.21.15.83:37378) 2024-09-21T03:26:45.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f8817fff640 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0ff4070 0x5559d13a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 -- 172.21.15.83:0/1245649595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0ff4070 msgr2=0x5559d13a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0ff4070 0x5559d13a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 -- 172.21.15.83:0/1245649595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 msgr2=0x5559d139c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d139c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 -- 172.21.15.83:0/1245649595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d139e120 con 0x5559d0fcb1b0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f88177fe640 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f880800c6a0 tx=0x7f88080ea030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24819eeb61cad9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.517+0000 7f87fd7fa640 1 -- 172.21.15.83:0/1245649595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808010720 con 0x5559d0fcb1b0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f87fd7fa640 1 -- 172.21.15.83:0/1245649595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88080ad490 con 0x5559d0fcb1b0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f87fd7fa640 1 -- 172.21.15.83:0/1245649595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080e5500 con 0x5559d0fcb1b0 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 -- 172.21.15.83:0/1245649595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 msgr2=0x5559d13a87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f880800c6a0 tx=0x7f88080ea030 comp rx=0 tx=0).stop 2024-09-21T03:26:45.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 -- 172.21.15.83:0/1245649595 shutdown_connections 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0ff4070 0x5559d13a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d139c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 --2- 172.21.15.83:0/1245649595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0fcb1b0 0x5559d13a87a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.518+0000 7f88244a3740 1 -- 172.21.15.83:0/1245649595 >> 172.21.15.83:0/1245649595 conn(0x5559d0fe8460 msgr2=0x5559d139c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.519+0000 7f88244a3740 1 -- 172.21.15.83:0/1245649595 shutdown_connections 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.519+0000 7f88244a3740 1 -- 172.21.15.83:0/1245649595 wait complete. 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.519+0000 7f88244a3740 1 -- start start 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.519+0000 7f88244a3740 1 Processor -- start 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b7130 con 0x5559d0fd3390 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37394/0 (socket says 172.21.15.83:37394) 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 -- 172.21.15.83:0/559267248 learned_addr learned my addr 172.21.15.83:0/559267248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b6a60 con 0x5559d0fcb1b0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b3cc0 con 0x5559d0ff4070 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88244a3740 10 client.0 _ll_register_callbacks cb 0x7f880c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f88177fe640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8817fff640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 -- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 msgr2=0x5559d13b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 -- 172.21.15.83:0/559267248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 msgr2=0x5559d13bc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13bc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.520+0000 7f8816ffd640 1 -- 172.21.15.83:0/559267248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d13ba1c0 con 0x5559d0fd3390 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f8816ffd640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f880c011e80 tx=0x7f880c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c934c2a7a8df59cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f87f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c028040 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f880c004f70 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88080ad490 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f880c03d050 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f88244a3740 10 client.17076 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559d1422fd0 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.521+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88080ad490 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f880c034070 con 0x5559d0fd3390 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 10 client.17076 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 10 client.17076 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 20 client.17076 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0f87da0 0x5559d1425730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 4 client.17076 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559d130f8a0 con 0x5559d0f87da0 2024-09-21T03:26:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88177fe640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0f87da0 0x5559d1425730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f87f6ffd640 10 client.17076 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.522+0000 7f88177fe640 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0f87da0 0x5559d1425730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b81f6f974f339259 server_cookie=212a7005cd97d768 in_seq=0 out_seq=0 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f87f6ffd640 1 -- 172.21.15.83:0/559267248 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559d130f8a0 con 0x5559d0f87da0 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f87f6ffd640 10 client.17076 handle_command_reply: tid=0 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 client.17076 shutdown 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 msgr2=0x5559d13b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f880c011e80 tx=0x7f880c0092a0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 shutdown_connections 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5559d0f87da0 0x5559d1425730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f87f6ffd640 0 client.17076 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5559d0ff4070 0x5559d13b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559d0fd3390 0x5559d13b9640 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 --2- 172.21.15.83:0/559267248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559d0fcb1b0 0x5559d13bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 >> 172.21.15.83:0/559267248 conn(0x5559d0fe8460 msgr2=0x5559d13a8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 shutdown_connections 2024-09-21T03:26:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 1 -- 172.21.15.83:0/559267248 wait complete. 2024-09-21T03:26:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.523+0000 7f88244a3740 20 client.17076 trim_cache size 0 max 16384 2024-09-21T03:26:45.529 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 msgr2=0x7f88000b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 0x7f88000b8c40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f88100094e0 tx=0x7f8810005da0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 msgr2=0x7f881810bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9f0 tx=0x7f880800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 shutdown_connections 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f88000b6730 0x7f88000b8c40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8818108cf0 0x7f88181ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88181083a0 0x7f881810c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 --2- 172.21.15.83:0/1736703685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818102a60 0x7f881810bbc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 >> 172.21.15.83:0/1736703685 conn(0x7f8818079730 msgr2=0x7f88180ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 shutdown_connections 2024-09-21T03:26:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:45.524+0000 7f881df2f640 1 -- 172.21.15.83:0/1736703685 wait complete. 2024-09-21T03:26:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:45 smithi083 ceph-mon[31219]: pgmap v635: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-09-21T03:26:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:45 smithi142 ceph-mon[34512]: pgmap v635: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-09-21T03:26:45.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:45 smithi193 ceph-mon[35224]: pgmap v635: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-09-21T03:26:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 565.0 s, 1907.4 tps, lat 52.717 ms stddev 56.243 2024-09-21T03:26:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:47 smithi083 ceph-mon[31219]: pgmap v636: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T03:26:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:47 smithi142 ceph-mon[34512]: pgmap v636: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T03:26:47.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:26:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:47 smithi193 ceph-mon[35224]: pgmap v636: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 660 op/s 2024-09-21T03:26:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.593+0000 7f62c021f640 1 -- 172.21.15.83:0/505388396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8116640 msgr2=0x7f62b8118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.593+0000 7f62c021f640 1 --2- 172.21.15.83:0/505388396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8116640 0x7f62b8118b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009ca0 tx=0x7f62b002a8f0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.598+0000 7f62c021f640 1 -- 172.21.15.83:0/505388396 shutdown_connections 2024-09-21T03:26:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.598+0000 7f62c021f640 1 --2- 172.21.15.83:0/505388396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8116640 0x7f62b8118b50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.598+0000 7f62c021f640 1 --2- 172.21.15.83:0/505388396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b810aee0 0x7f62b8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.598+0000 7f62c021f640 1 --2- 172.21.15.83:0/505388396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62b810a590 0x7f62b810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.598+0000 7f62c021f640 1 -- 172.21.15.83:0/505388396 >> 172.21.15.83:0/505388396 conn(0x7f62b8070ec0 msgr2=0x7f62b80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.601+0000 7f62c021f640 1 -- 172.21.15.83:0/505388396 shutdown_connections 2024-09-21T03:26:48.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.601+0000 7f62c021f640 1 -- 172.21.15.83:0/505388396 wait complete. 2024-09-21T03:26:48.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 Processor -- start 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 -- start start 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b810a590 0x7f62b81b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62b810aee0 0x7f62b81b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62b811a150 con 0x7f62b810a590 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8119fd0 con 0x7f62b810aee0 2024-09-21T03:26:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.602+0000 7f62c021f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62b811a2d0 con 0x7f62b8116640 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.603+0000 7f62be7bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.603+0000 7f62be7bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51336/0 (socket says 172.21.15.83:51336) 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.603+0000 7f62be7bc640 1 -- 172.21.15.83:0/605591256 learned_addr learned my addr 172.21.15.83:0/605591256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.604+0000 7f62be7bc640 1 -- 172.21.15.83:0/605591256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62b810aee0 msgr2=0x7f62b81b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.604+0000 7f62be7bc640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62b810aee0 0x7f62b81b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.604+0000 7f62be7bc640 1 -- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b810a590 msgr2=0x7f62b81b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.604+0000 7f62be7bc640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b810a590 0x7f62b81b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.604+0000 7f62be7bc640 1 -- 172.21.15.83:0/605591256 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b81b7a80 con 0x7f62b8116640 2024-09-21T03:26:48.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.606+0000 7f62be7bc640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62b0033040 tx=0x7f62b0031720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.606+0000 7f62aaffd640 1 -- 172.21.15.83:0/605591256 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0003b70 con 0x7f62b8116640 2024-09-21T03:26:48.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.606+0000 7f62aaffd640 1 -- 172.21.15.83:0/605591256 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b002d070 con 0x7f62b8116640 2024-09-21T03:26:48.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.608+0000 7f62aaffd640 1 -- 172.21.15.83:0/605591256 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b002f8a0 con 0x7f62b8116640 2024-09-21T03:26:48.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.611+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b81b7d10 con 0x7f62b8116640 2024-09-21T03:26:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.611+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b81b8190 con 0x7f62b8116640 2024-09-21T03:26:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.615+0000 7f62c6793740 1 Processor -- start 2024-09-21T03:26:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.618+0000 7f62aaffd640 1 -- 172.21.15.83:0/605591256 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f62b0003d10 con 0x7f62b8116640 2024-09-21T03:26:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.618+0000 7f62c6793740 1 -- start start 2024-09-21T03:26:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc005c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc8c1b0 0x559dc005c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559dbffd2ea0 con 0x559dbfcb5070 2024-09-21T03:26:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559dbffc22d0 con 0x559dbfc8c1b0 2024-09-21T03:26:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62c6793740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559dbffc60e0 con 0x559dbfc94390 2024-09-21T03:26:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62bd7ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62bd7ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51352/0 (socket says 172.21.15.83:51352) 2024-09-21T03:26:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62bd7ba640 1 -- 172.21.15.83:0/2504381416 learned_addr learned my addr 172.21.15.83:0/2504381416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.619+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc005c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 -- 172.21.15.83:0/2504381416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc8c1b0 msgr2=0x559dc005c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc8c1b0 0x559dc005c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 -- 172.21.15.83:0/2504381416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 msgr2=0x559dc005c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc005c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 -- 172.21.15.83:0/2504381416 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dbfd20650 con 0x559dbfc94390 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.622+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011fd0 tx=0x7f62b4011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204ed1a52c10ad22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.623+0000 7f62aaffd640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 0x7f62a40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.623+0000 7f62a3fff640 1 -- 172.21.15.83:0/2504381416 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4004050 con 0x559dbfc94390 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.623+0000 7f62a3fff640 1 -- 172.21.15.83:0/2504381416 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b400dbf0 con 0x559dbfc94390 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.623+0000 7f62a3fff640 1 -- 172.21.15.83:0/2504381416 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4003930 con 0x559dbfc94390 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.624+0000 7f62aaffd640 1 -- 172.21.15.83:0/605591256 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f62b0108a90 con 0x7f62b8116640 2024-09-21T03:26:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 -- 172.21.15.83:0/2504381416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 msgr2=0x559dc005cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011fd0 tx=0x7f62b4011e00 comp rx=0 tx=0).stop 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 -- 172.21.15.83:0/2504381416 shutdown_connections 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc94390 0x559dc005cd40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc8c1b0 0x559dc005c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 --2- 172.21.15.83:0/2504381416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc005c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62c6793740 1 -- 172.21.15.83:0/2504381416 >> 172.21.15.83:0/2504381416 conn(0x559dbfca9460 msgr2=0x559dc005a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.625+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 0x7f62a40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.626+0000 7f62c6793740 1 -- 172.21.15.83:0/2504381416 shutdown_connections 2024-09-21T03:26:48.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.626+0000 7f62c6793740 1 -- 172.21.15.83:0/2504381416 wait complete. 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.626+0000 7f62c6793740 1 -- start start 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.626+0000 7f62c6793740 1 Processor -- start 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0067f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc0073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559dbfe23700 con 0x559dbfc94390 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559dbff5d160 con 0x559dbfcb5070 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559dc006aaa0 con 0x559dbfc8c1b0 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62c6793740 10 client.0 _ll_register_callbacks cb 0x559dbff769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62be7bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc0073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62be7bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc0073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53712/0 (socket says 172.21.15.83:53712) 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.627+0000 7f62be7bc640 1 -- 172.21.15.83:0/434439410 learned_addr learned my addr 172.21.15.83:0/434439410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 -- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 msgr2=0x559dc0067a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 -- 172.21.15.83:0/434439410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 msgr2=0x559dc0073dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc0073dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 -- 172.21.15.83:0/434439410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dbff8ac90 con 0x559dbfc94390 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62bd7ba640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0067f60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011fd0 tx=0x7f62b4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35121e759cd3a833 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.628+0000 7f62a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400d740 con 0x559dbfc94390 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 0x7f62a40b8ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f62ac006fd0 tx=0x7f62ac006c20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b400dbf0 con 0x559dbfc94390 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f62b400d8e0 con 0x559dbfc94390 2024-09-21T03:26:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62b402c020 con 0x559dbfc94390 2024-09-21T03:26:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62c6793740 10 client.17085 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.629+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559dc0076840 con 0x559dbfc94390 2024-09-21T03:26:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b400dbf0 con 0x559dbfc94390 2024-09-21T03:26:48.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f62b4004c20 con 0x559dbfc94390 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 10 client.17085 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 10 client.17085 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 20 client.17085 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc739f0 0x559dc0079c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 4 client.17085 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.630+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dbffd4df0 con 0x559dbfc739f0 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.631+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc739f0 0x559dc0079c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.632+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc739f0 0x559dc0079c00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b82dd356b0676643 server_cookie=742013bd7b616ca3 in_seq=0 out_seq=0 2024-09-21T03:26:48.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.632+0000 7f62a17fa640 10 client.17085 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:26:48.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.633+0000 7f62a17fa640 1 -- 172.21.15.83:0/434439410 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dbffd4df0 con 0x559dbfc739f0 2024-09-21T03:26:48.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.633+0000 7f62a17fa640 10 client.17085 handle_command_reply: tid=0 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.634+0000 7f62c6793740 1 client.17085 shutdown 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.634+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 msgr2=0x559dc0067f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.634+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0067f60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011fd0 tx=0x7f62b4005f10 comp rx=0 tx=0).stop 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 shutdown_connections 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc0073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0067f60 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc739f0 0x559dc0079c00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 --2- 172.21.15.83:0/434439410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 >> 172.21.15.83:0/434439410 conn(0x559dbfca9460 msgr2=0x559dc005f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62a17fa640 0 client.17085 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:26:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 shutdown_connections 2024-09-21T03:26:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 1 -- 172.21.15.83:0/434439410 wait complete. 2024-09-21T03:26:48.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.635+0000 7f62c6793740 20 client.17085 trim_cache size 0 max 16384 2024-09-21T03:26:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.658+0000 7f62c6793740 1 Processor -- start 2024-09-21T03:26:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.658+0000 7f62c6793740 1 -- start start 2024-09-21T03:26:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.658+0000 7f62c6793740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62c6793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0073d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51364/0 (socket says 172.21.15.83:51364) 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 -- 172.21.15.83:0/3841409378 learned_addr learned my addr 172.21.15.83:0/3841409378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62c6793740 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc005f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81adc00 con 0x559dbfc94390 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81af740 con 0x559dbfcb5070 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ae2d0 con 0x559dbfc8c1b0 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 -- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 msgr2=0x559dc005f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc005f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc005f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 -- 172.21.15.83:0/3841409378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 msgr2=0x559dc0073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.659+0000 7f62bdfbb640 1 -- 172.21.15.83:0/3841409378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc005e5b0 con 0x559dbfc8c1b0 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.660+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc005f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.660+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0092d0 tx=0x7f62ac009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8c70c6fc51e451 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.660+0000 7f62a37fe640 1 -- 172.21.15.83:0/3841409378 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00b960 con 0x559dbfc8c1b0 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.660+0000 7f62a37fe640 1 -- 172.21.15.83:0/3841409378 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac02b070 con 0x559dbfc8c1b0 2024-09-21T03:26:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.660+0000 7f62a37fe640 1 -- 172.21.15.83:0/3841409378 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac020400 con 0x559dbfc8c1b0 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.661+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 msgr2=0x559dc0067ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.661+0000 7f62c6793740 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0092d0 tx=0x7f62ac009f90 comp rx=0 tx=0).stop 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.661+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 shutdown_connections 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.661+0000 7f62c6793740 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfcb5070 0x559dc005f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.661+0000 7f62c6793740 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfc94390 0x559dc0073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 --2- 172.21.15.83:0/3841409378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc0067ee0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 >> 172.21.15.83:0/3841409378 conn(0x559dbfca9460 msgr2=0x559dc00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 shutdown_connections 2024-09-21T03:26:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 -- 172.21.15.83:0/3841409378 wait complete. 2024-09-21T03:26:48.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 -- start start 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.662+0000 7f62c6793740 1 Processor -- start 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 0x559dc007ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc00db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62bdfbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62bdfbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62bd7ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 0x559dc007ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62bd7ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 0x559dc007ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:53734/0 (socket says 172.21.15.83:53734) 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62bdfbb640 1 -- 172.21.15.83:0/3354334020 learned_addr learned my addr 172.21.15.83:0/3354334020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81af070 con 0x559dbfcb5070 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ae9a0 con 0x559dbfc94390 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc00db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810b5e0 con 0x559dbfc8c1b0 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62c6793740 10 client.0 _ll_register_callbacks cb 0x7f62b4011850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.663+0000 7f62be7bc640 1 -- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 msgr2=0x559dc007a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62be7bc640 1 -- 172.21.15.83:0/3354334020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 msgr2=0x559dc007ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 0x559dc007ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62be7bc640 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc00dba30 con 0x559dbfcb5070 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62be7bc640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc00db3c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f62b00caea0 tx=0x7f62b00ce2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86578d17a2b8f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b002fc40 con 0x559dbfcb5070 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f62b00cdb30 con 0x559dbfcb5070 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac02b070 con 0x559dbfcb5070 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62b00e9050 con 0x559dbfcb5070 2024-09-21T03:26:48.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62c6793740 10 client.17088 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.664+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559dc00e3ee0 con 0x559dbfcb5070 2024-09-21T03:26:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b0036070 con 0x559dbfcb5070 2024-09-21T03:26:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f62b0004040 con 0x559dbfcb5070 2024-09-21T03:26:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 10 client.17088 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 10 client.17088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:26:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 20 client.17088 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc48da0 0x559dc00e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 4 client.17088 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f62ac02b070 con 0x559dbfc48da0 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.665+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc48da0 0x559dc00e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62bdfbb640 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc48da0 0x559dc00e66f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e38088b3891f19e2 server_cookie=f2746f35aa9450cc in_seq=0 out_seq=0 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62a17fa640 10 client.17088 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62a17fa640 1 -- 172.21.15.83:0/3354334020 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f62ac02b070 con 0x559dbfc48da0 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62a17fa640 10 client.17088 handle_command_reply: tid=0 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 client.17088 shutdown 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 msgr2=0x559dc00db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc00db3c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f62b00caea0 tx=0x7f62b00ce2e0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 shutdown_connections 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dbfcb5070 0x559dc00db3c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dbfc94390 0x559dc007ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559dbfc48da0 0x559dc00e66f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 --2- 172.21.15.83:0/3354334020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559dbfc8c1b0 0x559dc007a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.667+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 >> 172.21.15.83:0/3354334020 conn(0x559dbfca9460 msgr2=0x559dc005f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.668+0000 7f62a17fa640 0 client.17088 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.668+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 shutdown_connections 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.668+0000 7f62c6793740 1 -- 172.21.15.83:0/3354334020 wait complete. 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.668+0000 7f62c6793740 20 client.17088 trim_cache size 0 max 16384 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 msgr2=0x7f62a40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 0x7f62a40b8ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f62ac006fd0 tx=0x7f62ac006c20 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 msgr2=0x7f62b81b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62b0033040 tx=0x7f62b0031720 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 shutdown_connections 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f62a40b65a0 0x7f62a40b8ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b8116640 0x7f62b81b7350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62b810aee0 0x7f62b81b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 --2- 172.21.15.83:0/605591256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b810a590 0x7f62b81b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 >> 172.21.15.83:0/605591256 conn(0x7f62b8070ec0 msgr2=0x7f62b8071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 shutdown_connections 2024-09-21T03:26:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:48.669+0000 7f62c021f640 1 -- 172.21.15.83:0/605591256 wait complete. 2024-09-21T03:26:49.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:49 smithi083 ceph-mon[31219]: pgmap v637: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.2 MiB/s wr, 650 op/s 2024-09-21T03:26:49.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:26:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:49 smithi142 ceph-mon[34512]: pgmap v637: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.2 MiB/s wr, 650 op/s 2024-09-21T03:26:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:49 smithi193 ceph-mon[35224]: pgmap v637: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.2 MiB/s wr, 650 op/s 2024-09-21T03:26:50.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 -- 172.21.15.83:0/2419385077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45341028c0 msgr2=0x7f4534102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 --2- 172.21.15.83:0/2419385077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45341028c0 0x7f4534102cd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f453002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:50.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 -- 172.21.15.83:0/2419385077 shutdown_connections 2024-09-21T03:26:50.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 --2- 172.21.15.83:0/2419385077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4534108b50 0x7f453410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 --2- 172.21.15.83:0/2419385077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108200 0x7f4534108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 --2- 172.21.15.83:0/2419385077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45341028c0 0x7f4534102cd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.395+0000 7f453bed7640 1 -- 172.21.15.83:0/2419385077 >> 172.21.15.83:0/2419385077 conn(0x7f45340fe350 msgr2=0x7f45341007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.396+0000 7f453bed7640 1 -- 172.21.15.83:0/2419385077 shutdown_connections 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.396+0000 7f453bed7640 1 -- 172.21.15.83:0/2419385077 wait complete. 2024-09-21T03:26:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 Processor -- start 2024-09-21T03:26:50.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 -- start start 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 0x7f453419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 0x7f453419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 0x7f453419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112810 con 0x7f4534108200 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112690 con 0x7f45341028c0 2024-09-21T03:26:50.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f453bed7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112990 con 0x7f4534108b50 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f4539472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 0x7f453419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.397+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 0x7f453419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f4539472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 0x7f453419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44368/0 (socket says 172.21.15.83:44368) 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 0x7f453419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36746/0 (socket says 172.21.15.83:36746) 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f4539472640 1 -- 172.21.15.83:0/2382052500 learned_addr learned my addr 172.21.15.83:0/2382052500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 0x7f453419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 -- 172.21.15.83:0/2382052500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 msgr2=0x7f453419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 0x7f453419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 -- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 msgr2=0x7f453419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 0x7f453419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 -- 172.21.15.83:0/2382052500 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45341a02c0 con 0x7f4534108b50 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 0x7f453419fb90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f452c001440 tx=0x7f452c00d760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f452affd640 1 -- 172.21.15.83:0/2382052500 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c00c7c0 con 0x7f4534108b50 2024-09-21T03:26:50.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45341a05b0 con 0x7f4534108b50 2024-09-21T03:26:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f452affd640 1 -- 172.21.15.83:0/2382052500 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452c00ce10 con 0x7f4534108b50 2024-09-21T03:26:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f452affd640 1 -- 172.21.15.83:0/2382052500 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c017e50 con 0x7f4534108b50 2024-09-21T03:26:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.398+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341a6d10 con 0x7f4534108b50 2024-09-21T03:26:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.402+0000 7f454244b740 1 Processor -- start 2024-09-21T03:26:50.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.402+0000 7f452affd640 1 -- 172.21.15.83:0/2382052500 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f452c00c960 con 0x7f4534108b50 2024-09-21T03:26:50.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.402+0000 7f454244b740 1 -- start start 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba66070 0x55d04be0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd83ea0 con 0x55d04ba66070 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd732d0 con 0x55d04ba3d1b0 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f454244b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd770e0 con 0x55d04ba45390 2024-09-21T03:26:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36750/0 (socket says 172.21.15.83:36750) 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 -- 172.21.15.83:0/140205148 learned_addr learned my addr 172.21.15.83:0/140205148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539472640 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba66070 0x55d04be0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f452affd640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 0x7f45200b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 -- 172.21.15.83:0/140205148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 msgr2=0x55d04be0e2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539472640 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be0e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 -- 172.21.15.83:0/140205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba66070 msgr2=0x55d04be0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba66070 0x55d04be0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f4539c73640 1 -- 172.21.15.83:0/140205148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04bad1650 con 0x55d04ba3d1b0 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.403+0000 7f452affd640 1 -- 172.21.15.83:0/2382052500 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f452c0e7d90 con 0x7f4534108b50 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.404+0000 7f4539c73640 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d13919a6b442a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.404+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 0x7f45200b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.404+0000 7f451ffff640 1 -- 172.21.15.83:0/140205148 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530040070 con 0x55d04ba3d1b0 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.405+0000 7f451ffff640 1 -- 172.21.15.83:0/140205148 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4530004030 con 0x55d04ba3d1b0 2024-09-21T03:26:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.405+0000 7f451ffff640 1 -- 172.21.15.83:0/140205148 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530035040 con 0x55d04ba3d1b0 2024-09-21T03:26:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f453a474640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 0x7f45200b8c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f452c0aaef0 tx=0x7f452c00df90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 -- 172.21.15.83:0/140205148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 msgr2=0x55d04be0dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530032b90 comp rx=0 tx=0).stop 2024-09-21T03:26:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 -- 172.21.15.83:0/140205148 shutdown_connections 2024-09-21T03:26:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be0e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba3d1b0 0x55d04be0dda0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 --2- 172.21.15.83:0/140205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba66070 0x55d04be0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 -- 172.21.15.83:0/140205148 >> 172.21.15.83:0/140205148 conn(0x55d04ba5a460 msgr2=0x55d04be0b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 -- 172.21.15.83:0/140205148 shutdown_connections 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.406+0000 7f454244b740 1 -- 172.21.15.83:0/140205148 wait complete. 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 -- start start 2024-09-21T03:26:50.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 Processor -- start 2024-09-21T03:26:50.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba66070 0x55d04be252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d04bbd4700 con 0x55d04ba3d1b0 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.407+0000 7f454244b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd0e160 con 0x55d04ba45390 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f454244b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d04be1bfe0 con 0x55d04ba66070 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f454244b740 10 client.0 _ll_register_callbacks cb 0x55d04bd279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36766/0 (socket says 172.21.15.83:36766) 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44382/0 (socket says 172.21.15.83:44382) 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 -- 172.21.15.83:0/1958898083 learned_addr learned my addr 172.21.15.83:0/1958898083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539472640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba66070 0x55d04be252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 -- 172.21.15.83:0/1958898083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba66070 msgr2=0x55d04be252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba66070 0x55d04be252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 -- 172.21.15.83:0/1958898083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 msgr2=0x55d04be19010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 -- 172.21.15.83:0/1958898083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04bd3bc90 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539472640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f4539c73640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f453002a930 tx=0x7f453002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e9e505f1d0a900 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.408+0000 7f451d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530035070 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f4530005460 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4530004030 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4530040040 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 10 client.26636 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d04be27d90 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4530004030 con 0x55d04ba45390 2024-09-21T03:26:50.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4530004cf0 con 0x55d04ba45390 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 10 client.26636 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 10 client.26636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.409+0000 7f454244b740 20 client.26636 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.410+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba249f0 0x55d04be2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.410+0000 7f454244b740 4 client.26636 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.410+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04bd85df0 con 0x55d04ba249f0 2024-09-21T03:26:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.410+0000 7f4539472640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba249f0 0x55d04be2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.411+0000 7f451d7fa640 10 client.26636 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:26:50.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.411+0000 7f4539472640 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba249f0 0x55d04be2b0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=169e2581f6e8b66e server_cookie=ec5bbba8c38f3cc9 in_seq=0 out_seq=0 2024-09-21T03:26:50.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.412+0000 7f451d7fa640 1 -- 172.21.15.83:0/1958898083 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d04bd85df0 con 0x55d04ba249f0 2024-09-21T03:26:50.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.412+0000 7f451d7fa640 10 client.26636 handle_command_reply: tid=0 2024-09-21T03:26:50.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.412+0000 7f454244b740 1 client.26636 shutdown 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 msgr2=0x55d04be19550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f453002a930 tx=0x7f453002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 shutdown_connections 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba249f0 0x55d04be2b0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f451d7fa640 0 client.26636 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba66070 0x55d04be252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba45390 0x55d04be19550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 --2- 172.21.15.83:0/1958898083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be19010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 >> 172.21.15.83:0/1958898083 conn(0x55d04ba5a460 msgr2=0x55d04be10a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 shutdown_connections 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 1 -- 172.21.15.83:0/1958898083 wait complete. 2024-09-21T03:26:50.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.413+0000 7f454244b740 20 client.26636 trim_cache size 0 max 16384 2024-09-21T03:26:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f454244b740 1 Processor -- start 2024-09-21T03:26:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f454244b740 1 -- start start 2024-09-21T03:26:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f454244b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f454244b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f454244b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be16740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.430+0000 7f4539472640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36770/0 (socket says 172.21.15.83:36770) 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 -- 172.21.15.83:0/3428231557 learned_addr learned my addr 172.21.15.83:0/3428231557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45341ad250 con 0x55d04ba3d1b0 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45341aed90 con 0x55d04ba249f0 2024-09-21T03:26:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45341ad920 con 0x55d04ba45390 2024-09-21T03:26:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be16740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 -- 172.21.15.83:0/3428231557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 msgr2=0x55d04be16740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be16740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 -- 172.21.15.83:0/3428231557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 msgr2=0x55d04be16200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be16200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.431+0000 7f4539472640 1 -- 172.21.15.83:0/3428231557 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04be1a2d0 con 0x55d04ba249f0 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.432+0000 7f4539472640 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4534101bc0 tx=0x7f45240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3813d34a5b0e7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.432+0000 7f451f7fe640 1 -- 172.21.15.83:0/3428231557 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400fd00 con 0x55d04ba249f0 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.432+0000 7f451f7fe640 1 -- 172.21.15.83:0/3428231557 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45240248f0 con 0x55d04ba249f0 2024-09-21T03:26:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.432+0000 7f451f7fe640 1 -- 172.21.15.83:0/3428231557 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524022650 con 0x55d04ba249f0 2024-09-21T03:26:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 msgr2=0x55d04be0efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4534101bc0 tx=0x7f45240046c0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 shutdown_connections 2024-09-21T03:26:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba45390 0x55d04be16740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba3d1b0 0x55d04be16200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 --2- 172.21.15.83:0/3428231557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be0efa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 >> 172.21.15.83:0/3428231557 conn(0x55d04ba5a460 msgr2=0x55d04be0e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 shutdown_connections 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.433+0000 7f454244b740 1 -- 172.21.15.83:0/3428231557 wait complete. 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 -- start start 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 Processor -- start 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be2df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba45390 0x55d04be8aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45341ae6c0 con 0x55d04ba45390 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45341adff0 con 0x55d04ba249f0 2024-09-21T03:26:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45341ab250 con 0x55d04ba3d1b0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.434+0000 7f454244b740 10 client.0 _ll_register_callbacks cb 0x7f4530009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539472640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be2df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53938/0 (socket says 172.21.15.83:53938) 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 -- 172.21.15.83:0/351637842 learned_addr learned my addr 172.21.15.83:0/351637842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539472640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba45390 0x55d04be8aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 -- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 msgr2=0x55d04be2df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be2df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 -- 172.21.15.83:0/351637842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba45390 msgr2=0x55d04be8aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba45390 0x55d04be8aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 -- 172.21.15.83:0/351637842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04be8b110 con 0x55d04ba3d1b0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f4539c73640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3207e6ce816519cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f451d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:50.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530064060 con 0x55d04ba3d1b0 2024-09-21T03:26:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f45300316b0 con 0x55d04ba3d1b0 2024-09-21T03:26:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.435+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45240248f0 con 0x55d04ba3d1b0 2024-09-21T03:26:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.436+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4530035070 con 0x55d04ba3d1b0 2024-09-21T03:26:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.436+0000 7f454244b740 10 client.26608 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:50.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.436+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d04be95720 con 0x55d04ba3d1b0 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45240248f0 con 0x55d04ba3d1b0 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4530003f30 con 0x55d04ba3d1b0 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 10 client.26608 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 10 client.26608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 20 client.26608 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:50.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba66070 0x55d04be97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 4 client.26608 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.437+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d04bd818a0 con 0x55d04ba66070 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f4539472640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba66070 0x55d04be97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f451d7fa640 10 client.26608 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f4539472640 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba66070 0x55d04be97eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7054355e0fa842a3 server_cookie=d7d4f58d5d39d148 in_seq=0 out_seq=0 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f451d7fa640 1 -- 172.21.15.83:0/351637842 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d04bd818a0 con 0x55d04ba66070 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f451d7fa640 10 client.26608 handle_command_reply: tid=0 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.438+0000 7f454244b740 1 client.26608 shutdown 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 msgr2=0x55d04be2e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4530009900 tx=0x7f4530005e70 comp rx=0 tx=0).stop 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 shutdown_connections 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d04ba66070 0x55d04be97eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f451d7fa640 0 client.26608 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:26:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04ba45390 0x55d04be8aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04ba3d1b0 0x55d04be2e510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 --2- 172.21.15.83:0/351637842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04ba249f0 0x55d04be2df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.439+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 >> 172.21.15.83:0/351637842 conn(0x55d04ba5a460 msgr2=0x55d04be110f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.441+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 shutdown_connections 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.441+0000 7f454244b740 1 -- 172.21.15.83:0/351637842 wait complete. 2024-09-21T03:26:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.441+0000 7f454244b740 20 client.26608 trim_cache size 0 max 16384 2024-09-21T03:26:50.445 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 msgr2=0x7f45200b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 0x7f45200b8c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f452c0aaef0 tx=0x7f452c00df90 comp rx=0 tx=0).stop 2024-09-21T03:26:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 msgr2=0x7f453419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 0x7f453419fb90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f452c001440 tx=0x7f452c00d760 comp rx=0 tx=0).stop 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 shutdown_connections 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f45200b6780 0x7f45200b8c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534108b50 0x7f453419fb90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534108200 0x7f453419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 --2- 172.21.15.83:0/2382052500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45341028c0 0x7f453419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 >> 172.21.15.83:0/2382052500 conn(0x7f45340fe350 msgr2=0x7f4534107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 shutdown_connections 2024-09-21T03:26:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:50.442+0000 7f453bed7640 1 -- 172.21.15.83:0/2382052500 wait complete. 2024-09-21T03:26:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 570.0 s, 1917.0 tps, lat 51.660 ms stddev 58.877 2024-09-21T03:26:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:51 smithi083 ceph-mon[31219]: pgmap v638: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.3 MiB/s wr, 769 op/s 2024-09-21T03:26:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:51 smithi142 ceph-mon[34512]: pgmap v638: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.3 MiB/s wr, 769 op/s 2024-09-21T03:26:51.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:26:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:51 smithi193 ceph-mon[35224]: pgmap v638: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.3 MiB/s wr, 769 op/s 2024-09-21T03:26:52.152 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 -- 172.21.15.83:0/1737859620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38109e00 msgr2=0x7fba38110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 --2- 172.21.15.83:0/1737859620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38109e00 0x7fba38110630 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fba30009be0 tx=0x7fba3002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 -- 172.21.15.83:0/1737859620 shutdown_connections 2024-09-21T03:26:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 --2- 172.21.15.83:0/1737859620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38109e00 0x7fba38110630 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 --2- 172.21.15.83:0/1737859620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba381098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.326+0000 7fba3effc640 1 --2- 172.21.15.83:0/1737859620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38103bc0 0x7fba38103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.327+0000 7fba3effc640 1 -- 172.21.15.83:0/1737859620 >> 172.21.15.83:0/1737859620 conn(0x7fba3807b910 msgr2=0x7fba3807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.327+0000 7fba3effc640 1 -- 172.21.15.83:0/1737859620 shutdown_connections 2024-09-21T03:26:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.327+0000 7fba3effc640 1 -- 172.21.15.83:0/1737859620 wait complete. 2024-09-21T03:26:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.328+0000 7fba3effc640 1 Processor -- start 2024-09-21T03:26:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.328+0000 7fba3effc640 1 -- start start 2024-09-21T03:26:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba3819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38109e00 0x7fba3819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba38113ac0 con 0x7fba38103bc0 2024-09-21T03:26:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba38113940 con 0x7fba381094b0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3effc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba38113c40 con 0x7fba38109e00 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44406/0 (socket says 172.21.15.83:44406) 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 -- 172.21.15.83:0/3888235519 learned_addr learned my addr 172.21.15.83:0/3888235519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3d599640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38109e00 0x7fba3819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba37fff640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba3819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 -- 172.21.15.83:0/3888235519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38109e00 msgr2=0x7fba3819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38109e00 0x7fba3819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 -- 172.21.15.83:0/3888235519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 msgr2=0x7fba3819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba3819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.329+0000 7fba3cd98640 1 -- 172.21.15.83:0/3888235519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba381a03a0 con 0x7fba38103bc0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba37fff640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba3cd98640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9c0 tx=0x7fba2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba35ffb640 1 -- 172.21.15.83:0/3888235519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00cde0 con 0x7fba38103bc0 2024-09-21T03:26:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba381a0690 con 0x7fba38103bc0 2024-09-21T03:26:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba35ffb640 1 -- 172.21.15.83:0/3888235519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c0035f0 con 0x7fba38103bc0 2024-09-21T03:26:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba35ffb640 1 -- 172.21.15.83:0/3888235519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010690 con 0x7fba38103bc0 2024-09-21T03:26:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.330+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a6df0 con 0x7fba38103bc0 2024-09-21T03:26:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 Processor -- start 2024-09-21T03:26:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 -- start start 2024-09-21T03:26:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b77b82aea0 con 0x55b77b50d070 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b77b81a2d0 con 0x55b77b4e41b0 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.334+0000 7fba45570740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b77b81e0e0 con 0x55b77b4ec390 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53964/0 (socket says 172.21.15.83:53964) 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba3d599640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 -- 172.21.15.83:0/1952902104 learned_addr learned my addr 172.21.15.83:0/1952902104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba35ffb640 1 -- 172.21.15.83:0/3888235519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fba2c004020 con 0x7fba38103bc0 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 -- 172.21.15.83:0/1952902104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 msgr2=0x55b77b8b4be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8b4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 -- 172.21.15.83:0/1952902104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 msgr2=0x55b77b8b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 -- 172.21.15.83:0/1952902104 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77b578650 con 0x55b77b4ec390 2024-09-21T03:26:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.335+0000 7fba37fff640 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba24009c70 tx=0x7fba240117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633fb8494bba4e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba22ffd640 1 -- 172.21.15.83:0/1952902104 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24012450 con 0x55b77b4ec390 2024-09-21T03:26:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba22ffd640 1 -- 172.21.15.83:0/1952902104 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba24012aa0 con 0x55b77b4ec390 2024-09-21T03:26:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba22ffd640 1 -- 172.21.15.83:0/1952902104 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2400faf0 con 0x55b77b4ec390 2024-09-21T03:26:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba35ffb640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 0x7fba280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba35ffb640 1 -- 172.21.15.83:0/3888235519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fba2c0e7a30 con 0x7fba38103bc0 2024-09-21T03:26:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.336+0000 7fba3d599640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 0x7fba280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- 172.21.15.83:0/1952902104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 msgr2=0x55b77b8b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba24009c70 tx=0x7fba240117b0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba3d599640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 0x7fba280b8c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba30004af0 tx=0x7fba30005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- 172.21.15.83:0/1952902104 shutdown_connections 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8b5120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 --2- 172.21.15.83:0/1952902104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- 172.21.15.83:0/1952902104 >> 172.21.15.83:0/1952902104 conn(0x55b77b501460 msgr2=0x55b77b8b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- 172.21.15.83:0/1952902104 shutdown_connections 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- 172.21.15.83:0/1952902104 wait complete. 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.337+0000 7fba45570740 1 -- start start 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.338+0000 7fba45570740 1 Processor -- start 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8c0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b77b67b700 con 0x55b77b50d070 2024-09-21T03:26:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba37fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba37fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:53966/0 (socket says 172.21.15.83:53966) 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b77b7b5160 con 0x55b77b4e41b0 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba37fff640 1 -- 172.21.15.83:0/2141186473 learned_addr learned my addr 172.21.15.83:0/2141186473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b77b8c71e0 con 0x55b77b4ec390 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.339+0000 7fba45570740 10 client.0 _ll_register_callbacks cb 0x55b77b7ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba3d599640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba3cd98640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba37fff640 1 -- 172.21.15.83:0/2141186473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 msgr2=0x55b77b8c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba37fff640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba37fff640 1 -- 172.21.15.83:0/2141186473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 msgr2=0x55b77b8c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.340+0000 7fba37fff640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8c0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba37fff640 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77b7e2c90 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba3d599640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba37fff640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba24009990 tx=0x7fba240099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1231af2aaeffb619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba17fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24012660 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba24012aa0 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fba24012800 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fba24033050 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba45570740 10 client.26620 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.341+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b77b8d7370 con 0x55b77b4ec390 2024-09-21T03:26:53.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba24012aa0 con 0x55b77b4ec390 2024-09-21T03:26:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fba2400f7f0 con 0x55b77b4ec390 2024-09-21T03:26:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba45570740 10 client.26620 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba45570740 10 client.26620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:26:53.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba45570740 20 client.26620 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.342+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4cb9f0 0x55b77b8dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.343+0000 7fba45570740 4 client.26620 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.343+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77b82cdf0 con 0x55b77b4cb9f0 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.343+0000 7fba3cd98640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4cb9f0 0x55b77b8dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.343+0000 7fba17fff640 10 client.26620 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.343+0000 7fba3cd98640 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4cb9f0 0x55b77b8dab90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd4db3309b2b0099 server_cookie=c51fbd86392bb56c in_seq=0 out_seq=0 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba17fff640 1 -- 172.21.15.83:0/2141186473 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b77b82cdf0 con 0x55b77b4cb9f0 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba17fff640 10 client.26620 handle_command_reply: tid=0 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 client.26620 shutdown 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 msgr2=0x55b77b8bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba24009990 tx=0x7fba240099c0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 shutdown_connections 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4cb9f0 0x55b77b8dab90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8bfd30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 --2- 172.21.15.83:0/2141186473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 >> 172.21.15.83:0/2141186473 conn(0x55b77b501460 msgr2=0x55b77b8b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba17fff640 0 client.26620 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 shutdown_connections 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 1 -- 172.21.15.83:0/2141186473 wait complete. 2024-09-21T03:26:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.344+0000 7fba45570740 20 client.26620 trim_cache size 0 max 16384 2024-09-21T03:26:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.361+0000 7fba45570740 1 Processor -- start 2024-09-21T03:26:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.361+0000 7fba45570740 1 -- start start 2024-09-21T03:26:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.361+0000 7fba45570740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba45570740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba45570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba45570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba381ad290 con 0x55b77b50d070 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba45570740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba381aedd0 con 0x55b77b4e41b0 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba45570740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba381ad960 con 0x55b77b4ec390 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3cd98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44430/0 (socket says 172.21.15.83:44430) 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 -- 172.21.15.83:0/498940572 learned_addr learned my addr 172.21.15.83:0/498940572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba37fff640 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 -- 172.21.15.83:0/498940572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 msgr2=0x55b77b8be5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8be5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 -- 172.21.15.83:0/498940572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 msgr2=0x55b77b8c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 -- 172.21.15.83:0/498940572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77b8be230 con 0x55b77b50d070 2024-09-21T03:26:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba3d599640 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fba30033000 tx=0x7fba30040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a345dfd9f53f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.362+0000 7fba227fc640 1 -- 172.21.15.83:0/498940572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30036040 con 0x55b77b50d070 2024-09-21T03:26:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba227fc640 1 -- 172.21.15.83:0/498940572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba30050070 con 0x55b77b50d070 2024-09-21T03:26:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba227fc640 1 -- 172.21.15.83:0/498940572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300042a0 con 0x55b77b50d070 2024-09-21T03:26:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 -- 172.21.15.83:0/498940572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 msgr2=0x55b77b8beaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fba30033000 tx=0x7fba30040000 comp rx=0 tx=0).stop 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 -- 172.21.15.83:0/498940572 shutdown_connections 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b50d070 0x55b77b8beaf0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 --2- 172.21.15.83:0/498940572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b4e41b0 0x55b77b8c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.363+0000 7fba45570740 1 -- 172.21.15.83:0/498940572 >> 172.21.15.83:0/498940572 conn(0x55b77b501460 msgr2=0x55b77b8bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- 172.21.15.83:0/498940572 shutdown_connections 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- 172.21.15.83:0/498940572 wait complete. 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- start start 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 Processor -- start 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b50d070 0x55b77b8d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba381ae700 con 0x55b77b4e41b0 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba381ae030 con 0x55b77b50d070 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.364+0000 7fba45570740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba381ab320 con 0x55b77b4ec390 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba45570740 10 client.0 _ll_register_callbacks cb 0x7fba24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44442/0 (socket says 172.21.15.83:44442) 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba37fff640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 -- 172.21.15.83:0/539691483 learned_addr learned my addr 172.21.15.83:0/539691483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3d599640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b50d070 0x55b77b8d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 -- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 msgr2=0x55b77b8d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 -- 172.21.15.83:0/539691483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b50d070 msgr2=0x55b77b8d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b50d070 0x55b77b8d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 -- 172.21.15.83:0/539691483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77b9436c0 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba3cd98640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0ad860 tx=0x7fba2c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5f28745b381c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba17fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0adfd0 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fba2c0e5450 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.365+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba30050070 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fba2c0b23b0 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba45570740 10 client.17121 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b77b9449a0 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba30050070 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fba2c0d3070 con 0x55b77b4e41b0 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba45570740 10 client.17121 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.366+0000 7fba45570740 10 client.17121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.367+0000 7fba45570740 20 client.17121 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.367+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4a0da0 0x55b77b9470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.367+0000 7fba45570740 4 client.17121 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.367+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b77b8288a0 con 0x55b77b4a0da0 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.368+0000 7fba37fff640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4a0da0 0x55b77b9470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba17fff640 10 client.17121 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba37fff640 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4a0da0 0x55b77b9470c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d2dc4d05af1f55a server_cookie=e73d0b03944bb8d7 in_seq=0 out_seq=0 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba17fff640 1 -- 172.21.15.83:0/539691483 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b77b8288a0 con 0x55b77b4a0da0 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba17fff640 10 client.17121 handle_command_reply: tid=0 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 client.17121 shutdown 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 msgr2=0x55b77b8cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0ad860 tx=0x7fba2c003cd0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 shutdown_connections 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b77b4a0da0 0x55b77b9470c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b77b50d070 0x55b77b8d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b77b4ec390 0x55b77b8d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 --2- 172.21.15.83:0/539691483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b77b4e41b0 0x55b77b8cff20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 >> 172.21.15.83:0/539691483 conn(0x55b77b501460 msgr2=0x55b77b8be440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba17fff640 0 client.17121 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 shutdown_connections 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 1 -- 172.21.15.83:0/539691483 wait complete. 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.369+0000 7fba45570740 20 client.17121 trim_cache size 0 max 16384 2024-09-21T03:26:53.376 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 msgr2=0x7fba280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 0x7fba280b8c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba30004af0 tx=0x7fba30005fd0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 msgr2=0x7fba3819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9c0 tx=0x7fba2c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 shutdown_connections 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fba280b6700 0x7fba280b8c10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba38109e00 0x7fba3819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba381094b0 0x7fba3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 --2- 172.21.15.83:0/3888235519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba38103bc0 0x7fba3819a430 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 >> 172.21.15.83:0/3888235519 conn(0x7fba3807b910 msgr2=0x7fba381074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 shutdown_connections 2024-09-21T03:26:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:53.370+0000 7fba3effc640 1 -- 172.21.15.83:0/3888235519 wait complete. 2024-09-21T03:26:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:53 smithi083 ceph-mon[31219]: pgmap v639: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 574 op/s 2024-09-21T03:26:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:53 smithi142 ceph-mon[34512]: pgmap v639: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 574 op/s 2024-09-21T03:26:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:53 smithi193 ceph-mon[35224]: pgmap v639: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 574 op/s 2024-09-21T03:26:54.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:26:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 -- 172.21.15.83:0/2059964735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac8104880 msgr2=0x7ffac8104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 --2- 172.21.15.83:0/2059964735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac8104880 0x7ffac8104c90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009900 tx=0x7ffab802a410 comp rx=0 tx=0).stop 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 -- 172.21.15.83:0/2059964735 shutdown_connections 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 --2- 172.21.15.83:0/2059964735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 --2- 172.21.15.83:0/2059964735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac81052b0 0x7ffac810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 --2- 172.21.15.83:0/2059964735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac8104880 0x7ffac8104c90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 -- 172.21.15.83:0/2059964735 >> 172.21.15.83:0/2059964735 conn(0x7ffac8079730 msgr2=0x7ffac8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.108+0000 7ffacdb61640 1 -- 172.21.15.83:0/2059964735 shutdown_connections 2024-09-21T03:26:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.109+0000 7ffacdb61640 1 -- 172.21.15.83:0/2059964735 wait complete. 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.109+0000 7ffacdb61640 1 Processor -- start 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 -- start start 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac8104880 0x7ffac819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac81a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81036c0 con 0x7ffac81052b0 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8103540 con 0x7ffac810de30 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffacdb61640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8103840 con 0x7ffac8104880 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44464/0 (socket says 172.21.15.83:44464) 2024-09-21T03:26:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac8104880 0x7ffac819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 -- 172.21.15.83:0/1694086707 learned_addr learned my addr 172.21.15.83:0/1694086707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac7fff640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac81a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 -- 172.21.15.83:0/1694086707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac8104880 msgr2=0x7ffac819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac8104880 0x7ffac819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 -- 172.21.15.83:0/1694086707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 msgr2=0x7ffac81a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac81a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.110+0000 7ffac6ffd640 1 -- 172.21.15.83:0/1694086707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac81a2610 con 0x7ffac81052b0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffac7fff640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac81a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e9f0 tx=0x7ffabc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffac4ff9640 1 -- 172.21.15.83:0/1694086707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00ccc0 con 0x7ffac81052b0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac81a2900 con 0x7ffac81052b0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffac4ff9640 1 -- 172.21.15.83:0/1694086707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc0054d0 con 0x7ffac81052b0 2024-09-21T03:26:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffac4ff9640 1 -- 172.21.15.83:0/1694086707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc010430 con 0x7ffac81052b0 2024-09-21T03:26:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.111+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81a9060 con 0x7ffac81052b0 2024-09-21T03:26:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 Processor -- start 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 -- start start 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355f0390 0x5638359b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56383592eea0 con 0x5638355e81b0 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56383591e2d0 con 0x563835611070 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffad40d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638359220e0 con 0x5638355f0390 2024-09-21T03:26:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.114+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36860/0 (socket says 172.21.15.83:36860) 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 -- 172.21.15.83:0/1562102491 learned_addr learned my addr 172.21.15.83:0/1562102491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 -- 172.21.15.83:0/1562102491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355f0390 msgr2=0x5638359b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355f0390 0x5638359b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 -- 172.21.15.83:0/1562102491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 msgr2=0x5638359b8800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359b8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.115+0000 7ffac7fff640 1 -- 172.21.15.83:0/1562102491 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56383567c650 con 0x563835611070 2024-09-21T03:26:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.116+0000 7ffac4ff9640 1 -- 172.21.15.83:0/1694086707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ffabc0105d0 con 0x7ffac81052b0 2024-09-21T03:26:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.117+0000 7ffac7fff640 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffac00117e0 tx=0x7ffac0011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1006d5e821a4f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.117+0000 7ffaadffb640 1 -- 172.21.15.83:0/1562102491 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011310 con 0x563835611070 2024-09-21T03:26:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.117+0000 7ffaadffb640 1 -- 172.21.15.83:0/1562102491 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0011310 con 0x563835611070 2024-09-21T03:26:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.117+0000 7ffaadffb640 1 -- 172.21.15.83:0/1562102491 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004120 con 0x563835611070 2024-09-21T03:26:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffac4ff9640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 0x7ffab00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffac77fe640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 0x7ffab00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 -- 172.21.15.83:0/1562102491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 msgr2=0x5638359b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffac00117e0 tx=0x7ffac0011ca0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 -- 172.21.15.83:0/1562102491 shutdown_connections 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355f0390 0x5638359b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffac4ff9640 1 -- 172.21.15.83:0/1694086707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7ffabc0e7aa0 con 0x7ffac81052b0 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 --2- 172.21.15.83:0/1562102491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563835611070 0x5638359b82c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 -- 172.21.15.83:0/1562102491 >> 172.21.15.83:0/1562102491 conn(0x563835605460 msgr2=0x5638359b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 -- 172.21.15.83:0/1562102491 shutdown_connections 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffac77fe640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 0x7ffab00b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffab8000c00 tx=0x7ffab8031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.118+0000 7ffad40d5740 1 -- 172.21.15.83:0/1562102491 wait complete. 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 -- start start 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 Processor -- start 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 0x5638359c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563835611070 0x5638359c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56383577f700 con 0x563835611070 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5638358b9160 con 0x5638355f0390 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638359be970 con 0x5638355e81b0 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.119+0000 7ffad40d5740 10 client.0 _ll_register_callbacks cb 0x5638358d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563835611070 0x5638359c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36866/0 (socket says 172.21.15.83:36866) 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 -- 172.21.15.83:0/506145544 learned_addr learned my addr 172.21.15.83:0/506145544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac77fe640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 0x5638359c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 -- 172.21.15.83:0/506145544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 msgr2=0x5638359c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 0x5638359c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 -- 172.21.15.83:0/506145544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563835611070 msgr2=0x5638359c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563835611070 0x5638359c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 -- 172.21.15.83:0/506145544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638358e6c90 con 0x5638355f0390 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac77fe640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 0x5638359c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0aab50 tx=0x7ffabc0a9fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21edf41092ada45f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffaa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0ad230 con 0x5638355f0390 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac0011310 con 0x5638355f0390 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.120+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7ffabc0acb40 con 0x5638355f0390 2024-09-21T03:26:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ffabc0ad050 con 0x5638355f0390 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 10 client.26666 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5638359e36e0 con 0x5638355f0390 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0011310 con 0x5638355f0390 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ffabc0ac5b0 con 0x5638355f0390 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 10 client.26666 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 10 client.26666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 20 client.26666 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355cf9f0 0x5638359e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 4 client.26666 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.121+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563835930df0 con 0x5638355cf9f0 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.122+0000 7ffac77fe640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355cf9f0 0x5638359e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.122+0000 7ffaa6ffd640 10 client.26666 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:55.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.123+0000 7ffac77fe640 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355cf9f0 0x5638359e6bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01a6ade6c5581a5 server_cookie=5fdf6d648b91b723 in_seq=0 out_seq=0 2024-09-21T03:26:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.143+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/506145544 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563835930df0 con 0x5638355cf9f0 2024-09-21T03:26:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.143+0000 7ffaa6ffd640 10 client.26666 handle_command_reply: tid=0 2024-09-21T03:26:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.144+0000 7ffad40d5740 1 client.26666 shutdown 2024-09-21T03:26:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.145+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 msgr2=0x5638359ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.145+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0aab50 tx=0x7ffabc0a9fa0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 shutdown_connections 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355cf9f0 0x5638359e6bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563835611070 0x5638359c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359ce900 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 --2- 172.21.15.83:0/506145544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638355e81b0 0x5638359c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 >> 172.21.15.83:0/506145544 conn(0x563835605460 msgr2=0x5638359b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.149+0000 7ffaa6ffd640 0 client.26666 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.151+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 shutdown_connections 2024-09-21T03:26:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.151+0000 7ffad40d5740 1 -- 172.21.15.83:0/506145544 wait complete. 2024-09-21T03:26:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.153+0000 7ffad40d5740 20 client.26666 trim_cache size 0 max 16384 2024-09-21T03:26:55.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 Processor -- start 2024-09-21T03:26:55.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 -- start start 2024-09-21T03:26:55.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 0x5638359c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x5638359ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81ad400 con 0x5638355f0390 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81aef40 con 0x5638355e81b0 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffad40d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81adad0 con 0x563835611070 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44490/0 (socket says 172.21.15.83:44490) 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2782381071 learned_addr learned my addr 172.21.15.83:0/2782381071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffac77fe640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 0x5638359c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.176+0000 7ffac7fff640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x5638359ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2782381071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 msgr2=0x5638359ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x5638359ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2782381071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 msgr2=0x5638359c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 0x5638359c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2782381071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638359b6b30 con 0x5638355f0390 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac77fe640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 0x5638359c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0a9eb0 tx=0x7ffabc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535cdd331e0a65d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffaad7fa640 1 -- 172.21.15.83:0/2782381071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0ad230 con 0x5638355f0390 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffaad7fa640 1 -- 172.21.15.83:0/2782381071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc0c7070 con 0x5638355f0390 2024-09-21T03:26:55.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.177+0000 7ffaad7fa640 1 -- 172.21.15.83:0/2782381071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0a9780 con 0x5638355f0390 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- 172.21.15.83:0/2782381071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 msgr2=0x5638359c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0a9eb0 tx=0x7ffabc013000 comp rx=0 tx=0).stop 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- 172.21.15.83:0/2782381071 shutdown_connections 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x5638359ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355f0390 0x5638359c8ce0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2782381071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355e81b0 0x5638359c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- 172.21.15.83:0/2782381071 >> 172.21.15.83:0/2782381071 conn(0x563835605460 msgr2=0x5638359c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- 172.21.15.83:0/2782381071 shutdown_connections 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- 172.21.15.83:0/2782381071 wait complete. 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.178+0000 7ffad40d5740 1 -- start start 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 Processor -- start 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x563835a504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81ae870 con 0x5638355e81b0 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81ae1a0 con 0x5638355f0390 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81ab400 con 0x563835611070 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffad40d5740 10 client.0 _ll_register_callbacks cb 0x7ffabc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36886/0 (socket says 172.21.15.83:36886) 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2766467439 learned_addr learned my addr 172.21.15.83:0/2766467439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.179+0000 7ffac7fff640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x563835a504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2766467439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 msgr2=0x563835a504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x563835a504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2766467439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 msgr2=0x5638359f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 -- 172.21.15.83:0/2766467439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563835a50a00 con 0x5638355f0390 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac7fff640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x563835a504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffac6ffd640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffabc096040 tx=0x7ffabc004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426abaa0505b1cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.180+0000 7ffaa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0a9780 con 0x5638355f0390 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7ffabc0a9230 con 0x5638355f0390 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffabc0c7070 con 0x5638355f0390 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ffabc0ad230 con 0x5638355f0390 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffad40d5740 10 client.26672 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563835a51c60 con 0x5638355f0390 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc0ad3f0 con 0x5638355f0390 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.181+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ffabc0c3a90 con 0x5638355f0390 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 10 client.26672 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 20 client.26672 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355a4da0 0x563835a54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 4 client.26672 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffabc0c7070 con 0x5638355a4da0 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.182+0000 7ffac77fe640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355a4da0 0x563835a54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffac77fe640 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355a4da0 0x563835a54090 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742961c1c0fe477a server_cookie=7ef7153e234cea0c in_seq=0 out_seq=0 2024-09-21T03:26:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffaa6ffd640 10 client.26672 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:55.188 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffaa6ffd640 1 -- 172.21.15.83:0/2766467439 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffabc0c7070 con 0x5638355a4da0 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffaa6ffd640 10 client.26672 handle_command_reply: tid=0 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 client.26672 shutdown 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 msgr2=0x5638359f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffabc096040 tx=0x7ffabc004950 comp rx=0 tx=0).stop 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 shutdown_connections 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5638355a4da0 0x563835a54090 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563835611070 0x563835a504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638355f0390 0x5638359f4ec0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffaa6ffd640 0 client.26672 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 --2- 172.21.15.83:0/2766467439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638355e81b0 0x5638359f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 >> 172.21.15.83:0/2766467439 conn(0x563835605460 msgr2=0x5638359c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 shutdown_connections 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 1 -- 172.21.15.83:0/2766467439 wait complete. 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.183+0000 7ffad40d5740 20 client.26672 trim_cache size 0 max 16384 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 msgr2=0x7ffab00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 0x7ffab00b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffab8000c00 tx=0x7ffab8031400 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 msgr2=0x7ffac819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e9f0 tx=0x7ffabc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 shutdown_connections 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ffab00b6700 0x7ffab00b8c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac810de30 0x7ffac81a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffac81052b0 0x7ffac819cc40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 --2- 172.21.15.83:0/1694086707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffac8104880 0x7ffac819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 >> 172.21.15.83:0/1694086707 conn(0x7ffac8079730 msgr2=0x7ffac8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 shutdown_connections 2024-09-21T03:26:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:55.184+0000 7ffacdb61640 1 -- 172.21.15.83:0/1694086707 wait complete. 2024-09-21T03:26:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:55 smithi142 ceph-mon[34512]: pgmap v640: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-09-21T03:26:55.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:55 smithi193 ceph-mon[35224]: pgmap v640: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-09-21T03:26:55.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:55 smithi083 ceph-mon[31219]: pgmap v640: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 686 op/s 2024-09-21T03:26:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 575.0 s, 1727.6 tps, lat 58.178 ms stddev 63.053 2024-09-21T03:26:57.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:26:57.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:26:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:57 smithi142 ceph-mon[34512]: pgmap v641: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-09-21T03:26:57.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:57 smithi193 ceph-mon[35224]: pgmap v641: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-09-21T03:26:57.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:57 smithi083 ceph-mon[31219]: pgmap v641: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-09-21T03:26:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 -- 172.21.15.83:0/2346183486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 msgr2=0x7f2d54103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 --2- 172.21.15.83:0/2346183486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54103fb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d4402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 -- 172.21.15.83:0/2346183486 shutdown_connections 2024-09-21T03:26:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 --2- 172.21.15.83:0/2346183486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 --2- 172.21.15.83:0/2346183486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 0x7f2d541098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 --2- 172.21.15.83:0/2346183486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54103fb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 -- 172.21.15.83:0/2346183486 >> 172.21.15.83:0/2346183486 conn(0x7f2d540ff630 msgr2=0x7f2d54101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 -- 172.21.15.83:0/2346183486 shutdown_connections 2024-09-21T03:26:58.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.507+0000 7f2d59395640 1 -- 172.21.15.83:0/2346183486 wait complete. 2024-09-21T03:26:58.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.510+0000 7f2d59395640 1 Processor -- start 2024-09-21T03:26:58.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.510+0000 7f2d59395640 1 -- start start 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.510+0000 7f2d59395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.510+0000 7f2d59395640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 0x7f2d54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d59395640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d59395640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54113af0 con 0x7f2d54103ba0 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d59395640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54113970 con 0x7f2d54109e30 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d59395640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54113c70 con 0x7f2d541094e0 2024-09-21T03:26:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36914/0 (socket says 172.21.15.83:36914) 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 -- 172.21.15.83:0/1145932498 learned_addr learned my addr 172.21.15.83:0/1145932498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d527fc640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 0x7f2d54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 -- 172.21.15.83:0/1145932498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 msgr2=0x7f2d54075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 0x7f2d54075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 -- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 msgr2=0x7f2d54077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.511+0000 7f2d537fe640 1 -- 172.21.15.83:0/1145932498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d54076470 con 0x7f2d54109e30 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.512+0000 7f2d537fe640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00eb10 tx=0x7f2d4c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.512+0000 7f2d3bfff640 1 -- 172.21.15.83:0/1145932498 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c004b20 con 0x7f2d54109e30 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.513+0000 7f2d3bfff640 1 -- 172.21.15.83:0/1145932498 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d4c004cc0 con 0x7f2d54109e30 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.513+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d541acea0 con 0x7f2d54109e30 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.513+0000 7f2d3bfff640 1 -- 172.21.15.83:0/1145932498 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c010670 con 0x7f2d54109e30 2024-09-21T03:26:58.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.513+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541ad300 con 0x7f2d54109e30 2024-09-21T03:26:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.515+0000 7f2d5f909740 1 Processor -- start 2024-09-21T03:26:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.515+0000 7f2d3bfff640 1 -- 172.21.15.83:0/1145932498 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2d4c0108f0 con 0x7f2d54109e30 2024-09-21T03:26:58.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.515+0000 7f2d5f909740 1 -- start start 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530c5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559530bd5e60 con 0x559530897670 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559530bc5290 con 0x55953088f490 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d5f909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559530bc90a0 con 0x5595308b8350 2024-09-21T03:26:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36926/0 (socket says 172.21.15.83:36926) 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 -- 172.21.15.83:0/709587 learned_addr learned my addr 172.21.15.83:0/709587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d527fc640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d527fc640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530c5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d3bfff640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 0x7f2d3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 -- 172.21.15.83:0/709587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 msgr2=0x559530c5f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530c5f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 -- 172.21.15.83:0/709587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 msgr2=0x559530c5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.516+0000 7f2d52ffd640 1 -- 172.21.15.83:0/709587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559530923930 con 0x55953088f490 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d537fe640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 0x7f2d3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d527fc640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530c5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d44032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2bb23ed6d7eb645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d397fa640 1 -- 172.21.15.83:0/709587 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44040070 con 0x55953088f490 2024-09-21T03:26:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d397fa640 1 -- 172.21.15.83:0/709587 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d44004030 con 0x55953088f490 2024-09-21T03:26:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d397fa640 1 -- 172.21.15.83:0/709587 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44035040 con 0x55953088f490 2024-09-21T03:26:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.517+0000 7f2d3bfff640 1 -- 172.21.15.83:0/1145932498 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2d4c0e7d30 con 0x7f2d54109e30 2024-09-21T03:26:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 -- 172.21.15.83:0/709587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 msgr2=0x559530c5f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d537fe640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 0x7f2d3c0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0e5580 tx=0x7f2d4c0a9b30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d44032b20 comp rx=0 tx=0).stop 2024-09-21T03:26:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 -- 172.21.15.83:0/709587 shutdown_connections 2024-09-21T03:26:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c5f970 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 --2- 172.21.15.83:0/709587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530c5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 -- 172.21.15.83:0/709587 >> 172.21.15.83:0/709587 conn(0x5595308ac740 msgr2=0x559530c5d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 -- 172.21.15.83:0/709587 shutdown_connections 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.518+0000 7f2d5f909740 1 -- 172.21.15.83:0/709587 wait complete. 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.519+0000 7f2d5f909740 1 -- start start 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.519+0000 7f2d5f909740 1 Processor -- start 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.519+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.519+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d5f909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559530a26890 con 0x5595308b8350 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d5f909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559530b5fd40 con 0x55953088f490 2024-09-21T03:26:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d5f909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559530c6dbb0 con 0x559530897670 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54086/0 (socket says 172.21.15.83:54086) 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 -- 172.21.15.83:0/290524251 learned_addr learned my addr 172.21.15.83:0/290524251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d527fc640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d5f909740 10 client.0 _ll_register_callbacks cb 0x559530bc3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d527fc640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 -- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 msgr2=0x559530c6b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c6b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 -- 172.21.15.83:0/290524251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 msgr2=0x559530c76ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c76ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 -- 172.21.15.83:0/290524251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559530c77490 con 0x559530897670 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402a930 tx=0x7f2d44005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59184800addeb547 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.520+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44035070 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2d440316a0 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d44004030 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d44040040 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 10 client.26650 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559530c79d80 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d44004030 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2d44004bd0 con 0x559530897670 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 10 client.26650 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 10 client.26650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.521+0000 7f2d5f909740 20 client.26650 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559530876cd0 0x559530c7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d5f909740 4 client.26650 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:26:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559530bd7db0 con 0x559530876cd0 2024-09-21T03:26:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d527fc640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559530876cd0 0x559530c7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d527fc640 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559530876cd0 0x559530c7d0e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b0bb3cc8e8bdc6 server_cookie=d31f30b77cf56497 in_seq=0 out_seq=0 2024-09-21T03:26:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.522+0000 7f2d36ff5640 10 client.26650 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.523+0000 7f2d36ff5640 1 -- 172.21.15.83:0/290524251 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559530bd7db0 con 0x559530876cd0 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.523+0000 7f2d36ff5640 10 client.26650 handle_command_reply: tid=0 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.523+0000 7f2d5f909740 1 client.26650 shutdown 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.524+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 msgr2=0x559530c6b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.524+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402a930 tx=0x7f2d44005e20 comp rx=0 tx=0).stop 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 shutdown_connections 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559530876cd0 0x559530c7d0e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c76ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c6b540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 --2- 172.21.15.83:0/290524251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 >> 172.21.15.83:0/290524251 conn(0x5595308ac740 msgr2=0x559530c625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.525+0000 7f2d36ff5640 0 client.26650 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.526+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 shutdown_connections 2024-09-21T03:26:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.526+0000 7f2d5f909740 1 -- 172.21.15.83:0/290524251 wait complete. 2024-09-21T03:26:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.526+0000 7f2d5f909740 20 client.26650 trim_cache size 0 max 16384 2024-09-21T03:26:58.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.553+0000 7f2d5f909740 1 Processor -- start 2024-09-21T03:26:58.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.553+0000 7f2d5f909740 1 -- start start 2024-09-21T03:26:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c64a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b5c40 con 0x5595308b8350 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b7780 con 0x55953088f490 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d5f909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b6310 con 0x559530897670 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d527fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d527fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:36956/0 (socket says 172.21.15.83:36956) 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d527fc640 1 -- 172.21.15.83:0/3278607057 learned_addr learned my addr 172.21.15.83:0/3278607057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.554+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.555+0000 7f2d527fc640 1 -- 172.21.15.83:0/3278607057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 msgr2=0x559530c644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.555+0000 7f2d527fc640 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.555+0000 7f2d527fc640 1 -- 172.21.15.83:0/3278607057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 msgr2=0x559530c64a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.555+0000 7f2d527fc640 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c64a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.555+0000 7f2d527fc640 1 -- 172.21.15.83:0/3278607057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559530c66690 con 0x55953088f490 2024-09-21T03:26:58.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.557+0000 7f2d527fc640 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d54108e70 tx=0x7f2d48005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec99a5a6043198d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.557+0000 7f2d38ff9640 1 -- 172.21.15.83:0/3278607057 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004050 con 0x55953088f490 2024-09-21T03:26:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.557+0000 7f2d38ff9640 1 -- 172.21.15.83:0/3278607057 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d48011dc0 con 0x55953088f490 2024-09-21T03:26:58.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.559+0000 7f2d5f909740 1 -- 172.21.15.83:0/3278607057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 msgr2=0x559530c76e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.559+0000 7f2d5f909740 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d54108e70 tx=0x7f2d48005f90 comp rx=0 tx=0).stop 2024-09-21T03:26:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.564+0000 7f2d5f909740 1 -- 172.21.15.83:0/3278607057 shutdown_connections 2024-09-21T03:26:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.564+0000 7f2d5f909740 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595308b8350 0x559530c64a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.564+0000 7f2d5f909740 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559530897670 0x559530c644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.564+0000 7f2d5f909740 1 --2- 172.21.15.83:0/3278607057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c76e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.564+0000 7f2d5f909740 1 -- 172.21.15.83:0/3278607057 >> 172.21.15.83:0/3278607057 conn(0x5595308ac740 msgr2=0x559530c69680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.565+0000 7f2d5f909740 1 -- 172.21.15.83:0/3278607057 shutdown_connections 2024-09-21T03:26:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.565+0000 7f2d5f909740 1 -- 172.21.15.83:0/3278607057 wait complete. 2024-09-21T03:26:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.566+0000 7f2d5f909740 1 -- start start 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.566+0000 7f2d5f909740 1 Processor -- start 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c7ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530ce0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b70b0 con 0x559530897670 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b69e0 con 0x55953088f490 2024-09-21T03:26:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541b3c40 con 0x5595308b8350 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.567+0000 7f2d5f909740 10 client.0 _ll_register_callbacks cb 0x7f2d480098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.568+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.568+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c7d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44568/0 (socket says 172.21.15.83:44568) 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.568+0000 7f2d52ffd640 1 -- 172.21.15.83:0/2981950071 learned_addr learned my addr 172.21.15.83:0/2981950071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.568+0000 7f2d527fc640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c7ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.568+0000 7f2d527fc640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530ce0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.569+0000 7f2d527fc640 1 -- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 msgr2=0x559530c7ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.569+0000 7f2d527fc640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c7ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.569+0000 7f2d527fc640 1 -- 172.21.15.83:0/2981950071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 msgr2=0x559530c7d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.569+0000 7f2d527fc640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c7d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.569+0000 7f2d527fc640 1 -- 172.21.15.83:0/2981950071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559530ce0e90 con 0x5595308b8350 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.570+0000 7f2d527fc640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530ce0820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d48012010 tx=0x7f2d48009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2091c7b70357049b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.570+0000 7f2d36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004230 con 0x5595308b8350 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d48011dc0 con 0x5595308b8350 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2d480043d0 con 0x5595308b8350 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d4805c020 con 0x5595308b8350 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d5f909740 10 client.26656 fetch_fsmap learned FSMap version 44 2024-09-21T03:26:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.571+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559530ce79a0 con 0x5595308b8350 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d48011dc0 con 0x5595308b8350 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f2d480129b0 con 0x5595308b8350 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 10 client.26656 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 20 client.26656 populate_metadata read hostname 'smithi083' 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55953084c080 0x559530ce9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 4 client.26656 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:26:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.572+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559530bd3860 con 0x55953084c080 2024-09-21T03:26:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.573+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55953084c080 0x559530ce9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.574+0000 7f2d52ffd640 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55953084c080 0x559530ce9ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff1fb228f78f752 server_cookie=d074ce000730f250 in_seq=0 out_seq=0 2024-09-21T03:26:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.574+0000 7f2d36ff5640 10 client.26656 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:58.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.576+0000 7f2d36ff5640 1 -- 172.21.15.83:0/2981950071 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559530bd3860 con 0x55953084c080 2024-09-21T03:26:58.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.576+0000 7f2d36ff5640 10 client.26656 handle_command_reply: tid=0 2024-09-21T03:26:58.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 client.26656 shutdown 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 msgr2=0x559530ce0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530ce0820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d48012010 tx=0x7f2d48009670 comp rx=0 tx=0).stop 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 shutdown_connections 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55953084c080 0x559530ce9ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595308b8350 0x559530ce0820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559530897670 0x559530c7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 --2- 172.21.15.83:0/2981950071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953088f490 0x559530c7ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 >> 172.21.15.83:0/2981950071 conn(0x5595308ac740 msgr2=0x559530c624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.577+0000 7f2d36ff5640 0 client.26656 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.578+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 shutdown_connections 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.578+0000 7f2d5f909740 1 -- 172.21.15.83:0/2981950071 wait complete. 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.578+0000 7f2d5f909740 20 client.26656 trim_cache size 0 max 16384 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:26:58.580 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:26:58.581 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "2338c509-a119-4e4c-8946-5b1a3b649fe8", 2024-09-21T03:26:58.581 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:26:58.581 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.578+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 msgr2=0x7f2d3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.578+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 0x7f2d3c0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0e5580 tx=0x7f2d4c0a9b30 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 msgr2=0x7f2d54075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00eb10 tx=0x7f2d4c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 shutdown_connections 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2d3c0b6680 0x7f2d3c0b8b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d54109e30 0x7f2d54075c50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d541094e0 0x7f2d54075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 --2- 172.21.15.83:0/1145932498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54103ba0 0x7f2d54077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 >> 172.21.15.83:0/1145932498 conn(0x7f2d540ff630 msgr2=0x7f2d54101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 shutdown_connections 2024-09-21T03:26:58.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:26:58.579+0000 7f2d59395640 1 -- 172.21.15.83:0/1145932498 wait complete. 2024-09-21T03:26:59.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:26:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: pgmap v642: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:26:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:26:59 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: pgmap v642: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:26:59 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: pgmap v642: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 567 op/s 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:26:59.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:26:59 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 -- 172.21.15.83:0/3098337672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 msgr2=0x7f814810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 --2- 172.21.15.83:0/3098337672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814810f4c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8140009cd0 tx=0x7f814002a970 comp rx=0 tx=0).stop 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 -- 172.21.15.83:0/3098337672 shutdown_connections 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 --2- 172.21.15.83:0/3098337672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814810f4c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 --2- 172.21.15.83:0/3098337672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81481032f0 0x7f8148103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 --2- 172.21.15.83:0/3098337672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8148108740 0x7f8148108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 -- 172.21.15.83:0/3098337672 >> 172.21.15.83:0/3098337672 conn(0x7f8148077800 msgr2=0x7f8148077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 -- 172.21.15.83:0/3098337672 shutdown_connections 2024-09-21T03:27:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.201+0000 7f814e157640 1 -- 172.21.15.83:0/3098337672 wait complete. 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 Processor -- start 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 -- start start 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81481032f0 0x7f814819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8148108740 0x7f814819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113a90 con 0x7f8148103c40 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113910 con 0x7f81481032f0 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f814e157640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113c10 con 0x7f8148108740 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f813ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f813ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47368/0 (socket says 172.21.15.83:47368) 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f813ffff640 1 -- 172.21.15.83:0/1259476290 learned_addr learned my addr 172.21.15.83:0/1259476290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f81477fe640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81481032f0 0x7f814819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.202+0000 7f8147fff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8148108740 0x7f814819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 -- 172.21.15.83:0/1259476290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8148108740 msgr2=0x7f814819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8148108740 0x7f814819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 -- 172.21.15.83:0/1259476290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81481032f0 msgr2=0x7f814819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81481032f0 0x7f814819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 -- 172.21.15.83:0/1259476290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81481a0430 con 0x7f8148103c40 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f813ffff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f813000e9f0 tx=0x7f813000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f81457fa640 1 -- 172.21.15.83:0/1259476290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813000cd80 con 0x7f8148103c40 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481a0720 con 0x7f8148103c40 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f81457fa640 1 -- 172.21.15.83:0/1259476290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81300054d0 con 0x7f8148103c40 2024-09-21T03:27:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f81457fa640 1 -- 172.21.15.83:0/1259476290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130010690 con 0x7f8148103c40 2024-09-21T03:27:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.203+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a6e80 con 0x7f8148103c40 2024-09-21T03:27:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.206+0000 7f81546cb740 1 Processor -- start 2024-09-21T03:27:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.206+0000 7f81457fa640 1 -- 172.21.15.83:0/1259476290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8130004e90 con 0x7f8148103c40 2024-09-21T03:27:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 -- start start 2024-09-21T03:27:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81457fa640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 0x7f81340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 0x5597e9ef6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 0x5597e9ef6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5597e9e6ce50 con 0x5597e9b2e390 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5597e9e5c280 con 0x5597e9b4f070 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f81546cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5597e9e60090 con 0x5597e9b261b0 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f813ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f813ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38468/0 (socket says 172.21.15.83:38468) 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.208+0000 7f813ffff640 1 -- 172.21.15.83:0/3350248663 learned_addr learned my addr 172.21.15.83:0/3350248663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.209+0000 7f8147fff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 0x7f81340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.209+0000 7f81477fe640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 0x5597e9ef6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.209+0000 7f8147fff640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 0x5597e9ef6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.209+0000 7f813ffff640 1 -- 172.21.15.83:0/3350248663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 msgr2=0x5597e9ef6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f81457fa640 1 -- 172.21.15.83:0/1259476290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f81300e7b00 con 0x7f8148103c40 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f813ffff640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 0x5597e9ef6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f813ffff640 1 -- 172.21.15.83:0/3350248663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 msgr2=0x5597e9ef6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f813ffff640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 0x5597e9ef6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f813ffff640 1 -- 172.21.15.83:0/3350248663 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e9bba650 con 0x5597e9b261b0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f81477fe640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 0x5597e9ef6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f8147fff640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 0x5597e9ef6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f813ffff640 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81300aa100 tx=0x7f8130013d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b92d7875cbd6413 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.210+0000 7f8147fff640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 0x7f81340b8c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8140004ab0 tx=0x7f81400099a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.211+0000 7f813dffb640 1 -- 172.21.15.83:0/3350248663 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300acba0 con 0x5597e9b261b0 2024-09-21T03:27:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.211+0000 7f813dffb640 1 -- 172.21.15.83:0/3350248663 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81300ad450 con 0x5597e9b261b0 2024-09-21T03:27:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.211+0000 7f813dffb640 1 -- 172.21.15.83:0/3350248663 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300acba0 con 0x5597e9b261b0 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 -- 172.21.15.83:0/3350248663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 msgr2=0x5597e9ef67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81300aa100 tx=0x7f8130013d80 comp rx=0 tx=0).stop 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 -- 172.21.15.83:0/3350248663 shutdown_connections 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b2e390 0x5597e9ef6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9ef67b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 --2- 172.21.15.83:0/3350248663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b4f070 0x5597e9ef6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 -- 172.21.15.83:0/3350248663 >> 172.21.15.83:0/3350248663 conn(0x5597e9b43460 msgr2=0x5597e9ef4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 -- 172.21.15.83:0/3350248663 shutdown_connections 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.212+0000 7f81546cb740 1 -- 172.21.15.83:0/3350248663 wait complete. 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.213+0000 7f81546cb740 1 -- start start 2024-09-21T03:27:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.213+0000 7f81546cb740 1 Processor -- start 2024-09-21T03:27:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5597e9cbd700 con 0x5597e9b4f070 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5597e9df7110 con 0x5597e9b2e390 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5597e9f0d1e0 con 0x5597e9b261b0 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:39116/0 (socket says 172.21.15.83:39116) 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f81546cb740 10 client.0 _ll_register_callbacks cb 0x5597e9e10990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 -- 172.21.15.83:0/2518522296 learned_addr learned my addr 172.21.15.83:0/2518522296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f8147fff640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 -- 172.21.15.83:0/2518522296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 msgr2=0x5597e9efd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 -- 172.21.15.83:0/2518522296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 msgr2=0x5597e9efe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.214+0000 7f813ffff640 1 -- 172.21.15.83:0/2518522296 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e9e24c40 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f813ffff640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f81300ae1f0 tx=0x7f81300ae0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e80ef7a4bd3570a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130005180 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81300ad450 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f81300b6ed0 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f81300b7360 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 10 client.26702 fetch_fsmap learned FSMap version 44 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5597e9f18e10 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81300ad450 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f81300b2080 con 0x5597e9b2e390 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 10 client.26702 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:27:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 10 client.26702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 20 client.26702 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9b0d9f0 0x5597e9f1c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 4 client.26702 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.215+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e9e6eda0 con 0x5597e9b0d9f0 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.216+0000 7f81477fe640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9b0d9f0 0x5597e9f1c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.216+0000 7f8126ffd640 10 client.26702 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.216+0000 7f81477fe640 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9b0d9f0 0x5597e9f1c480 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1b5a4d3520e215 server_cookie=7083b93b6ddeb95b in_seq=0 out_seq=0 2024-09-21T03:27:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f8126ffd640 1 -- 172.21.15.83:0/2518522296 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597e9e6eda0 con 0x5597e9b0d9f0 2024-09-21T03:27:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f8126ffd640 10 client.26702 handle_command_reply: tid=0 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 client.26702 shutdown 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 msgr2=0x5597e9efdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f81300ae1f0 tx=0x7f81300ae0b0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 shutdown_connections 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9b0d9f0 0x5597e9f1c480 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efdd50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 --2- 172.21.15.83:0/2518522296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 >> 172.21.15.83:0/2518522296 conn(0x5597e9b43460 msgr2=0x5597e9ef7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f8126ffd640 0 client.26702 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 shutdown_connections 2024-09-21T03:27:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.217+0000 7f81546cb740 1 -- 172.21.15.83:0/2518522296 wait complete. 2024-09-21T03:27:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.218+0000 7f81546cb740 20 client.26702 trim_cache size 0 max 16384 2024-09-21T03:27:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.258+0000 7f81546cb740 1 Processor -- start 2024-09-21T03:27:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.258+0000 7f81546cb740 1 -- start start 2024-09-21T03:27:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.258+0000 7f81546cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.258+0000 7f81546cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.258+0000 7f81546cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f81546cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81481ad3d0 con 0x5597e9b4f070 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f81546cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81481aef10 con 0x5597e9b2e390 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f81546cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81481adaa0 con 0x5597e9b261b0 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:39132/0 (socket says 172.21.15.83:39132) 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 -- 172.21.15.83:0/1267820690 learned_addr learned my addr 172.21.15.83:0/1267820690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f8147fff640 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 -- 172.21.15.83:0/1267820690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 msgr2=0x5597e9efc090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efc090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 -- 172.21.15.83:0/1267820690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 msgr2=0x5597e9efaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 -- 172.21.15.83:0/1267820690 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e9ef7090 con 0x5597e9b2e390 2024-09-21T03:27:00.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.259+0000 7f813ffff640 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f813000c6a0 tx=0x7f81300ae040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285517c933c567c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.263+0000 7f813d7fa640 1 -- 172.21.15.83:0/1267820690 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130005180 con 0x5597e9b2e390 2024-09-21T03:27:00.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.263+0000 7f813d7fa640 1 -- 172.21.15.83:0/1267820690 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8130098070 con 0x5597e9b2e390 2024-09-21T03:27:00.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.263+0000 7f813d7fa640 1 -- 172.21.15.83:0/1267820690 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300ad710 con 0x5597e9b2e390 2024-09-21T03:27:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 -- 172.21.15.83:0/1267820690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 msgr2=0x5597e9efc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f813000c6a0 tx=0x7f81300ae040 comp rx=0 tx=0).stop 2024-09-21T03:27:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 -- 172.21.15.83:0/1267820690 shutdown_connections 2024-09-21T03:27:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9efaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b2e390 0x5597e9efc5d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 --2- 172.21.15.83:0/1267820690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b261b0 0x5597e9efc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.265+0000 7f81546cb740 1 -- 172.21.15.83:0/1267820690 >> 172.21.15.83:0/1267820690 conn(0x5597e9b43460 msgr2=0x5597e9f0c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.269+0000 7f81546cb740 1 -- 172.21.15.83:0/1267820690 shutdown_connections 2024-09-21T03:27:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.269+0000 7f81546cb740 1 -- 172.21.15.83:0/1267820690 wait complete. 2024-09-21T03:27:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.269+0000 7f81546cb740 1 -- start start 2024-09-21T03:27:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.270+0000 7f81546cb740 1 Processor -- start 2024-09-21T03:27:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.270+0000 7f81546cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81546cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 0x5597e9f0fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:39148/0 (socket says 172.21.15.83:39148) 2024-09-21T03:27:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f813ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 0x5597e9f0fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f813ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 0x5597e9f0fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38486/0 (socket says 172.21.15.83:38486) 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81546cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9f84e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81477fe640 1 -- 172.21.15.83:0/852392355 learned_addr learned my addr 172.21.15.83:0/852392355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81481ae840 con 0x5597e9b4f070 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81481ae170 con 0x5597e9b261b0 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.271+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81481ab3d0 con 0x5597e9b2e390 2024-09-21T03:27:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.272+0000 7f81546cb740 10 client.0 _ll_register_callbacks cb 0x7f8130005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:00.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.274+0000 7f8147fff640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9f84e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.275+0000 7f81477fe640 1 -- 172.21.15.83:0/852392355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 msgr2=0x5597e9f0fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.275+0000 7f81477fe640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 0x5597e9f0fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.275+0000 7f81477fe640 1 -- 172.21.15.83:0/852392355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 msgr2=0x5597e9f84e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.275+0000 7f81477fe640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9f84e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.275+0000 7f81477fe640 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e9f85380 con 0x5597e9b261b0 2024-09-21T03:27:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.279+0000 7f81477fe640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8138005760 tx=0x7f8138010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faad403a9df69553 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:00.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.281+0000 7f8126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.284+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138009b60 con 0x5597e9b261b0 2024-09-21T03:27:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.284+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f8138005220 con 0x5597e9b261b0 2024-09-21T03:27:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.285+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8130098070 con 0x5597e9b261b0 2024-09-21T03:27:00.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.290+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8138005420 con 0x5597e9b261b0 2024-09-21T03:27:00.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.291+0000 7f81546cb740 10 client.26708 fetch_fsmap learned FSMap version 44 2024-09-21T03:27:00.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.291+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5597e9f865b0 con 0x5597e9b261b0 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f813801cb10 con 0x5597e9b261b0 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f8138004c70 con 0x5597e9b261b0 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 10 client.26708 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 10 client.26708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 20 client.26708 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9ae2da0 0x5597e9f88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 4 client.26708 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:27:00.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8130098070 con 0x5597e9ae2da0 2024-09-21T03:27:00.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.292+0000 7f813ffff640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9ae2da0 0x5597e9f88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:00.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.293+0000 7f8126ffd640 10 client.26708 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:00.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.293+0000 7f813ffff640 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9ae2da0 0x5597e9f88d80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84d5fec3e76380b server_cookie=2073ecbeb3a88f27 in_seq=0 out_seq=0 2024-09-21T03:27:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.293+0000 7f8126ffd640 1 -- 172.21.15.83:0/852392355 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x7f8130098070 con 0x5597e9ae2da0 2024-09-21T03:27:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.294+0000 7f8126ffd640 10 client.26708 handle_command_reply: tid=0 2024-09-21T03:27:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.294+0000 7f81546cb740 1 client.26708 shutdown 2024-09-21T03:27:00.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.298+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 msgr2=0x5597e9f0f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.298+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8138005760 tx=0x7f8138010800 comp rx=0 tx=0).stop 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 shutdown_connections 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5597e9ae2da0 0x5597e9f88d80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597e9b4f070 0x5597e9f84e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597e9b2e390 0x5597e9f0fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f8126ffd640 0 client.26708 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 --2- 172.21.15.83:0/852392355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5597e9b261b0 0x5597e9f0f870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.300+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 >> 172.21.15.83:0/852392355 conn(0x5597e9b43460 msgr2=0x5597e9efc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.301+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 shutdown_connections 2024-09-21T03:27:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.301+0000 7f81546cb740 1 -- 172.21.15.83:0/852392355 wait complete. 2024-09-21T03:27:00.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.301+0000 7f81546cb740 20 client.26708 trim_cache size 0 max 16384 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "2338c509-a119-4e4c-8946-5b1a3b649fe8", 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:27:00.304 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:27:00.305 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.306 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.308 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.309 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:00.312 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.303+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 msgr2=0x7f81340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.303+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 0x7f81340b8c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8140004ab0 tx=0x7f81400099a0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.303+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 msgr2=0x7f814819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.303+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f813000e9f0 tx=0x7f813000edf0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 shutdown_connections 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81340b6700 0x7f81340b8c10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8148108740 0x7f814819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8148103c40 0x7f814819aa60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 --2- 172.21.15.83:0/1259476290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81481032f0 0x7f814819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 >> 172.21.15.83:0/1259476290 conn(0x7f8148077800 msgr2=0x7f8148100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 shutdown_connections 2024-09-21T03:27:00.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:00.305+0000 7f814e157640 1 -- 172.21.15.83:0/1259476290 wait complete. 2024-09-21T03:27:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:00.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:00.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:00 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:00.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:00 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:00.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:00 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 580.0 s, 1450.2 tps, lat 68.275 ms stddev 73.979 2024-09-21T03:27:01.100 INFO:tasks.cephfs.filesystem:scrub status for tag:2338c509-a119-4e4c-8946-5b1a3b649fe8 - {'path': '/', 'tag': '2338c509-a119-4e4c-8946-5b1a3b649fe8', 'options': 'recursive,force'} 2024-09-21T03:27:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:01 smithi142 ceph-mon[34512]: pgmap v643: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-09-21T03:27:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:01 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:01 smithi193 ceph-mon[35224]: pgmap v643: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-09-21T03:27:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:01 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:01.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:01 smithi083 ceph-mon[31219]: pgmap v643: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 713 op/s 2024-09-21T03:27:01.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:01 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:02.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:02 smithi083 ceph-mon[31219]: pgmap v644: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T03:27:02.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:02.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:02 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:02.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:02 smithi142 ceph-mon[34512]: pgmap v644: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T03:27:02.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:02.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:02 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:03.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:02 smithi193 ceph-mon[35224]: pgmap v644: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 572 op/s 2024-09-21T03:27:03.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:03.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:02 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:04.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:03 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:04.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:03 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:27:04.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:03 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:27:05.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:05 smithi142 ceph-mon[34512]: pgmap v645: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T03:27:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:05 smithi193 ceph-mon[35224]: pgmap v645: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T03:27:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:05 smithi083 ceph-mon[31219]: pgmap v645: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T03:27:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 585.0 s, 226.0 tps, lat 418.516 ms stddev 244.638 2024-09-21T03:27:07.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:07 smithi142 ceph-mon[34512]: pgmap v646: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-09-21T03:27:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:07 smithi193 ceph-mon[35224]: pgmap v646: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-09-21T03:27:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:07 smithi083 ceph-mon[31219]: pgmap v646: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 524 op/s 2024-09-21T03:27:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:09 smithi142 ceph-mon[34512]: pgmap v647: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 528 op/s 2024-09-21T03:27:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:09 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:09 smithi193 ceph-mon[35224]: pgmap v647: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 528 op/s 2024-09-21T03:27:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:09 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:09 smithi083 ceph-mon[31219]: pgmap v647: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 528 op/s 2024-09-21T03:27:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:09 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:10 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:10 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:10.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:10 smithi142 ceph-mon[34512]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:27:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:10 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:10 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:10 smithi193 ceph-mon[35224]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:27:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:10 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:10.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:10 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:10.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:10 smithi083 ceph-mon[31219]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:27:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 590.0 s, 534.8 tps, lat 194.641 ms stddev 319.155 2024-09-21T03:27:11.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:11 smithi142 ceph-mon[34512]: pgmap v649: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-09-21T03:27:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:11 smithi193 ceph-mon[35224]: pgmap v649: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-09-21T03:27:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:11 smithi083 ceph-mon[31219]: pgmap v649: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-09-21T03:27:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:13 smithi142 ceph-mon[34512]: pgmap v650: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-21T03:27:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:13 smithi193 ceph-mon[35224]: pgmap v650: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-21T03:27:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:13 smithi083 ceph-mon[31219]: pgmap v650: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-21T03:27:15.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:15 smithi142 ceph-mon[34512]: pgmap v651: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.71k op/s 2024-09-21T03:27:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:15 smithi193 ceph-mon[35224]: pgmap v651: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.71k op/s 2024-09-21T03:27:15.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:15 smithi083 ceph-mon[31219]: pgmap v651: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 MiB/s wr, 1.71k op/s 2024-09-21T03:27:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 595.0 s, 1565.4 tps, lat 65.553 ms stddev 84.400 2024-09-21T03:27:17.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:17 smithi193 ceph-mon[35224]: pgmap v652: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-09-21T03:27:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:17 smithi083 ceph-mon[31219]: pgmap v652: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-09-21T03:27:17.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:17 smithi142 ceph-mon[34512]: pgmap v652: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-09-21T03:27:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:19 smithi193 ceph-mon[35224]: pgmap v653: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-09-21T03:27:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:19 smithi193 ceph-mon[35224]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:27:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:19 smithi083 ceph-mon[31219]: pgmap v653: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-09-21T03:27:19.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:19 smithi083 ceph-mon[31219]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:27:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:19 smithi142 ceph-mon[34512]: pgmap v653: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-09-21T03:27:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:19 smithi142 ceph-mon[34512]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:27:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:20 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:20.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:20 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:20 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:27:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 600.0 s, 2052.8 tps, lat 48.590 ms stddev 56.068 2024-09-21T03:27:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:21 smithi193 ceph-mon[35224]: pgmap v655: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T03:27:21.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:21 smithi083 ceph-mon[31219]: pgmap v655: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T03:27:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:21 smithi142 ceph-mon[34512]: pgmap v655: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T03:27:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:23 smithi193 ceph-mon[35224]: pgmap v656: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-09-21T03:27:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:23 smithi083 ceph-mon[31219]: pgmap v656: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-09-21T03:27:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:23 smithi142 ceph-mon[34512]: pgmap v656: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-09-21T03:27:25.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:25 smithi193 ceph-mon[35224]: pgmap v657: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 650 op/s 2024-09-21T03:27:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:25 smithi083 ceph-mon[31219]: pgmap v657: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 650 op/s 2024-09-21T03:27:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:25 smithi142 ceph-mon[34512]: pgmap v657: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 650 op/s 2024-09-21T03:27:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 605.0 s, 1984.4 tps, lat 50.625 ms stddev 59.365 2024-09-21T03:27:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:27 smithi193 ceph-mon[35224]: pgmap v658: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.4 MiB/s wr, 651 op/s 2024-09-21T03:27:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:27 smithi083 ceph-mon[31219]: pgmap v658: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.4 MiB/s wr, 651 op/s 2024-09-21T03:27:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:27 smithi142 ceph-mon[34512]: pgmap v658: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.4 MiB/s wr, 651 op/s 2024-09-21T03:27:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:29 smithi193 ceph-mon[35224]: pgmap v659: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-21T03:27:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:29 smithi083 ceph-mon[31219]: pgmap v659: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-21T03:27:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:29 smithi142 ceph-mon[34512]: pgmap v659: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-21T03:27:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 610.0 s, 2385.4 tps, lat 41.121 ms stddev 47.728 2024-09-21T03:27:31.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:27:31.436 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:31.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:31 smithi083 ceph-mon[31219]: pgmap v660: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-21T03:27:31.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:31 smithi193 ceph-mon[35224]: pgmap v660: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-21T03:27:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:31 smithi142 ceph-mon[34512]: pgmap v660: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 -- 172.21.15.83:0/188904822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0103180 msgr2=0x7faae0103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 --2- 172.21.15.83:0/188904822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0103180 0x7faae0103590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faad0009900 tx=0x7faad002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 -- 172.21.15.83:0/188904822 shutdown_connections 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 --2- 172.21.15.83:0/188904822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0108d00 0x7faae010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 --2- 172.21.15.83:0/188904822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae01083b0 0x7faae01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 --2- 172.21.15.83:0/188904822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae0103180 0x7faae0103590 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 -- 172.21.15.83:0/188904822 >> 172.21.15.83:0/188904822 conn(0x7faae0077c60 msgr2=0x7faae00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 -- 172.21.15.83:0/188904822 shutdown_connections 2024-09-21T03:27:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.001+0000 7faae66aa640 1 -- 172.21.15.83:0/188904822 wait complete. 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 Processor -- start 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 -- start start 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 0x7faae019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 0x7faae019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 0x7faae019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faae01129c0 con 0x7faae01083b0 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faae0112840 con 0x7faae0108d00 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.002+0000 7faae66aa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faae0112b40 con 0x7faae0103180 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 0x7faae019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 0x7faae019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 0x7faae019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34696/0 (socket says 172.21.15.83:34696) 2024-09-21T03:27:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadf7fe640 1 -- 172.21.15.83:0/2819712376 learned_addr learned my addr 172.21.15.83:0/2819712376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faae4c47640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 0x7faae019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 -- 172.21.15.83:0/2819712376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 msgr2=0x7faae019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 0x7faae019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 -- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 msgr2=0x7faae019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 0x7faae019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 -- 172.21.15.83:0/2819712376 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae01a6620 con 0x7faae0103180 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faae4c47640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 0x7faae019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadffff640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 0x7faae019b390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad002a930 tx=0x7faad00045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadd7fa640 1 -- 172.21.15.83:0/2819712376 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0035020 con 0x7faae0103180 2024-09-21T03:27:32.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faadd7fa640 1 -- 172.21.15.83:0/2819712376 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad0042070 con 0x7faae0103180 2024-09-21T03:27:32.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.003+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae0199180 con 0x7faae0103180 2024-09-21T03:27:32.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.004+0000 7faadd7fa640 1 -- 172.21.15.83:0/2819712376 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0031930 con 0x7faae0103180 2024-09-21T03:27:32.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.004+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01996b0 con 0x7faae0103180 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 Processor -- start 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 -- start start 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad44650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad44b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadd7fa640 1 -- 172.21.15.83:0/2819712376 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7faad0031ad0 con 0x7faae0103180 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5625dacbae50 con 0x5625da99d070 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5625dacaa280 con 0x5625da97c390 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faaecc1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625dacae090 con 0x5625da9741b0 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55914/0 (socket says 172.21.15.83:55914) 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 -- 172.21.15.83:0/869556953 learned_addr learned my addr 172.21.15.83:0/869556953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faae4c47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadffff640 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 -- 172.21.15.83:0/869556953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 msgr2=0x5625dad44b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad44b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 -- 172.21.15.83:0/869556953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 msgr2=0x5625dad44650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.007+0000 7faadf7fe640 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad44650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faadf7fe640 1 -- 172.21.15.83:0/869556953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625daa08650 con 0x5625da97c390 2024-09-21T03:27:32.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faadf7fe640 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d00 tx=0x7faad4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09462a83963e300 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faac67fc640 1 -- 172.21.15.83:0/869556953 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad401e070 con 0x5625da97c390 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faac67fc640 1 -- 172.21.15.83:0/869556953 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad4019ae0 con 0x5625da97c390 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faac67fc640 1 -- 172.21.15.83:0/869556953 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400e950 con 0x5625da97c390 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faadd7fa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 0x7faac80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faae4c47640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 0x7faac80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.008+0000 7faadd7fa640 1 -- 172.21.15.83:0/2819712376 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7faad010c080 con 0x7faae0103180 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 -- 172.21.15.83:0/869556953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 msgr2=0x5625dad450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d00 tx=0x7faad4011810 comp rx=0 tx=0).stop 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faae4c47640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 0x7faac80b8c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faad8009fd0 tx=0x7faad8009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 -- 172.21.15.83:0/869556953 shutdown_connections 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad450d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 --2- 172.21.15.83:0/869556953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad44650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 -- 172.21.15.83:0/869556953 >> 172.21.15.83:0/869556953 conn(0x5625da991460 msgr2=0x5625dad42540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 -- 172.21.15.83:0/869556953 shutdown_connections 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.009+0000 7faaecc1e740 1 -- 172.21.15.83:0/869556953 wait complete. 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.010+0000 7faaecc1e740 1 -- start start 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.010+0000 7faaecc1e740 1 Processor -- start 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.011+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad58ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.011+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad59280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad5e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faadffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad58ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad59280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad59280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55930/0 (socket says 172.21.15.83:55930) 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faaecc1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5625dab0b700 con 0x5625da9741b0 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faaecc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5625dac45110 con 0x5625da97c390 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faaecc1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625dad52e30 con 0x5625da99d070 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faaecc1e740 10 client.0 _ll_register_callbacks cb 0x5625dac5e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faadffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad58ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34718/0 (socket says 172.21.15.83:34718) 2024-09-21T03:27:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faadffff640 1 -- 172.21.15.83:0/2945661239 learned_addr learned my addr 172.21.15.83:0/2945661239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.012+0000 7faae4c47640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad5e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 -- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 msgr2=0x5625dad59280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad59280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 -- 172.21.15.83:0/2945661239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 msgr2=0x5625dad58ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad58ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 -- 172.21.15.83:0/2945661239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625dac72c40 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faae4c47640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad5e2a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faad8003f20 tx=0x7faad8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757143a2685b752f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.013+0000 7faabf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad800fe60 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faad4019ae0 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7faad80047d0 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faad8012020 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 10 client.26680 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5625dad672a0 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad4019ae0 con 0x5625da99d070 2024-09-21T03:27:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7faad8027070 con 0x5625da99d070 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 10 client.26680 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 20 client.26680 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da95b9f0 0x5625dad6ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 4 client.26680 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.014+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625dacbcda0 con 0x5625da95b9f0 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.015+0000 7faadffff640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da95b9f0 0x5625dad6ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.015+0000 7faadffff640 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da95b9f0 0x5625dad6ad60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a250887a21696a server_cookie=590cce694b5508f0 in_seq=0 out_seq=0 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.015+0000 7faabf7fe640 10 client.26680 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faabf7fe640 1 -- 172.21.15.83:0/2945661239 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625dacbcda0 con 0x5625da95b9f0 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faabf7fe640 10 client.26680 handle_command_reply: tid=0 2024-09-21T03:27:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faaecc1e740 1 client.26680 shutdown 2024-09-21T03:27:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 msgr2=0x5625dad5e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad5e2a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faad8003f20 tx=0x7faad8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 shutdown_connections 2024-09-21T03:27:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.016+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da95b9f0 0x5625dad6ad60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad5e2a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 --2- 172.21.15.83:0/2945661239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad58ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 >> 172.21.15.83:0/2945661239 conn(0x5625da991460 msgr2=0x5625dad47830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faabf7fe640 0 client.26680 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 shutdown_connections 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 1 -- 172.21.15.83:0/2945661239 wait complete. 2024-09-21T03:27:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.017+0000 7faaecc1e740 20 client.26680 trim_cache size 0 max 16384 2024-09-21T03:27:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 Processor -- start 2024-09-21T03:27:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 -- start start 2024-09-21T03:27:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad5e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faae01ad3b0 con 0x5625da99d070 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faae01aef50 con 0x5625da97c390 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faaecc1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faae01ada80 con 0x5625da9741b0 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.040+0000 7faae4c47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.041+0000 7faadffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.041+0000 7faadffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41348/0 (socket says 172.21.15.83:41348) 2024-09-21T03:27:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.041+0000 7faadffff640 1 -- 172.21.15.83:0/3570468550 learned_addr learned my addr 172.21.15.83:0/3570468550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadf7fe640 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad5e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadffff640 1 -- 172.21.15.83:0/3570468550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 msgr2=0x5625dad5e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadffff640 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad5e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadffff640 1 -- 172.21.15.83:0/3570468550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 msgr2=0x5625dad5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadffff640 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadffff640 1 -- 172.21.15.83:0/3570468550 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625dad54330 con 0x5625da9741b0 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.042+0000 7faadf7fe640 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad5e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.043+0000 7faadffff640 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faad00ce590 tx=0x7faad010e600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deeea5eb8fe72f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.043+0000 7faac5ffb640 1 -- 172.21.15.83:0/3570468550 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00d1910 con 0x5625da9741b0 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.043+0000 7faac5ffb640 1 -- 172.21.15.83:0/3570468550 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad00d1ad0 con 0x5625da9741b0 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.043+0000 7faac5ffb640 1 -- 172.21.15.83:0/3570468550 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00d1730 con 0x5625da9741b0 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 -- 172.21.15.83:0/3570468550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 msgr2=0x5625dad3d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faad00ce590 tx=0x7faad010e600 comp rx=0 tx=0).stop 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 -- 172.21.15.83:0/3570468550 shutdown_connections 2024-09-21T03:27:32.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da99d070 0x5625dad5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad5e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 --2- 172.21.15.83:0/3570468550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da9741b0 0x5625dad3d280 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.044+0000 7faaecc1e740 1 -- 172.21.15.83:0/3570468550 >> 172.21.15.83:0/3570468550 conn(0x5625da991460 msgr2=0x5625dad53040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.045+0000 7faaecc1e740 1 -- 172.21.15.83:0/3570468550 shutdown_connections 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.045+0000 7faaecc1e740 1 -- 172.21.15.83:0/3570468550 wait complete. 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.045+0000 7faaecc1e740 1 -- start start 2024-09-21T03:27:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.045+0000 7faaecc1e740 1 Processor -- start 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faae01ae820 con 0x5625da9741b0 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faae01ae150 con 0x5625da97c390 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faae01ab3b0 con 0x5625da99d070 2024-09-21T03:27:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34738/0 (socket says 172.21.15.83:34738) 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadffff640 1 -- 172.21.15.83:0/1762538290 learned_addr learned my addr 172.21.15.83:0/1762538290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faaecc1e740 10 client.0 _ll_register_callbacks cb 0x7faad8003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faae4c47640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadf7fe640 1 -- 172.21.15.83:0/1762538290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 msgr2=0x5625dad632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.046+0000 7faadf7fe640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faadf7fe640 1 -- 172.21.15.83:0/1762538290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 msgr2=0x5625dad64610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faadf7fe640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faadf7fe640 1 -- 172.21.15.83:0/1762538290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625dad638d0 con 0x5625da97c390 2024-09-21T03:27:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faadffff640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faadf7fe640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad64b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad4005890 tx=0x7faad4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aada8f463a80f8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faabf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4028020 con 0x5625da97c390 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7faad4019840 con 0x5625da97c390 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faad00d1ad0 con 0x5625da97c390 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faad4033050 con 0x5625da97c390 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faaecc1e740 10 client.26723 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5625dadd5f40 con 0x5625da97c390 2024-09-21T03:27:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.047+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad00d1ad0 con 0x5625da97c390 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7faad4003ca0 con 0x5625da97c390 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 10 client.26723 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 20 client.26723 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da930da0 0x5625dadd87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 4 client.26723 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625dacb8850 con 0x5625da930da0 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faadffff640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da930da0 0x5625dadd87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faabf7fe640 10 client.26723 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.048+0000 7faadffff640 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da930da0 0x5625dadd87e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143702cefe571339 server_cookie=78ad167c748bf984 in_seq=0 out_seq=0 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faabf7fe640 1 -- 172.21.15.83:0/1762538290 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x5625dacb8850 con 0x5625da930da0 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faabf7fe640 10 client.26723 handle_command_reply: tid=0 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 client.26723 shutdown 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 msgr2=0x5625dad64b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad64b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad4005890 tx=0x7faad4005720 comp rx=0 tx=0).stop 2024-09-21T03:27:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 shutdown_connections 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5625da930da0 0x5625dadd87e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625da99d070 0x5625dad632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625da97c390 0x5625dad64b70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 --2- 172.21.15.83:0/1762538290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625da9741b0 0x5625dad64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 >> 172.21.15.83:0/1762538290 conn(0x5625da991460 msgr2=0x5625dad53040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faabf7fe640 0 client.26723 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 shutdown_connections 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 1 -- 172.21.15.83:0/1762538290 wait complete. 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.049+0000 7faaecc1e740 20 client.26723 trim_cache size 0 max 16384 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:27:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:27:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:27:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:27:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 msgr2=0x7faac80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 0x7faac80b8c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faad8009fd0 tx=0x7faad8009500 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 msgr2=0x7faae019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 0x7faae019b390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad002a930 tx=0x7faad00045f0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 shutdown_connections 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7faac80b6730 0x7faac80b8c40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae0108d00 0x7faae019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae01083b0 0x7faae019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 --2- 172.21.15.83:0/2819712376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faae0103180 0x7faae019b390 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad002a930 tx=0x7faad00045f0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.050+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 >> 172.21.15.83:0/2819712376 conn(0x7faae0077c60 msgr2=0x7faae0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.051+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 shutdown_connections 2024-09-21T03:27:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:32.052+0000 7faae66aa640 1 -- 172.21.15.83:0/2819712376 wait complete. 2024-09-21T03:27:32.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:27:32.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:27:33.359 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:33.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:33 smithi083 ceph-mon[31219]: pgmap v661: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 593 op/s 2024-09-21T03:27:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:33 smithi193 ceph-mon[35224]: pgmap v661: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 593 op/s 2024-09-21T03:27:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:33 smithi142 ceph-mon[34512]: pgmap v661: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 593 op/s 2024-09-21T03:27:33.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.924+0000 7f2330ecd640 1 -- 172.21.15.83:0/1988115252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c108cf0 msgr2=0x7f232c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:33.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.924+0000 7f2330ecd640 1 --2- 172.21.15.83:0/1988115252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c108cf0 0x7f232c10f520 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2324009ca0 tx=0x7f232402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.925+0000 7f2330ecd640 1 -- 172.21.15.83:0/1988115252 shutdown_connections 2024-09-21T03:27:33.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.925+0000 7f2330ecd640 1 --2- 172.21.15.83:0/1988115252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c108cf0 0x7f232c10f520 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.925+0000 7f2330ecd640 1 --2- 172.21.15.83:0/1988115252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c1083a0 0x7f232c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.925+0000 7f2330ecd640 1 --2- 172.21.15.83:0/1988115252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.925+0000 7f2330ecd640 1 -- 172.21.15.83:0/1988115252 >> 172.21.15.83:0/1988115252 conn(0x7f232c079730 msgr2=0x7f232c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.926+0000 7f2330ecd640 1 -- 172.21.15.83:0/1988115252 shutdown_connections 2024-09-21T03:27:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.926+0000 7f2330ecd640 1 -- 172.21.15.83:0/1988115252 wait complete. 2024-09-21T03:27:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 Processor -- start 2024-09-21T03:27:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 -- start start 2024-09-21T03:27:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c108cf0 0x7f232c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112960 con 0x7f232c1083a0 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1127e0 con 0x7f232c102a60 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.927+0000 7f2330ecd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112ae0 con 0x7f232c108cf0 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f2323fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c108cf0 0x7f232c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f2323fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34750/0 (socket says 172.21.15.83:34750) 2024-09-21T03:27:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f2323fff640 1 -- 172.21.15.83:0/508225919 learned_addr learned my addr 172.21.15.83:0/508225919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 -- 172.21.15.83:0/508225919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c108cf0 msgr2=0x7f232c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c108cf0 0x7f232c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 -- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 msgr2=0x7f232c0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 -- 172.21.15.83:0/508225919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1003d0 con 0x7f232c102a60 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f2323fff640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.928+0000 7f232a59c640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c101080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e9f0 tx=0x7f231c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:33.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.929+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c00ccc0 con 0x7f232c102a60 2024-09-21T03:27:33.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.929+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c0108e0 con 0x7f232c102a60 2024-09-21T03:27:33.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.929+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1acc80 con 0x7f232c102a60 2024-09-21T03:27:33.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.929+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1ad1c0 con 0x7f232c102a60 2024-09-21T03:27:33.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.929+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c018880 con 0x7f232c102a60 2024-09-21T03:27:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.930+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c100970 con 0x7f232c102a60 2024-09-21T03:27:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.932+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f231c010430 con 0x7f232c102a60 2024-09-21T03:27:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.932+0000 7f23237fe640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 0x7f23180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.933+0000 7f2323fff640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 0x7f23180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.933+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f231c0e7990 con 0x7f232c102a60 2024-09-21T03:27:33.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.935+0000 7f2323fff640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 0x7f23180b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f232c069ff0 tx=0x7f2314005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:33.937+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f231c0b2310 con 0x7f232c102a60 2024-09-21T03:27:34.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.288+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f232c02e8d0 con 0x7f232c102a60 2024-09-21T03:27:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.290+0000 7f23237fe640 1 -- 172.21.15.83:0/508225919 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11985 (secure 0 0 0) 0x7f231c0b7250 con 0x7f232c102a60 2024-09-21T03:27:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:27:34.292 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"btime":"2024-09-21T03:27:19:181587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:27:18.256865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:27:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.300+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 msgr2=0x7f23180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.300+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 0x7f23180b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f232c069ff0 tx=0x7f2314005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.300+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 msgr2=0x7f232c101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.300+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c101080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e9f0 tx=0x7f231c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 shutdown_connections 2024-09-21T03:27:34.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f23180b6700 0x7f23180b8c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f232c108cf0 0x7f232c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f232c1083a0 0x7f232c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 --2- 172.21.15.83:0/508225919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f232c102a60 0x7f232c101080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.302+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 >> 172.21.15.83:0/508225919 conn(0x7f232c079730 msgr2=0x7f232c06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:34.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.306+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 shutdown_connections 2024-09-21T03:27:34.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:34.306+0000 7f2330ecd640 1 -- 172.21.15.83:0/508225919 wait complete. 2024-09-21T03:27:35.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:27:35.431 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:35.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:35 smithi083 ceph-mon[31219]: pgmap v662: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-21T03:27:35.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:35 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/508225919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:35 smithi193 ceph-mon[35224]: pgmap v662: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-21T03:27:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:35 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/508225919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:35 smithi142 ceph-mon[34512]: pgmap v662: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-21T03:27:35.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:35 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/508225919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 615.0 s, 1765.4 tps, lat 57.447 ms stddev 65.244 2024-09-21T03:27:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 -- 172.21.15.83:0/1725889301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc101f70 msgr2=0x7f33bc102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 --2- 172.21.15.83:0/1725889301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc101f70 0x7f33bc102380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009900 tx=0x7f33b802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 -- 172.21.15.83:0/1725889301 shutdown_connections 2024-09-21T03:27:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 --2- 172.21.15.83:0/1725889301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc109e00 0x7f33bc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 --2- 172.21.15.83:0/1725889301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc1094b0 0x7f33bc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 --2- 172.21.15.83:0/1725889301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc101f70 0x7f33bc102380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 -- 172.21.15.83:0/1725889301 >> 172.21.15.83:0/1725889301 conn(0x7f33bc077c60 msgr2=0x7f33bc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 -- 172.21.15.83:0/1725889301 shutdown_connections 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.859+0000 7f33c34cf640 1 -- 172.21.15.83:0/1725889301 wait complete. 2024-09-21T03:27:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 Processor -- start 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 -- start start 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 0x7f33bc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 0x7f33bc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc113ac0 con 0x7f33bc101f70 2024-09-21T03:27:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc113940 con 0x7f33bc109e00 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c34cf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc113c40 con 0x7f33bc1094b0 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34772/0 (socket says 172.21.15.83:34772) 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c126b640 1 -- 172.21.15.83:0/3911499206 learned_addr learned my addr 172.21.15.83:0/3911499206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 0x7f33bc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.860+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 0x7f33bc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 -- 172.21.15.83:0/3911499206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 msgr2=0x7f33bc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 0x7f33bc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 -- 172.21.15.83:0/3911499206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 msgr2=0x7f33bc19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 0x7f33bc19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 -- 172.21.15.83:0/3911499206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1a0480 con 0x7f33bc101f70 2024-09-21T03:27:36.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 0x7f33bc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 0x7f33bc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c126b640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005940 tx=0x7f33b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33b27fc640 1 -- 172.21.15.83:0/3911499206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b803f030 con 0x7f33bc101f70 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc1a0710 con 0x7f33bc101f70 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33b27fc640 1 -- 172.21.15.83:0/3911499206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b8003cb0 con 0x7f33bc101f70 2024-09-21T03:27:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33b27fc640 1 -- 172.21.15.83:0/3911499206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8035070 con 0x7f33bc101f70 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.861+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1a6ed0 con 0x7f33bc101f70 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.864+0000 7f33c9a43740 1 Processor -- start 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 -- start start 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5565a91d3ea0 con 0x5565a8eb6070 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5565a91c32d0 con 0x5565a8e95390 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c9a43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5565a91c70e0 con 0x5565a8e8d1b0 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34786/0 (socket says 172.21.15.83:34786) 2024-09-21T03:27:36.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 -- 172.21.15.83:0/2074054271 learned_addr learned my addr 172.21.15.83:0/2074054271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 -- 172.21.15.83:0/2074054271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 msgr2=0x5565a925d800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 -- 172.21.15.83:0/2074054271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 msgr2=0x5565a925dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.865+0000 7f33c1a6c640 1 -- 172.21.15.83:0/2074054271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a8f21650 con 0x5565a8eb6070 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.866+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f33b4010710 tx=0x7f33b4010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c745563a170e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.866+0000 7f33a77fe640 1 -- 172.21.15.83:0/2074054271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b400e8d0 con 0x5565a8eb6070 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.866+0000 7f33a77fe640 1 -- 172.21.15.83:0/2074054271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b4012450 con 0x5565a8eb6070 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.866+0000 7f33b27fc640 1 -- 172.21.15.83:0/3911499206 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f33b8042030 con 0x7f33bc101f70 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.866+0000 7f33a77fe640 1 -- 172.21.15.83:0/2074054271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4011640 con 0x5565a8eb6070 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- 172.21.15.83:0/2074054271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 msgr2=0x5565a925d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f33b4010710 tx=0x7f33b4010bd0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- 172.21.15.83:0/2074054271 shutdown_connections 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 --2- 172.21.15.83:0/2074054271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8eb6070 0x5565a925d2c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- 172.21.15.83:0/2074054271 >> 172.21.15.83:0/2074054271 conn(0x5565a8eaa460 msgr2=0x5565a925b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- 172.21.15.83:0/2074054271 shutdown_connections 2024-09-21T03:27:36.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- 172.21.15.83:0/2074054271 wait complete. 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 -- start start 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33b27fc640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 0x7f33a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c9a43740 1 Processor -- start 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 0x7f33a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.867+0000 7f33b27fc640 1 -- 172.21.15.83:0/3911499206 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f33b8107f20 con 0x7f33bc101f70 2024-09-21T03:27:36.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 0x7f33a80b8aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33b40104d0 tx=0x7f33b4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a9268e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8eb6070 0x5565a9269380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5565a9024700 con 0x5565a8e8d1b0 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5565a915e160 con 0x5565a8eb6070 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5565a926baf0 con 0x5565a8e95390 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c9a43740 10 client.0 _ll_register_callbacks cb 0x5565a91779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:36.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34794/0 (socket says 172.21.15.83:34794) 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c126b640 1 -- 172.21.15.83:0/381670371 learned_addr learned my addr 172.21.15.83:0/381670371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a9268e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.868+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8eb6070 0x5565a9269380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 -- 172.21.15.83:0/381670371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8eb6070 msgr2=0x5565a9269380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8eb6070 0x5565a9269380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 -- 172.21.15.83:0/381670371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 msgr2=0x5565a9268900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 -- 172.21.15.83:0/381670371 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a918bc90 con 0x5565a8e95390 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c126b640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a9268e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005d00 tx=0x7f33ac011bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0455e63335d57bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac010c30 con 0x5565a8e95390 2024-09-21T03:27:36.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f33ac010dd0 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b4012450 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f33ac004020 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c9a43740 10 client.26701 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5565a9277730 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.869+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b4012450 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f33ac01a410 con 0x5565a8e95390 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 10 client.26701 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 10 client.26701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 20 client.26701 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8e749f0 0x5565a927ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 4 client.26701 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a91d5df0 con 0x5565a8e749f0 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c126b640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8e749f0 0x5565a927ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33a4ff9640 10 client.26701 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.870+0000 7f33c126b640 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8e749f0 0x5565a927ac50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a1d112c2c3281de server_cookie=fdf23ef55e41aa16 in_seq=0 out_seq=0 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33a4ff9640 1 -- 172.21.15.83:0/381670371 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565a91d5df0 con 0x5565a8e749f0 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33a4ff9640 10 client.26701 handle_command_reply: tid=0 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 client.26701 shutdown 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 msgr2=0x5565a9268e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a9268e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005d00 tx=0x7f33ac011bc0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 shutdown_connections 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8e749f0 0x5565a927ac50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33a4ff9640 0 client.26701 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8eb6070 0x5565a9269380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a9268e40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 --2- 172.21.15.83:0/381670371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e8d1b0 0x5565a9268900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 >> 172.21.15.83:0/381670371 conn(0x5565a8eaa460 msgr2=0x5565a92604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.871+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 shutdown_connections 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.873+0000 7f33c9a43740 1 -- 172.21.15.83:0/381670371 wait complete. 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.873+0000 7f33c9a43740 20 client.26701 trim_cache size 0 max 16384 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 Processor -- start 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 -- start start 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1ad480 con 0x5565a8e749f0 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1af020 con 0x5565a8e95390 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c9a43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1adb50 con 0x5565a8e8d1b0 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c0a6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34804/0 (socket says 172.21.15.83:34804) 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 -- 172.21.15.83:0/301812124 learned_addr learned my addr 172.21.15.83:0/301812124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c1a6c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41404/0 (socket says 172.21.15.83:41404) 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 -- 172.21.15.83:0/301812124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 msgr2=0x5565a925aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 -- 172.21.15.83:0/301812124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 msgr2=0x5565a925b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c126b640 1 -- 172.21.15.83:0/301812124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a927b160 con 0x5565a8e749f0 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.890+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.891+0000 7f33c126b640 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f33b80ca440 tx=0x7f33b80f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2e588d06b71586 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.891+0000 7f33a6ffd640 1 -- 172.21.15.83:0/301812124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8049040 con 0x5565a8e749f0 2024-09-21T03:27:36.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.891+0000 7f33a6ffd640 1 -- 172.21.15.83:0/301812124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b803b460 con 0x5565a8e749f0 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.891+0000 7f33a6ffd640 1 -- 172.21.15.83:0/301812124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80b50b0 con 0x5565a8e749f0 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- 172.21.15.83:0/301812124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 msgr2=0x5565a9260ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f33b80ca440 tx=0x7f33b80f1040 comp rx=0 tx=0).stop 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- 172.21.15.83:0/301812124 shutdown_connections 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e95390 0x5565a925b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e8d1b0 0x5565a925aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 --2- 172.21.15.83:0/301812124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a9260ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- 172.21.15.83:0/301812124 >> 172.21.15.83:0/301812124 conn(0x5565a8eaa460 msgr2=0x5565a92603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- 172.21.15.83:0/301812124 shutdown_connections 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- 172.21.15.83:0/301812124 wait complete. 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 -- start start 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.892+0000 7f33c9a43740 1 Processor -- start 2024-09-21T03:27:36.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a926d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c9a43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a92e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c0a6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a926d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a926d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34820/0 (socket says 172.21.15.83:34820) 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.893+0000 7f33c0a6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:56044/0 (socket says 172.21.15.83:56044) 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.894+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1ae8f0 con 0x5565a8e749f0 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.894+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1ae220 con 0x5565a8e8d1b0 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.894+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1ab480 con 0x5565a8e95390 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.894+0000 7f33c126b640 1 -- 172.21.15.83:0/3096203266 learned_addr learned my addr 172.21.15.83:0/3096203266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.894+0000 7f33c1a6c640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a92e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c9a43740 10 client.0 _ll_register_callbacks cb 0x7f33ac00bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 -- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 msgr2=0x5565a92e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a92e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 -- 172.21.15.83:0/3096203266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 msgr2=0x5565a926d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a926d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a926de80 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33c0a6a640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e0d20170738c4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.895+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0043d0 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f33ac017070 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565a91d18a0 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f33ac01aa50 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 10 client.26747 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5565a92e5ad0 con 0x5565a8e8d1b0 2024-09-21T03:27:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5565a91d18a0 con 0x5565a8e8d1b0 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f33ac004660 con 0x5565a8e8d1b0 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 10 client.26747 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 10 client.26747 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 20 client.26747 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8eb6070 0x5565a92e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 4 client.26747 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.896+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565a91cecd0 con 0x5565a8eb6070 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c126b640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8eb6070 0x5565a92e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33a4ff9640 10 client.26747 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c126b640 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8eb6070 0x5565a92e8200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbdf3b28a9d034d1 server_cookie=e9f71d3b9567a7d3 in_seq=0 out_seq=0 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33a4ff9640 1 -- 172.21.15.83:0/3096203266 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565a91cecd0 con 0x5565a8eb6070 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33a4ff9640 10 client.26747 handle_command_reply: tid=0 2024-09-21T03:27:36.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 client.26747 shutdown 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 msgr2=0x5565a926d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac010ee0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 shutdown_connections 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5565a8eb6070 0x5565a92e8200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:27:36.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565a8e95390 0x5565a92e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565a8e8d1b0 0x5565a926d600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33a4ff9640 0 client.26747 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 --2- 172.21.15.83:0/3096203266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565a8e749f0 0x5565a926d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 >> 172.21.15.83:0/3096203266 conn(0x5565a8eaa460 msgr2=0x5565a925aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 shutdown_connections 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.897+0000 7f33c9a43740 1 -- 172.21.15.83:0/3096203266 wait complete. 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c9a43740 20 client.26747 trim_cache size 0 max 16384 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 msgr2=0x7f33a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 0x7f33a80b8aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33b40104d0 tx=0x7f33b4005fd0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 msgr2=0x7f33bc19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005940 tx=0x7f33b80042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 shutdown_connections 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33a80b6590 0x7f33a80b8aa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33bc109e00 0x7f33bc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc1094b0 0x7f33bc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 --2- 172.21.15.83:0/3911499206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33bc101f70 0x7f33bc19a510 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 >> 172.21.15.83:0/3911499206 conn(0x7f33bc077c60 msgr2=0x7f33bc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 shutdown_connections 2024-09-21T03:27:36.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:36.898+0000 7f33c34cf640 1 -- 172.21.15.83:0/3911499206 wait complete. 2024-09-21T03:27:37.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:37 smithi193 ceph-mon[35224]: pgmap v663: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-21T03:27:37.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:37 smithi083 ceph-mon[31219]: pgmap v663: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-21T03:27:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:37 smithi142 ceph-mon[34512]: pgmap v663: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-21T03:27:37.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:27:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.996+0000 7f6ad98b1640 1 -- 172.21.15.83:0/1167859943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad4102a60 msgr2=0x7f6ad4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.996+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/1167859943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad4102a60 0x7f6ad4102e70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac402a410 comp rx=0 tx=0).stop 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 -- 172.21.15.83:0/1167859943 shutdown_connections 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/1167859943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4108cf0 0x7f6ad410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/1167859943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad41083a0 0x7f6ad41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/1167859943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad4102a60 0x7f6ad4102e70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 -- 172.21.15.83:0/1167859943 >> 172.21.15.83:0/1167859943 conn(0x7f6ad4079730 msgr2=0x7f6ad4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 -- 172.21.15.83:0/1167859943 shutdown_connections 2024-09-21T03:27:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 -- 172.21.15.83:0/1167859943 wait complete. 2024-09-21T03:27:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.997+0000 7f6ad98b1640 1 Processor -- start 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad98b1640 1 -- start start 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad98b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 0x7f6ad4104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad98b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad41083a0 0x7f6ad4104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 0x7f6ad4104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 0x7f6ad4104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:41426/0 (socket says 172.21.15.83:41426) 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/4242474659 learned_addr learned my addr 172.21.15.83:0/4242474659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.998+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 0x7f6ad4104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.999+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4112960 con 0x7f6ad41083a0 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.999+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41127e0 con 0x7f6ad4108cf0 2024-09-21T03:27:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.999+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4112ae0 con 0x7f6ad4102a60 2024-09-21T03:27:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:38.999+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 0x7f6ad4104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 -- 172.21.15.83:0/4242474659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 msgr2=0x7f6ad4104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 0x7f6ad4104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 -- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad41083a0 msgr2=0x7f6ad4104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad41083a0 0x7f6ad4104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41a8700 con 0x7f6ad4108cf0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 0x7f6ad4104cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009fd0 tx=0x7f6acc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad88af640 1 -- 172.21.15.83:0/4242474659 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00ce20 con 0x7f6ad4108cf0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad88af640 1 -- 172.21.15.83:0/4242474659 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6acc005020 con 0x7f6ad4108cf0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad88af640 1 -- 172.21.15.83:0/4242474659 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc010690 con 0x7f6ad4108cf0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.000+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41a89f0 con 0x7f6ad4108cf0 2024-09-21T03:27:39.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.001+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a8e80 con 0x7f6ad4108cf0 2024-09-21T03:27:39.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.005+0000 7f6ad88af640 1 -- 172.21.15.83:0/4242474659 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6acc0054d0 con 0x7f6ad4108cf0 2024-09-21T03:27:39.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 Processor -- start 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 -- start start 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad88af640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 0x7f6abc0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44c67800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44c67d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b44bddea0 con 0x559b448c0070 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b44bcd2d0 con 0x559b4489f390 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6adfe25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b44bd10e0 con 0x559b448971b0 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad88af640 1 -- 172.21.15.83:0/4242474659 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6acc0e7d90 con 0x7f6ad4108cf0 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34852/0 (socket says 172.21.15.83:34852) 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/830425305 learned_addr learned my addr 172.21.15.83:0/830425305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:39.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 0x7f6abc0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44c67d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.006+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44c67800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/830425305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 msgr2=0x559b44c67800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44c67800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/830425305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 msgr2=0x559b44c67d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44c67d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/830425305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4492b650 con 0x559b448c0070 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44c67d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4000c00 tx=0x7f6ac4031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bb213f0a1d3e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 0x7f6abc0b8ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80058c0 tx=0x7f6ac80127c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ab9ffb640 1 -- 172.21.15.83:0/830425305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4040070 con 0x559b448c0070 2024-09-21T03:27:39.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ab9ffb640 1 -- 172.21.15.83:0/830425305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac4004030 con 0x559b448c0070 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.007+0000 7f6ab9ffb640 1 -- 172.21.15.83:0/830425305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4035040 con 0x559b448c0070 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.008+0000 7f6adfe25740 1 -- 172.21.15.83:0/830425305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 msgr2=0x559b44c672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.008+0000 7f6adfe25740 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4000c00 tx=0x7f6ac4031b10 comp rx=0 tx=0).stop 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 -- 172.21.15.83:0/830425305 shutdown_connections 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44c67d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44c67800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 --2- 172.21.15.83:0/830425305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c672c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 -- 172.21.15.83:0/830425305 >> 172.21.15.83:0/830425305 conn(0x559b448b4460 msgr2=0x559b44c651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 -- 172.21.15.83:0/830425305 shutdown_connections 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.009+0000 7f6adfe25740 1 -- 172.21.15.83:0/830425305 wait complete. 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 -- start start 2024-09-21T03:27:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 Processor -- start 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c7d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c73830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b44a2e700 con 0x559b448c0070 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559b44b68160 con 0x559b448971b0 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b44c6d910 con 0x559b4489f390 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6adfe25740 10 client.0 _ll_register_callbacks cb 0x559b44b819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6ad27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c7d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:56074/0 (socket says 172.21.15.83:56074) 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.010+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/191643169 learned_addr learned my addr 172.21.15.83:0/191643169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c73830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 -- 172.21.15.83:0/191643169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 msgr2=0x559b44c73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 -- 172.21.15.83:0/191643169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 msgr2=0x559b44c73830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c73830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 -- 172.21.15.83:0/191643169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b44b95c90 con 0x559b4489f390 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c7d8a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4100b70 tx=0x7f6ac8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a651c1bbe9d855b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800daa0 con 0x559b4489f390 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6ac800dc40 con 0x559b4489f390 2024-09-21T03:27:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ac4004030 con 0x559b4489f390 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6ac800de40 con 0x559b4489f390 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.011+0000 7f6adfe25740 10 client.26719 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559b44c925e0 con 0x559b4489f390 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac4004030 con 0x559b4489f390 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6ac8028410 con 0x559b4489f390 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 10 client.26719 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 10 client.26719 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 20 client.26719 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b4487e9f0 0x559b44c95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 4 client.26719 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b44bdfdf0 con 0x559b4487e9f0 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.012+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b4487e9f0 0x559b44c95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.013+0000 7f6ab2ffd640 10 client.26719 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:27:39.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.013+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b4487e9f0 0x559b44c95b80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82dc90e8b1eb2e7b server_cookie=d9ce2a48753ec25c in_seq=0 out_seq=0 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/191643169 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559b44bdfdf0 con 0x559b4487e9f0 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6ab2ffd640 10 client.26719 handle_command_reply: tid=0 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 client.26719 shutdown 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 msgr2=0x559b44c7d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c7d8a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4100b70 tx=0x7f6ac8005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 shutdown_connections 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448c0070 0x559b44c73830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b4487e9f0 0x559b44c95b80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6ab2ffd640 0 client.26719 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c7d8a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 --2- 172.21.15.83:0/191643169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b448971b0 0x559b44c73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.014+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 >> 172.21.15.83:0/191643169 conn(0x559b448b4460 msgr2=0x559b44c63750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.016+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 shutdown_connections 2024-09-21T03:27:39.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.016+0000 7f6adfe25740 1 -- 172.21.15.83:0/191643169 wait complete. 2024-09-21T03:27:39.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.016+0000 7f6adfe25740 20 client.26719 trim_cache size 0 max 16384 2024-09-21T03:27:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 Processor -- start 2024-09-21T03:27:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 -- start start 2024-09-21T03:27:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 0x559b44c65220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 0x559b44c70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c70640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ad340 con 0x559b448971b0 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41aeee0 con 0x559b4487e9f0 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6adfe25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ada10 con 0x559b4489f390 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.032+0000 7f6ad27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 0x559b44c70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 0x559b44c65220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c70640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 0x559b44c70100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34870/0 (socket says 172.21.15.83:34870) 2024-09-21T03:27:39.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad27fc640 1 -- 172.21.15.83:0/1890317532 learned_addr learned my addr 172.21.15.83:0/1890317532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/1890317532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 msgr2=0x559b44c70640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c70640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/1890317532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 msgr2=0x559b44c70100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 0x559b44c70100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/1890317532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b44c6a0d0 con 0x559b4487e9f0 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c70640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 0x559b44c65220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559b44c96280 tx=0x7f6ac4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e97213c2dca0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ab97fa640 1 -- 172.21.15.83:0/1890317532 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4040040 con 0x559b4487e9f0 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ab97fa640 1 -- 172.21.15.83:0/1890317532 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac40055e0 con 0x559b4487e9f0 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.033+0000 7f6ab97fa640 1 -- 172.21.15.83:0/1890317532 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4035040 con 0x559b4487e9f0 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.034+0000 7f6adfe25740 1 -- 172.21.15.83:0/1890317532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 msgr2=0x559b44c65220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.034+0000 7f6adfe25740 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 0x559b44c65220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559b44c96280 tx=0x7f6ac4005a20 comp rx=0 tx=0).stop 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 -- 172.21.15.83:0/1890317532 shutdown_connections 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b4489f390 0x559b44c70640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b448971b0 0x559b44c70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 --2- 172.21.15.83:0/1890317532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4487e9f0 0x559b44c65220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 -- 172.21.15.83:0/1890317532 >> 172.21.15.83:0/1890317532 conn(0x559b448b4460 msgr2=0x559b44c64b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 -- 172.21.15.83:0/1890317532 shutdown_connections 2024-09-21T03:27:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 -- 172.21.15.83:0/1890317532 wait complete. 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 -- start start 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.035+0000 7f6adfe25740 1 Processor -- start 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 0x559b44ca72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44ca77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44ca2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ae7b0 con 0x559b4487e9f0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ae0e0 con 0x559b4489f390 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ab3a0 con 0x559b448971b0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6adfe25740 10 client.0 _ll_register_callbacks cb 0x7f6ac8010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 0x559b44ca72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44ca77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 0x559b44ca72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34878/0 (socket says 172.21.15.83:34878) 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/2757687059 learned_addr learned my addr 172.21.15.83:0/2757687059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad37fe640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44ca2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 -- 172.21.15.83:0/2757687059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 msgr2=0x559b44ca2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44ca2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 -- 172.21.15.83:0/2757687059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 msgr2=0x559b44ca72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 0x559b44ca72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 -- 172.21.15.83:0/2757687059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b44ca7da0 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ad27fc640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44ca77e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80287e0 tx=0x7f6ac8028690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94298c8396ec4c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.036+0000 7f6ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac80107f0 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ac40055e0 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6ac8028c60 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6ac8028e60 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6adfe25740 10 client.26725 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559b44d00810 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac40055e0 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.037+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6ac80048c0 con 0x559b448971b0 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 10 client.26725 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 10 client.26725 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:27:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 20 client.26725 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b448c0070 0x559b44d02f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 4 client.26725 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b44bdb8a0 con 0x559b448c0070 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b448c0070 0x559b44d02f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6ab2ffd640 10 client.26725 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.038+0000 7f6ad2ffd640 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b448c0070 0x559b44d02f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ec803c19fbf786f server_cookie=56279848aa60bf13 in_seq=0 out_seq=0 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6ab2ffd640 1 -- 172.21.15.83:0/2757687059 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b44bdb8a0 con 0x559b448c0070 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6ab2ffd640 10 client.26725 handle_command_reply: tid=0 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 client.26725 shutdown 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 msgr2=0x559b44ca77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44ca77e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80287e0 tx=0x7f6ac8028690 comp rx=0 tx=0).stop 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 shutdown_connections 2024-09-21T03:27:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b4489f390 0x559b44ca2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x559b448c0070 0x559b44d02f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6ab2ffd640 0 client.26725 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b448971b0 0x559b44ca77e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 --2- 172.21.15.83:0/2757687059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4487e9f0 0x559b44ca72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 >> 172.21.15.83:0/2757687059 conn(0x559b448b4460 msgr2=0x559b44c71390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 shutdown_connections 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 1 -- 172.21.15.83:0/2757687059 wait complete. 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.039+0000 7f6adfe25740 20 client.26725 trim_cache size 0 max 16384 2024-09-21T03:27:39.045 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 msgr2=0x7f6abc0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 0x7f6abc0b8ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80058c0 tx=0x7f6ac80127c0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 msgr2=0x7f6ad4104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 0x7f6ad4104cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009fd0 tx=0x7f6acc00ef10 comp rx=0 tx=0).stop 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 shutdown_connections 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6abc0b67d0 0x7f6abc0b8ce0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4108cf0 0x7f6ad4104cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad41083a0 0x7f6ad4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 --2- 172.21.15.83:0/4242474659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad4102a60 0x7f6ad4104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.041+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 >> 172.21.15.83:0/4242474659 conn(0x7f6ad4079730 msgr2=0x7f6ad4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.042+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 shutdown_connections 2024-09-21T03:27:39.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:39.042+0000 7f6ad98b1640 1 -- 172.21.15.83:0/4242474659 wait complete. 2024-09-21T03:27:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:39 smithi193 ceph-mon[35224]: pgmap v664: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-09-21T03:27:39.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:39 smithi083 ceph-mon[31219]: pgmap v664: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-09-21T03:27:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:39 smithi142 ceph-mon[34512]: pgmap v664: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-09-21T03:27:40.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:27:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 620.0 s, 1307.6 tps, lat 75.924 ms stddev 79.628 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 -- 172.21.15.83:0/2648895218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c1083a0 msgr2=0x7f3a7c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2648895218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c1083a0 0x7f3a7c1087b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a6802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 -- 172.21.15.83:0/2648895218 shutdown_connections 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2648895218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2648895218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c1083a0 0x7f3a7c1087b0 secure :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a6802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2648895218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c102a60 0x7f3a7c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.146+0000 7f3a82b11640 1 -- 172.21.15.83:0/2648895218 >> 172.21.15.83:0/2648895218 conn(0x7f3a7c079730 msgr2=0x7f3a7c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.147+0000 7f3a82b11640 1 -- 172.21.15.83:0/2648895218 shutdown_connections 2024-09-21T03:27:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.147+0000 7f3a82b11640 1 -- 172.21.15.83:0/2648895218 wait complete. 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.147+0000 7f3a82b11640 1 Processor -- start 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 -- start start 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 0x7f3a7c10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c110a30 0x7f3a7c1a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c113a80 con 0x7f3a7c102a60 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c113900 con 0x7f3a7c108cf0 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a82b11640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c113c00 con 0x7f3a7c110a30 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 0x7f3a7c10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:58980/0 (socket says 172.21.15.83:58980) 2024-09-21T03:27:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 -- 172.21.15.83:0/2492140537 learned_addr learned my addr 172.21.15.83:0/2492140537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 0x7f3a7c10ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42554/0 (socket says 172.21.15.83:42554) 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a810ae640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c110a30 0x7f3a7c1a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 -- 172.21.15.83:0/2492140537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c110a30 msgr2=0x7f3a7c1a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c110a30 0x7f3a7c1a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 -- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 msgr2=0x7f3a7c10ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 0x7f3a7c10ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.148+0000 7f3a7bfff640 1 -- 172.21.15.83:0/2492140537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c1a8d30 con 0x7f3a7c108cf0 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a680040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a79ffb640 1 -- 172.21.15.83:0/2492140537 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68035070 con 0x7f3a7c108cf0 2024-09-21T03:27:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c19a2c0 con 0x7f3a7c108cf0 2024-09-21T03:27:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a79ffb640 1 -- 172.21.15.83:0/2492140537 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a680044b0 con 0x7f3a7c108cf0 2024-09-21T03:27:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a79ffb640 1 -- 172.21.15.83:0/2492140537 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68049630 con 0x7f3a7c108cf0 2024-09-21T03:27:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.149+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c19a7a0 con 0x7f3a7c108cf0 2024-09-21T03:27:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.154+0000 7f3a89085740 1 Processor -- start 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.154+0000 7f3a89085740 1 -- start start 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a79ffb640 1 -- 172.21.15.83:0/2492140537 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3a680497d0 con 0x7f3a7c108cf0 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e4067be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3e3fddea0 con 0x55c3e3c9f390 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3e3fcd2d0 con 0x55c3e3c971b0 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a89085740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3e3fd10e0 con 0x55c3e3cc0070 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a810ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a810ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45142/0 (socket says 172.21.15.83:45142) 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e4067be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42556/0 (socket says 172.21.15.83:42556) 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a810ae640 1 -- 172.21.15.83:0/1859325144 learned_addr learned my addr 172.21.15.83:0/1859325144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 -- 172.21.15.83:0/1859325144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 msgr2=0x55c3e40676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 -- 172.21.15.83:0/1859325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 msgr2=0x55c3e4068120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a808ad640 1 -- 172.21.15.83:0/1859325144 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e3d2b650 con 0x55c3e3c971b0 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a810ae640 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.155+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a808ad640 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e4067be0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009bd0 tx=0x7f3a70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b93a404a59d6d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a79ffb640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 0x7f3a6c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a66ffd640 1 -- 172.21.15.83:0/1859325144 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004230 con 0x55c3e3c971b0 2024-09-21T03:27:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a810ae640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 0x7f3a6c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a79ffb640 1 -- 172.21.15.83:0/2492140537 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f3a68110050 con 0x7f3a7c108cf0 2024-09-21T03:27:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a66ffd640 1 -- 172.21.15.83:0/1859325144 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a70009e40 con 0x55c3e3c971b0 2024-09-21T03:27:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.156+0000 7f3a66ffd640 1 -- 172.21.15.83:0/1859325144 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004c00 con 0x55c3e3c971b0 2024-09-21T03:27:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a810ae640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 0x7f3a6c0b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005f40 tx=0x7f3a740096a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 -- 172.21.15.83:0/1859325144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 msgr2=0x55c3e4067be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e4067be0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009bd0 tx=0x7f3a70011810 comp rx=0 tx=0).stop 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 -- 172.21.15.83:0/1859325144 shutdown_connections 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e4067be0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 --2- 172.21.15.83:0/1859325144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e40676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 -- 172.21.15.83:0/1859325144 >> 172.21.15.83:0/1859325144 conn(0x55c3e3cb4460 msgr2=0x55c3e4065590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.157+0000 7f3a89085740 1 -- 172.21.15.83:0/1859325144 shutdown_connections 2024-09-21T03:27:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.158+0000 7f3a89085740 1 -- 172.21.15.83:0/1859325144 wait complete. 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.159+0000 7f3a89085740 1 -- start start 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.159+0000 7f3a89085740 1 Processor -- start 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e407f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3e3e2e700 con 0x55c3e3c971b0 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3e3f68160 con 0x55c3e3c9f390 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4075e80 con 0x55c3e3cc0070 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42566/0 (socket says 172.21.15.83:42566) 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 -- 172.21.15.83:0/2594609712 learned_addr learned my addr 172.21.15.83:0/2594609712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a810ae640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e407f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a89085740 10 client.0 _ll_register_callbacks cb 0x55c3e3f819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 -- 172.21.15.83:0/2594609712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 msgr2=0x55c3e407f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e407f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 -- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 msgr2=0x55c3e4073230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.160+0000 7f3a808ad640 1 -- 172.21.15.83:0/2594609712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e3f95c90 con 0x55c3e3c971b0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3a70019a50 tx=0x7f3a70019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8be2a1f3689164c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70028020 con 0x55c3e3c971b0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f3a70024470 con 0x55c3e3c971b0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c3e3fdfdf0 con 0x55c3e3c971b0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3a7001e040 con 0x55c3e3c971b0 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a89085740 10 client.17238 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c3e4081b70 con 0x55c3e3c971b0 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.161+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c3e3fdfdf0 con 0x55c3e3c971b0 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f3a70018070 con 0x55c3e3c971b0 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 10 client.17238 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 20 client.17238 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3c7e9f0 0x55c3e4084f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 4 client.17238 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3e3fdb8a0 con 0x55c3e3c7e9f0 2024-09-21T03:27:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3c7e9f0 0x55c3e4084f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a5bfff640 10 client.17238 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.162+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3c7e9f0 0x55c3e4084f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d91ed1e78855929e server_cookie=73b7e54c0d7feae9 in_seq=0 out_seq=0 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.163+0000 7f3a5bfff640 1 -- 172.21.15.83:0/2594609712 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3e3fdb8a0 con 0x55c3e3c7e9f0 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.163+0000 7f3a5bfff640 10 client.17238 handle_command_reply: tid=0 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.163+0000 7f3a89085740 1 client.17238 shutdown 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 msgr2=0x55c3e4072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3a70019a50 tx=0x7f3a70019a80 comp rx=0 tx=0).stop 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 shutdown_connections 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3c7e9f0 0x55c3e4084f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cc0070 0x55c3e407f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 --2- 172.21.15.83:0/2594609712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e4072cf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 >> 172.21.15.83:0/2594609712 conn(0x55c3e3cb4460 msgr2=0x55c3e406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.164+0000 7f3a5bfff640 0 client.17238 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:27:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.165+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 shutdown_connections 2024-09-21T03:27:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.165+0000 7f3a89085740 1 -- 172.21.15.83:0/2594609712 wait complete. 2024-09-21T03:27:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.165+0000 7f3a89085740 20 client.17238 trim_cache size 0 max 16384 2024-09-21T03:27:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 Processor -- start 2024-09-21T03:27:41.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 -- start start 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4070420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ada30 con 0x55c3e3c9f390 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1af5d0 con 0x55c3e3c971b0 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a89085740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ae100 con 0x55c3e3cb4460 2024-09-21T03:27:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a810ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45164/0 (socket says 172.21.15.83:45164) 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a810ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42572/0 (socket says 172.21.15.83:42572) 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.181+0000 7f3a808ad640 1 -- 172.21.15.83:0/2398492532 learned_addr learned my addr 172.21.15.83:0/2398492532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 -- 172.21.15.83:0/2398492532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 msgr2=0x55c3e406fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e406fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 -- 172.21.15.83:0/2398492532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 msgr2=0x55c3e4070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 -- 172.21.15.83:0/2398492532 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e4075430 con 0x55c3e3cb4460 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a808ad640 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005870 tx=0x7f3a700096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbba49de97cca8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a667fc640 1 -- 172.21.15.83:0/2398492532 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70028070 con 0x55c3e3cb4460 2024-09-21T03:27:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a667fc640 1 -- 172.21.15.83:0/2398492532 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a70023910 con 0x55c3e3cb4460 2024-09-21T03:27:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.182+0000 7f3a667fc640 1 -- 172.21.15.83:0/2398492532 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700035f0 con 0x55c3e3cb4460 2024-09-21T03:27:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 -- 172.21.15.83:0/2398492532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 msgr2=0x55c3e407f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005870 tx=0x7f3a700096a0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 -- 172.21.15.83:0/2398492532 shutdown_connections 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c9f390 0x55c3e4070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c971b0 0x55c3e406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 --2- 172.21.15.83:0/2398492532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e407f0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 -- 172.21.15.83:0/2398492532 >> 172.21.15.83:0/2398492532 conn(0x55c3e406e640 msgr2=0x55c3e4071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 -- 172.21.15.83:0/2398492532 shutdown_connections 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.183+0000 7f3a89085740 1 -- 172.21.15.83:0/2398492532 wait complete. 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.184+0000 7f3a89085740 1 -- start start 2024-09-21T03:27:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.184+0000 7f3a89085740 1 Processor -- start 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.184+0000 7f3a89085740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e407bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4077020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e407bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e4077730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a808ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e407bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42584/0 (socket says 172.21.15.83:42584) 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a808ad640 1 -- 172.21.15.83:0/1816269811 learned_addr learned my addr 172.21.15.83:0/1816269811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1aeea0 con 0x55c3e3c971b0 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ae7d0 con 0x55c3e3c9f390 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1aba30 con 0x55c3e3cb4460 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e4077730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a7bfff640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a89085740 10 client.0 _ll_register_callbacks cb 0x7f3a68113a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 -- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 msgr2=0x55c3e4077020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4077020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 -- 172.21.15.83:0/1816269811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 msgr2=0x55c3e407bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e407bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.185+0000 7f3a810ae640 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e40ee880 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a810ae640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e4077730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55c3e40759f0 tx=0x7f3a74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c32b25637ea3f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004b20 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f3a74004cc0 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a70023910 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3a7401e7b0 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 10 client.26743 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c3e40efa20 con 0x55c3e3cb4460 2024-09-21T03:27:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a70023910 con 0x55c3e3cb4460 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f3a74005510 con 0x55c3e3cb4460 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 10 client.26743 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 10 client.26743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 20 client.26743 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3cc0070 0x55c3e40f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 4 client.26743 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.186+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3e3fd8cd0 con 0x55c3e3cc0070 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a808ad640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3cc0070 0x55c3e40f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a5bfff640 10 client.26743 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a808ad640 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3cc0070 0x55c3e40f2230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=192bacd0200f013e server_cookie=177f96b294ff40da in_seq=0 out_seq=0 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a5bfff640 1 -- 172.21.15.83:0/1816269811 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3e3fd8cd0 con 0x55c3e3cc0070 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a5bfff640 10 client.26743 handle_command_reply: tid=0 2024-09-21T03:27:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a89085740 1 client.26743 shutdown 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 msgr2=0x55c3e4077730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.187+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e4077730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55c3e40759f0 tx=0x7f3a74004a40 comp rx=0 tx=0).stop 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 shutdown_connections 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55c3e3cc0070 0x55c3e40f2230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3e3cb4460 0x55c3e4077730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3e3c9f390 0x55c3e4077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a5bfff640 0 client.26743 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 --2- 172.21.15.83:0/1816269811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3e3c971b0 0x55c3e407bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 >> 172.21.15.83:0/1816269811 conn(0x55c3e406e640 msgr2=0x55c3e40686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 shutdown_connections 2024-09-21T03:27:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 1 -- 172.21.15.83:0/1816269811 wait complete. 2024-09-21T03:27:41.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a89085740 20 client.26743 trim_cache size 0 max 16384 2024-09-21T03:27:41.195 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 msgr2=0x7f3a6c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 0x7f3a6c0b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005f40 tx=0x7f3a740096a0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 msgr2=0x7f3a7c1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.188+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a680040c0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 shutdown_connections 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3a6c0b6680 0x7f3a6c0b8b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a7c110a30 0x7f3a7c1a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a7c108cf0 0x7f3a7c1104f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 --2- 172.21.15.83:0/2492140537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7c102a60 0x7f3a7c10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 >> 172.21.15.83:0/2492140537 conn(0x7f3a7c079730 msgr2=0x7f3a7c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 shutdown_connections 2024-09-21T03:27:41.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:41.190+0000 7f3a82b11640 1 -- 172.21.15.83:0/2492140537 wait complete. 2024-09-21T03:27:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:41 smithi193 ceph-mon[35224]: pgmap v665: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 767 op/s 2024-09-21T03:27:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:41 smithi083 ceph-mon[31219]: pgmap v665: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 767 op/s 2024-09-21T03:27:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:41 smithi142 ceph-mon[34512]: pgmap v665: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 767 op/s 2024-09-21T03:27:42.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:27:42.492 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.074+0000 7f6d2b324640 1 -- 172.21.15.83:0/3597225465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d24102a60 msgr2=0x7f6d24102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3597225465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d24102a60 0x7f6d24102e70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009900 tx=0x7f6d1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 -- 172.21.15.83:0/3597225465 shutdown_connections 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3597225465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24108cf0 0x7f6d2410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3597225465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d241083a0 0x7f6d241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3597225465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d24102a60 0x7f6d24102e70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 -- 172.21.15.83:0/3597225465 >> 172.21.15.83:0/3597225465 conn(0x7f6d24079730 msgr2=0x7f6d24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 -- 172.21.15.83:0/3597225465 shutdown_connections 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 -- 172.21.15.83:0/3597225465 wait complete. 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 Processor -- start 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.075+0000 7f6d2b324640 1 -- start start 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24102a60 0x7f6d2419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d24108cf0 0x7f6d241a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241118a0 con 0x7f6d241083a0 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24111720 con 0x7f6d24108cf0 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d2b324640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24111a20 con 0x7f6d24102a60 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42592/0 (socket says 172.21.15.83:42592) 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 -- 172.21.15.83:0/3102385118 learned_addr learned my addr 172.21.15.83:0/3102385118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d290c0640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24102a60 0x7f6d2419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d298c1640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d24108cf0 0x7f6d241a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 -- 172.21.15.83:0/3102385118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24102a60 msgr2=0x7f6d2419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24102a60 0x7f6d2419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 -- 172.21.15.83:0/3102385118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d24108cf0 msgr2=0x7f6d241a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d24108cf0 0x7f6d241a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 -- 172.21.15.83:0/3102385118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d241a25e0 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.076+0000 7f6d288bf640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e9c0 tx=0x7f6d1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.077+0000 7f6d227fc640 1 -- 172.21.15.83:0/3102385118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800cd80 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.077+0000 7f6d227fc640 1 -- 172.21.15.83:0/3102385118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d180054d0 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.077+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d241a28d0 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.077+0000 7f6d227fc640 1 -- 172.21.15.83:0/3102385118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010430 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.077+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d241a9030 con 0x7f6d241083a0 2024-09-21T03:27:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.080+0000 7f6d31898740 1 Processor -- start 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.080+0000 7f6d227fc640 1 -- 172.21.15.83:0/3102385118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6d18004fd0 con 0x7f6d241083a0 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.080+0000 7f6d31898740 1 -- start start 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f2347b1b0 0x558f2384bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 0x558f2384c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f237c1ea0 con 0x558f234a4070 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558f237b12d0 con 0x558f2347b1b0 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d31898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f237b50e0 con 0x558f23483390 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42596/0 (socket says 172.21.15.83:42596) 2024-09-21T03:27:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 -- 172.21.15.83:0/4153442763 learned_addr learned my addr 172.21.15.83:0/4153442763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d290c0640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f2347b1b0 0x558f2384bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d288bf640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 0x558f2384c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 -- 172.21.15.83:0/4153442763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 msgr2=0x558f2384c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 0x558f2384c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 -- 172.21.15.83:0/4153442763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f2347b1b0 msgr2=0x558f2384bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f2347b1b0 0x558f2384bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 -- 172.21.15.83:0/4153442763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2350f650 con 0x558f234a4070 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d288bf640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 0x558f2384c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009fd0 tx=0x7f6d140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aae40f2819b8cad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.081+0000 7f6d0b7fe640 1 -- 172.21.15.83:0/4153442763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004d20 con 0x558f234a4070 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.082+0000 7f6d227fc640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 0x7f6d0c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.082+0000 7f6d0b7fe640 1 -- 172.21.15.83:0/4153442763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d14004050 con 0x558f234a4070 2024-09-21T03:27:43.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.082+0000 7f6d0b7fe640 1 -- 172.21.15.83:0/4153442763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14005240 con 0x558f234a4070 2024-09-21T03:27:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.082+0000 7f6d290c0640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 0x7f6d0c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.082+0000 7f6d227fc640 1 -- 172.21.15.83:0/3102385118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6d180e7a90 con 0x7f6d241083a0 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- 172.21.15.83:0/4153442763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 msgr2=0x558f2384b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d290c0640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 0x7f6d0c0b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005ab0 tx=0x7f6d1c000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009fd0 tx=0x7f6d140118d0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- 172.21.15.83:0/4153442763 shutdown_connections 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f23483390 0x558f2384c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f2347b1b0 0x558f2384bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 --2- 172.21.15.83:0/4153442763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2384b6a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- 172.21.15.83:0/4153442763 >> 172.21.15.83:0/4153442763 conn(0x558f23498460 msgr2=0x558f23849590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- 172.21.15.83:0/4153442763 shutdown_connections 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- 172.21.15.83:0/4153442763 wait complete. 2024-09-21T03:27:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 -- start start 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.083+0000 7f6d31898740 1 Processor -- start 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f23857320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f234a4070 0x558f23863170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f23612700 con 0x558f2347b1b0 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558f2374c160 con 0x558f23483390 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558f23859e80 con 0x558f234a4070 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d31898740 10 client.0 _ll_register_callbacks cb 0x558f237659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42606/0 (socket says 172.21.15.83:42606) 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 -- 172.21.15.83:0/1859735394 learned_addr learned my addr 172.21.15.83:0/1859735394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d298c1640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f234a4070 0x558f23863170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d288bf640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f23857320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 -- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f234a4070 msgr2=0x558f23863170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f234a4070 0x558f23863170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 -- 172.21.15.83:0/1859735394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 msgr2=0x558f23857320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f23857320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.084+0000 7f6d290c0640 1 -- 172.21.15.83:0/1859735394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f23779c90 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d290c0640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c033000 tx=0x7f6d1c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2edd75f4df892c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c04e070 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6d1c004980 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d14004050 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d1c05a050 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 10 client.17256 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558f23865bc0 con 0x558f2347b1b0 2024-09-21T03:27:43.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d14004050 con 0x558f2347b1b0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6d1c0324b0 con 0x558f2347b1b0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 10 client.17256 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 20 client.17256 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f234629f0 0x558f238690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 4 client.17256 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.085+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f237c3df0 con 0x558f234629f0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.086+0000 7f6d288bf640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f234629f0 0x558f238690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.087+0000 7f6d288bf640 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f234629f0 0x558f238690c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ea82135659a6840 server_cookie=f85100693aa3ebd8 in_seq=0 out_seq=0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.087+0000 7f6d08ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d08ff9640 1 -- 172.21.15.83:0/1859735394 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f237c3df0 con 0x558f234629f0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d08ff9640 10 client.17256 handle_command_reply: tid=0 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 client.17256 shutdown 2024-09-21T03:27:43.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 msgr2=0x558f23856de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c033000 tx=0x7f6d1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 shutdown_connections 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f234629f0 0x558f238690c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f234a4070 0x558f23863170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f23857320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 --2- 172.21.15.83:0/1859735394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f2347b1b0 0x558f23856de0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 >> 172.21.15.83:0/1859735394 conn(0x558f23498460 msgr2=0x558f2384e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.088+0000 7f6d08ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.089+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 shutdown_connections 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.089+0000 7f6d31898740 1 -- 172.21.15.83:0/1859735394 wait complete. 2024-09-21T03:27:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.089+0000 7f6d31898740 20 client.17256 trim_cache size 0 max 16384 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 Processor -- start 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 -- start start 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f23857360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f234a4070 0x558f23853fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241b1800 con 0x558f23483390 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241b33a0 con 0x558f234a4070 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d31898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241b1ed0 con 0x558f2347b1b0 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42622/0 (socket says 172.21.15.83:42622) 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 -- 172.21.15.83:0/2821924724 learned_addr learned my addr 172.21.15.83:0/2821924724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d290c0640 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f23857360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d298c1640 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f234a4070 0x558f23853fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 -- 172.21.15.83:0/2821924724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 msgr2=0x558f23857360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f23857360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 -- 172.21.15.83:0/2821924724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f234a4070 msgr2=0x558f23853fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f234a4070 0x558f23853fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.104+0000 7f6d288bf640 1 -- 172.21.15.83:0/2821924724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f23849f00 con 0x558f23483390 2024-09-21T03:27:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.105+0000 7f6d288bf640 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800c6a0 tx=0x7f6d180d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b7c4da30dbc320 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.105+0000 7f6d0affd640 1 -- 172.21.15.83:0/2821924724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180a97d0 con 0x558f23483390 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.105+0000 7f6d0affd640 1 -- 172.21.15.83:0/2821924724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d180ef040 con 0x558f23483390 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.105+0000 7f6d0affd640 1 -- 172.21.15.83:0/2821924724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180a90a0 con 0x558f23483390 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 -- 172.21.15.83:0/2821924724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 msgr2=0x558f2384cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800c6a0 tx=0x7f6d180d1040 comp rx=0 tx=0).stop 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 -- 172.21.15.83:0/2821924724 shutdown_connections 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f234a4070 0x558f23853fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f23483390 0x558f2384cde0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 --2- 172.21.15.83:0/2821924724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f23857360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 -- 172.21.15.83:0/2821924724 >> 172.21.15.83:0/2821924724 conn(0x558f23498460 msgr2=0x558f2384c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 -- 172.21.15.83:0/2821924724 shutdown_connections 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.106+0000 7f6d31898740 1 -- 172.21.15.83:0/2821924724 wait complete. 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 -- start start 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 Processor -- start 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f2385f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f2385ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241b2c70 con 0x558f234a4070 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241b25a0 con 0x558f23483390 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d241af800 con 0x558f2347b1b0 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.107+0000 7f6d31898740 10 client.0 _ll_register_callbacks cb 0x7f6d1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d290c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f2385f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42628/0 (socket says 172.21.15.83:42628) 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 -- 172.21.15.83:0/4292632901 learned_addr learned my addr 172.21.15.83:0/4292632901 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d288bf640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f2385ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 -- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 msgr2=0x558f2385f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f2385f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 -- 172.21.15.83:0/4292632901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 msgr2=0x558f2385ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f2385ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 -- 172.21.15.83:0/4292632901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2385dc70 con 0x558f234a4070 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d290c0640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f2385f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d298c1640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009f70 tx=0x7f6d14004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59963cf7c6dd68a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14009c50 con 0x558f234a4070 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6d14004050 con 0x558f234a4070 2024-09-21T03:27:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.108+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d180ef040 con 0x558f234a4070 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d14004250 con 0x558f234a4070 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 10 client.17262 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558f238d3ae0 con 0x558f234a4070 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d180ef040 con 0x558f234a4070 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6d14005800 con 0x558f234a4070 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 10 client.17262 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 10 client.17262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 20 client.17262 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f23437da0 0x558f238d5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 4 client.17262 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f237bf8a0 con 0x558f23437da0 2024-09-21T03:27:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.109+0000 7f6d290c0640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f23437da0 0x558f238d5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d08ff9640 10 client.17262 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d290c0640 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f23437da0 0x558f238d5fb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b270e44a5976e143 server_cookie=13ee058b8fa452eb in_seq=0 out_seq=0 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d08ff9640 1 -- 172.21.15.83:0/4292632901 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f237bf8a0 con 0x558f23437da0 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d08ff9640 10 client.17262 handle_command_reply: tid=0 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 client.17262 shutdown 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 msgr2=0x558f2385d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009f70 tx=0x7f6d14004c00 comp rx=0 tx=0).stop 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 shutdown_connections 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x558f23437da0 0x558f238d5fb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f234a4070 0x558f2385d4e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d08ff9640 0 client.17262 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f23483390 0x558f2385ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 --2- 172.21.15.83:0/4292632901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f2347b1b0 0x558f2385f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 >> 172.21.15.83:0/4292632901 conn(0x558f23498460 msgr2=0x558f2384c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.110+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 shutdown_connections 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d31898740 1 -- 172.21.15.83:0/4292632901 wait complete. 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d31898740 20 client.17262 trim_cache size 0 max 16384 2024-09-21T03:27:43.117 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 msgr2=0x7f6d0c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 0x7f6d0c0b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005ab0 tx=0x7f6d1c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 msgr2=0x7f6d2419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e9c0 tx=0x7f6d1800ee80 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 shutdown_connections 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d0c0b6730 0x7f6d0c0b8c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d24108cf0 0x7f6d241a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d241083a0 0x7f6d2419cc10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 --2- 172.21.15.83:0/3102385118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d24102a60 0x7f6d2419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 >> 172.21.15.83:0/3102385118 conn(0x7f6d24079730 msgr2=0x7f6d24100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 shutdown_connections 2024-09-21T03:27:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:43.111+0000 7f6d2b324640 1 -- 172.21.15.83:0/3102385118 wait complete. 2024-09-21T03:27:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:43 smithi193 ceph-mon[35224]: pgmap v666: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-09-21T03:27:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:43 smithi083 ceph-mon[31219]: pgmap v666: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-09-21T03:27:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:43 smithi142 ceph-mon[34512]: pgmap v666: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-09-21T03:27:44.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:27:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.929+0000 7f6646aaf640 1 -- 172.21.15.83:0/3722979292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 msgr2=0x7f66401109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.929+0000 7f6646aaf640 1 --2- 172.21.15.83:0/3722979292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401109a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6638009ca0 tx=0x7f663802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.930+0000 7f6646aaf640 1 -- 172.21.15.83:0/3722979292 shutdown_connections 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.930+0000 7f6646aaf640 1 --2- 172.21.15.83:0/3722979292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401109a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.930+0000 7f6646aaf640 1 --2- 172.21.15.83:0/3722979292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f664010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.930+0000 7f6646aaf640 1 --2- 172.21.15.83:0/3722979292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 0x7f664010a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.930+0000 7f6646aaf640 1 -- 172.21.15.83:0/3722979292 >> 172.21.15.83:0/3722979292 conn(0x7f664006e9f0 msgr2=0x7f664006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.931+0000 7f6646aaf640 1 -- 172.21.15.83:0/3722979292 shutdown_connections 2024-09-21T03:27:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.931+0000 7f6646aaf640 1 -- 172.21.15.83:0/3722979292 wait complete. 2024-09-21T03:27:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 Processor -- start 2024-09-21T03:27:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 -- start start 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 0x7f66401a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f66401a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f664011e2d0 con 0x7f664010bfa0 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f664011e150 con 0x7f664010a3c0 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6646aaf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f664011e450 con 0x7f664010b650 2024-09-21T03:27:44.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f66462ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f6645aad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 0x7f66401a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f66462ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42648/0 (socket says 172.21.15.83:42648) 2024-09-21T03:27:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f66462ae640 1 -- 172.21.15.83:0/360475405 learned_addr learned my addr 172.21.15.83:0/360475405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.932+0000 7f66452ac640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f66401a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.935+0000 7f66452ac640 1 -- 172.21.15.83:0/360475405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 msgr2=0x7f66401a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.935+0000 7f66452ac640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 0x7f66401a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.935+0000 7f66452ac640 1 -- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 msgr2=0x7f66401aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.935+0000 7f66452ac640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f66452ac640 1 -- 172.21.15.83:0/360475405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66401aac60 con 0x7f664010b650 2024-09-21T03:27:44.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f66452ac640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f66401a5290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ec0 tx=0x7f663000ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f6636ffd640 1 -- 172.21.15.83:0/360475405 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630004020 con 0x7f664010b650 2024-09-21T03:27:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f6636ffd640 1 -- 172.21.15.83:0/360475405 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663000cd60 con 0x7f664010b650 2024-09-21T03:27:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f6636ffd640 1 -- 172.21.15.83:0/360475405 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300055a0 con 0x7f664010b650 2024-09-21T03:27:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.936+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66401aaf50 con 0x7f664010b650 2024-09-21T03:27:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.937+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6640119cf0 con 0x7f664010b650 2024-09-21T03:27:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.939+0000 7f664e285740 1 Processor -- start 2024-09-21T03:27:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 -- start start 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e09b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e09bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56005e011ea0 con 0x56005dccb1b0 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56005e0012d0 con 0x56005dcd3390 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f664e285740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56005e0050e0 con 0x56005dcf4070 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f66462ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f66462ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45250/0 (socket says 172.21.15.83:45250) 2024-09-21T03:27:44.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f66462ae640 1 -- 172.21.15.83:0/420898426 learned_addr learned my addr 172.21.15.83:0/420898426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.940+0000 7f6645aad640 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e09b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f6636ffd640 1 -- 172.21.15.83:0/360475405 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6630026020 con 0x7f664010b650 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 -- 172.21.15.83:0/420898426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 msgr2=0x56005e09bee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e09bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 -- 172.21.15.83:0/420898426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 msgr2=0x56005e09b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e09b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 -- 172.21.15.83:0/420898426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005dd5f650 con 0x56005dcf4070 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f66462ae640 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663802f420 tx=0x7f663802f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6ad426b5751f61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f662bfff640 1 -- 172.21.15.83:0/420898426 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638031cf0 con 0x56005dcf4070 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f662bfff640 1 -- 172.21.15.83:0/420898426 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638036040 con 0x56005dcf4070 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f662bfff640 1 -- 172.21.15.83:0/420898426 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380042a0 con 0x56005dcf4070 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.941+0000 7f6636ffd640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 0x7f662c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f6636ffd640 1 -- 172.21.15.83:0/360475405 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f66300efb30 con 0x7f664010b650 2024-09-21T03:27:44.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f6645aad640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 0x7f662c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 -- 172.21.15.83:0/420898426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 msgr2=0x56005e09b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f6645aad640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 0x7f662c0b8c10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x56005e09a0e0 tx=0x7f663c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663802f420 tx=0x7f663802f450 comp rx=0 tx=0).stop 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 -- 172.21.15.83:0/420898426 shutdown_connections 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e09bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e09b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 --2- 172.21.15.83:0/420898426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e09b460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.942+0000 7f664e285740 1 -- 172.21.15.83:0/420898426 >> 172.21.15.83:0/420898426 conn(0x56005dce8460 msgr2=0x56005e099350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 -- 172.21.15.83:0/420898426 shutdown_connections 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 -- 172.21.15.83:0/420898426 wait complete. 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 -- start start 2024-09-21T03:27:44.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 Processor -- start 2024-09-21T03:27:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.943+0000 7f664e285740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcd3390 0x56005e0a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f664e285740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 0x56005e0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f664e285740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56005de62700 con 0x56005dccb1b0 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f664e285740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56005df9c160 con 0x56005dcf4070 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f664e285740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56005e0a9be0 con 0x56005dcd3390 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f664e285740 10 client.0 _ll_register_callbacks cb 0x56005dfb59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f6645aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f6645aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42674/0 (socket says 172.21.15.83:42674) 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f6645aad640 1 -- 172.21.15.83:0/1025324247 learned_addr learned my addr 172.21.15.83:0/1025324247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 0x56005e0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66452ac640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcd3390 0x56005e0a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 -- 172.21.15.83:0/1025324247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcd3390 msgr2=0x56005e0a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcd3390 0x56005e0a7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 -- 172.21.15.83:0/1025324247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 msgr2=0x56005e0a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 -- 172.21.15.83:0/1025324247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005dfc9c90 con 0x56005dcf4070 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66462ae640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 0x56005e0b2ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f663802a900 tx=0x7f6638009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf99a2b860e442d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.944+0000 7f66297fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638052020 con 0x56005dcf4070 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f6638031cf0 con 0x56005dcf4070 2024-09-21T03:27:44.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6638036040 con 0x56005dcf4070 2024-09-21T03:27:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f663805d050 con 0x56005dcf4070 2024-09-21T03:27:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 10 client.26807 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56005e0b58d0 con 0x56005dcf4070 2024-09-21T03:27:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638036040 con 0x56005dcf4070 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f663804d030 con 0x56005dcf4070 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 10 client.26807 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 20 client.26807 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.947+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dcb29f0 0x56005e0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.948+0000 7f664e285740 4 client.26807 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.948+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005e013df0 con 0x56005dcb29f0 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.948+0000 7f6645aad640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dcb29f0 0x56005e0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.948+0000 7f6645aad640 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dcb29f0 0x56005e0b8c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942a01d1d315603a server_cookie=a5cd60ed05fa02c5 in_seq=0 out_seq=0 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.948+0000 7f66297fa640 10 client.26807 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f66297fa640 1 -- 172.21.15.83:0/1025324247 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56005e013df0 con 0x56005dcb29f0 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f66297fa640 10 client.26807 handle_command_reply: tid=0 2024-09-21T03:27:44.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 client.26807 shutdown 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 msgr2=0x56005e0b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 0x56005e0b2ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f663802a900 tx=0x7f6638009460 comp rx=0 tx=0).stop 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 shutdown_connections 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dcb29f0 0x56005e0b8c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcf4070 0x56005e0b2ed0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcd3390 0x56005e0a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 --2- 172.21.15.83:0/1025324247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f66297fa640 0 client.26807 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.949+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 >> 172.21.15.83:0/1025324247 conn(0x56005dce8460 msgr2=0x56005e09e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.950+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 shutdown_connections 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.950+0000 7f664e285740 1 -- 172.21.15.83:0/1025324247 wait complete. 2024-09-21T03:27:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.950+0000 7f664e285740 20 client.26807 trim_cache size 0 max 16384 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 Processor -- start 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 -- start start 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 0x56005e0a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b5b40 con 0x56005dcd3390 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b76e0 con 0x56005dccb1b0 2024-09-21T03:27:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f664e285740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b6210 con 0x56005dcf4070 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f66452ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f66452ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42690/0 (socket says 172.21.15.83:42690) 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f66452ac640 1 -- 172.21.15.83:0/3167931164 learned_addr learned my addr 172.21.15.83:0/3167931164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 0x56005e0a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f66462ae640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 -- 172.21.15.83:0/3167931164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 msgr2=0x56005e0a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e0a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 -- 172.21.15.83:0/3167931164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 msgr2=0x56005e0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f6645aad640 1 -- 172.21.15.83:0/3167931164 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005e09d750 con 0x56005dccb1b0 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.965+0000 7f66452ac640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.966+0000 7f6645aad640 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 0x56005e0a6fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6638009a30 tx=0x7f663c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6ccbb697b2ffc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.966+0000 7f662b7fe640 1 -- 172.21.15.83:0/3167931164 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c003a40 con 0x56005dccb1b0 2024-09-21T03:27:44.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.966+0000 7f662b7fe640 1 -- 172.21.15.83:0/3167931164 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c0054d0 con 0x56005dccb1b0 2024-09-21T03:27:44.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.966+0000 7f662b7fe640 1 -- 172.21.15.83:0/3167931164 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c004c70 con 0x56005dccb1b0 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 -- 172.21.15.83:0/3167931164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 msgr2=0x56005e0a6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 0x56005e0a6fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6638009a30 tx=0x7f663c01b000 comp rx=0 tx=0).stop 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 -- 172.21.15.83:0/3167931164 shutdown_connections 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e0a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dcd3390 0x56005e0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 --2- 172.21.15.83:0/3167931164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dccb1b0 0x56005e0a6fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 -- 172.21.15.83:0/3167931164 >> 172.21.15.83:0/3167931164 conn(0x56005dce8460 msgr2=0x56005e0a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 -- 172.21.15.83:0/3167931164 shutdown_connections 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.967+0000 7f664e285740 1 -- 172.21.15.83:0/3167931164 wait complete. 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.968+0000 7f664e285740 1 -- start start 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.968+0000 7f664e285740 1 Processor -- start 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.968+0000 7f664e285740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e0ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e1333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f6645aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f6645aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42700/0 (socket says 172.21.15.83:42700) 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f6645aad640 1 -- 172.21.15.83:0/1142089395 learned_addr learned my addr 172.21.15.83:0/1142089395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b6fb0 con 0x56005dccb1b0 2024-09-21T03:27:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b68e0 con 0x56005dcd3390 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b3b60 con 0x56005dcf4070 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.969+0000 7f664e285740 10 client.0 _ll_register_callbacks cb 0x7f663c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 -- 172.21.15.83:0/1142089395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 msgr2=0x56005e1333c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e1333c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 -- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 msgr2=0x56005e0ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e0ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005e0bade0 con 0x56005dccb1b0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f6645aad640 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f663c003b00 tx=0x7f663c003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908f52ada0bf0e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.970+0000 7f66297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c003e10 con 0x56005dccb1b0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f663c02b630 con 0x56005dccb1b0 2024-09-21T03:27:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c0054d0 con 0x56005dccb1b0 2024-09-21T03:27:44.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f663c03f020 con 0x56005dccb1b0 2024-09-21T03:27:44.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 10 client.17280 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:44.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56005e134990 con 0x56005dccb1b0 2024-09-21T03:27:44.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c0054d0 con 0x56005dccb1b0 2024-09-21T03:27:44.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f663c0449d0 con 0x56005dccb1b0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 10 client.17280 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 20 client.17280 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dc87da0 0x56005e137120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 4 client.17280 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.971+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56005e00f8a0 con 0x56005dc87da0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f66452ac640 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dc87da0 0x56005e137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f66452ac640 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dc87da0 0x56005e137120 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c6a0ffd9cefa05 server_cookie=71c80c80c23b5d3f in_seq=0 out_seq=0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f66297fa640 10 client.17280 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f66297fa640 1 -- 172.21.15.83:0/1142089395 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56005e00f8a0 con 0x56005dc87da0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f66297fa640 10 client.17280 handle_command_reply: tid=0 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.972+0000 7f664e285740 1 client.17280 shutdown 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 msgr2=0x56005e0b9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f663c003b00 tx=0x7f663c003b30 comp rx=0 tx=0).stop 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 shutdown_connections 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56005dc87da0 0x56005e137120 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56005dcf4070 0x56005e1333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56005dcd3390 0x56005e0ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 --2- 172.21.15.83:0/1142089395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005dccb1b0 0x56005e0b9fb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 >> 172.21.15.83:0/1142089395 conn(0x56005dce8460 msgr2=0x56005e0a7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f66297fa640 0 client.17280 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 shutdown_connections 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 1 -- 172.21.15.83:0/1142089395 wait complete. 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.973+0000 7f664e285740 20 client.17280 trim_cache size 0 max 16384 2024-09-21T03:27:44.978 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:27:44.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 msgr2=0x7f662c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 0x7f662c0b8c10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x56005e09a0e0 tx=0x7f663c005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:44.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 msgr2=0x7f66401a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f66401a5290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ec0 tx=0x7f663000ed10 comp rx=0 tx=0).stop 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 shutdown_connections 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f662c0b6700 0x7f662c0b8c10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f664010bfa0 0x7f66401aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f664010b650 0x7f66401a5290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 --2- 172.21.15.83:0/360475405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f664010a3c0 0x7f66401a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 >> 172.21.15.83:0/360475405 conn(0x7f664006e9f0 msgr2=0x7f664010f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 shutdown_connections 2024-09-21T03:27:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:44.974+0000 7f6646aaf640 1 -- 172.21.15.83:0/360475405 wait complete. 2024-09-21T03:27:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:45 smithi193 ceph-mon[35224]: pgmap v667: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-21T03:27:45.601 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:45 smithi083 ceph-mon[31219]: pgmap v667: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-21T03:27:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:45 smithi142 ceph-mon[34512]: pgmap v667: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-21T03:27:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 625.0 s, 1404.2 tps, lat 71.187 ms stddev 78.343 2024-09-21T03:27:46.633 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:27:46.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:27:46.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:47 smithi083 ceph-mon[31219]: pgmap v668: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-09-21T03:27:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:47 smithi193 ceph-mon[35224]: pgmap v668: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-09-21T03:27:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:47 smithi142 ceph-mon[34512]: pgmap v668: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 684 op/s 2024-09-21T03:27:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 -- 172.21.15.83:0/241962088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c102a60 msgr2=0x7f820c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 --2- 172.21.15.83:0/241962088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c102a60 0x7f820c102e70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 -- 172.21.15.83:0/241962088 shutdown_connections 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 --2- 172.21.15.83:0/241962088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c108cf0 0x7f820c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 --2- 172.21.15.83:0/241962088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1083a0 0x7f820c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 --2- 172.21.15.83:0/241962088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c102a60 0x7f820c102e70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.081+0000 7f8212250640 1 -- 172.21.15.83:0/241962088 >> 172.21.15.83:0/241962088 conn(0x7f820c079730 msgr2=0x7f820c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.082+0000 7f8212250640 1 -- 172.21.15.83:0/241962088 shutdown_connections 2024-09-21T03:27:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.082+0000 7f8212250640 1 -- 172.21.15.83:0/241962088 wait complete. 2024-09-21T03:27:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.082+0000 7f8212250640 1 Processor -- start 2024-09-21T03:27:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.082+0000 7f8212250640 1 -- start start 2024-09-21T03:27:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.083+0000 7f8212250640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 0x7f820c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.083+0000 7f8212250640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.083+0000 7f8212250640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 0x7f820c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.084+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 0x7f820c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.083+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 0x7f820c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f8212250640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f820c112960 con 0x7f820c108cf0 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59122/0 (socket says 172.21.15.83:59122) 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 -- 172.21.15.83:0/4093468397 learned_addr learned my addr 172.21.15.83:0/4093468397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.084+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 0x7f820c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:45288/0 (socket says 172.21.15.83:45288) 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1127e0 con 0x7f820c1083a0 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f820c112ae0 con 0x7f820c102a60 2024-09-21T03:27:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 -- 172.21.15.83:0/4093468397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 msgr2=0x7f820c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 0x7f820c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 -- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 msgr2=0x7f820c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 0x7f820c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820affd640 1 -- 172.21.15.83:0/4093468397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a03f0 con 0x7f820c1083a0 2024-09-21T03:27:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.085+0000 7f820bfff640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 0x7f820c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f820affd640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8200009be0 tx=0x7f820000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f8208ff9640 1 -- 172.21.15.83:0/4093468397 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200005020 con 0x7f820c1083a0 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a06e0 con 0x7f820c1083a0 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f8208ff9640 1 -- 172.21.15.83:0/4093468397 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8200005670 con 0x7f820c1083a0 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f8208ff9640 1 -- 172.21.15.83:0/4093468397 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200019670 con 0x7f820c1083a0 2024-09-21T03:27:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.086+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c104130 con 0x7f820c1083a0 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f8208ff9640 1 -- 172.21.15.83:0/4093468397 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f82000046c0 con 0x7f820c1083a0 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 Processor -- start 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 -- start start 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 0x562c15606970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1523e670 0x562c15606eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c1557ce60 con 0x562c1525f350 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c1556c290 con 0x562c15236490 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f82187c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c155700a0 con 0x562c1523e670 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.089+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42738/0 (socket says 172.21.15.83:42738) 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820bfff640 1 -- 172.21.15.83:0/3044507970 learned_addr learned my addr 172.21.15.83:0/3044507970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820affd640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1523e670 0x562c15606eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820bfff640 1 -- 172.21.15.83:0/3044507970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1523e670 msgr2=0x562c15606eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820bfff640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1523e670 0x562c15606eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820bfff640 1 -- 172.21.15.83:0/3044507970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 msgr2=0x562c15606970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f8208ff9640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 0x7f81f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820b7fe640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 0x562c15606970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f820affd640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 0x7f81f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.090+0000 7f8208ff9640 1 -- 172.21.15.83:0/4093468397 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f82000eb1b0 con 0x7f820c1083a0 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f820bfff640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 0x562c15606970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f820bfff640 1 -- 172.21.15.83:0/3044507970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c152ca930 con 0x562c1525f350 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f820b7fe640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 0x562c15606970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f820affd640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 0x7f81f40b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f82000e8a00 tx=0x7f8200004b40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f820bfff640 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f820400f410 tx=0x7f820400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8eaa068c3f1aff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f81f1ffb640 1 -- 172.21.15.83:0/3044507970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204015070 con 0x562c1525f350 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f81f1ffb640 1 -- 172.21.15.83:0/3044507970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204011bb0 con 0x562c1525f350 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.091+0000 7f81f1ffb640 1 -- 172.21.15.83:0/3044507970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204010c00 con 0x562c1525f350 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- 172.21.15.83:0/3044507970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 msgr2=0x562c15606430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f820400f410 tx=0x7f820400f8d0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- 172.21.15.83:0/3044507970 shutdown_connections 2024-09-21T03:27:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1523e670 0x562c15606eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c15236490 0x562c15606970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 --2- 172.21.15.83:0/3044507970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1525f350 0x562c15606430 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- 172.21.15.83:0/3044507970 >> 172.21.15.83:0/3044507970 conn(0x562c15253740 msgr2=0x562c15604320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- 172.21.15.83:0/3044507970 shutdown_connections 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- 172.21.15.83:0/3044507970 wait complete. 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.092+0000 7f82187c4740 1 -- start start 2024-09-21T03:27:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 Processor -- start 2024-09-21T03:27:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 0x562c15612070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1525f350 0x562c1561df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c153cd890 con 0x562c15236490 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c15506d40 con 0x562c1523e670 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c15614c10 con 0x562c1525f350 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f82187c4740 10 client.0 _ll_register_callbacks cb 0x562c1556ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 0x562c15612070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 0x562c15612070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42752/0 (socket says 172.21.15.83:42752) 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59134/0 (socket says 172.21.15.83:59134) 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820bfff640 1 -- 172.21.15.83:0/2808821221 learned_addr learned my addr 172.21.15.83:0/2808821221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.093+0000 7f820bfff640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1525f350 0x562c1561df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 -- 172.21.15.83:0/2808821221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1525f350 msgr2=0x562c1561df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1525f350 0x562c1561df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 -- 172.21.15.83:0/2808821221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 msgr2=0x562c15612070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 0x562c15612070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 -- 172.21.15.83:0/2808821221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1561e4f0 con 0x562c1523e670 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f820b7fe640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009230 tx=0x7f81fc032e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f386e2656adb2dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f81eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc040070 con 0x562c1523e670 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f81fc0318b0 con 0x562c1523e670 2024-09-21T03:27:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.094+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8204011bb0 con 0x562c1523e670 2024-09-21T03:27:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.095+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f81fc035040 con 0x562c1523e670 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.095+0000 7f82187c4740 10 client.26819 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.095+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562c15620d90 con 0x562c1523e670 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204011bb0 con 0x562c1523e670 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f81fc004750 con 0x562c1523e670 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 10 client.26819 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 20 client.26819 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1521dcd0 0x562c15624610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 4 client.26819 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.096+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c1557edb0 con 0x562c1521dcd0 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.097+0000 7f820bfff640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1521dcd0 0x562c15624610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.097+0000 7f81eaffd640 10 client.26819 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.097+0000 7f820bfff640 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1521dcd0 0x562c15624610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc0627317fdfe6b server_cookie=f2f4c47afb4f87e7 in_seq=0 out_seq=0 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f81eaffd640 1 -- 172.21.15.83:0/2808821221 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c1557edb0 con 0x562c1521dcd0 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f81eaffd640 10 client.26819 handle_command_reply: tid=0 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 client.26819 shutdown 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 msgr2=0x562c156125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009230 tx=0x7f81fc032e70 comp rx=0 tx=0).stop 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 shutdown_connections 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1521dcd0 0x562c15624610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c1525f350 0x562c1561df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f81eaffd640 0 client.26819 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156125d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 --2- 172.21.15.83:0/2808821221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c15236490 0x562c15612070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 >> 172.21.15.83:0/2808821221 conn(0x562c15253740 msgr2=0x562c15609610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 shutdown_connections 2024-09-21T03:27:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 1 -- 172.21.15.83:0/2808821221 wait complete. 2024-09-21T03:27:48.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.098+0000 7f82187c4740 20 client.26819 trim_cache size 0 max 16384 2024-09-21T03:27:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 Processor -- start 2024-09-21T03:27:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 -- start start 2024-09-21T03:27:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c15611360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1ad3e0 con 0x562c1521dcd0 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1aef80 con 0x562c1523e670 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f82187c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1adab0 con 0x562c15236490 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42756/0 (socket says 172.21.15.83:42756) 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f820bfff640 1 -- 172.21.15.83:0/2894532645 learned_addr learned my addr 172.21.15.83:0/2894532645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.120+0000 7f820b7fe640 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c15611360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 -- 172.21.15.83:0/2894532645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 msgr2=0x562c15611360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c15611360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 -- 172.21.15.83:0/2894532645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 msgr2=0x562c156106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 -- 172.21.15.83:0/2894532645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c15612750 con 0x562c1521dcd0 2024-09-21T03:27:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f820bfff640 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8204010850 tx=0x7f82040107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf4ed7cf45a869 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f81f17fa640 1 -- 172.21.15.83:0/2894532645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204015040 con 0x562c1521dcd0 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f81f17fa640 1 -- 172.21.15.83:0/2894532645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204010910 con 0x562c1521dcd0 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.121+0000 7f81f17fa640 1 -- 172.21.15.83:0/2894532645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204005290 con 0x562c1521dcd0 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- 172.21.15.83:0/2894532645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 msgr2=0x562c1561de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8204010850 tx=0x7f82040107e0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- 172.21.15.83:0/2894532645 shutdown_connections 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1523e670 0x562c156106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c15611360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 --2- 172.21.15.83:0/2894532645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1521dcd0 0x562c1561de00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- 172.21.15.83:0/2894532645 >> 172.21.15.83:0/2894532645 conn(0x562c15253740 msgr2=0x562c1560cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- 172.21.15.83:0/2894532645 shutdown_connections 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- 172.21.15.83:0/2894532645 wait complete. 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.122+0000 7f82187c4740 1 -- start start 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 Processor -- start 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1521dcd0 0x562c15688300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c156888c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1ae850 con 0x562c1523e670 2024-09-21T03:27:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1ae180 con 0x562c1521dcd0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1ab3e0 con 0x562c15236490 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f82187c4740 10 client.0 _ll_register_callbacks cb 0x7f81fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42762/0 (socket says 172.21.15.83:42762) 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 -- 172.21.15.83:0/3880171052 learned_addr learned my addr 172.21.15.83:0/3880171052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c156888c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1521dcd0 0x562c15688300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 -- 172.21.15.83:0/3880171052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 msgr2=0x562c156888c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c156888c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 -- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1521dcd0 msgr2=0x562c15688300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.123+0000 7f820bfff640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1521dcd0 0x562c15688300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f820bfff640 1 -- 172.21.15.83:0/3880171052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c15616d50 con 0x562c1523e670 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f820b7fe640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c156888c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f820bfff640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f82040099a0 tx=0x7f820400fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e835ef26b91e753 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204015070 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8204010910 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f8204025040 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8204010ac0 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f82187c4740 10 client.17298 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562c1568faf0 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204010910 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.124+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f820402b070 con 0x562c1523e670 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 10 client.17298 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 20 client.17298 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1525f350 0x562c15692050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 4 client.17298 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:27:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c1557a860 con 0x562c1525f350 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f820b7fe640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1525f350 0x562c15692050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f81eaffd640 10 client.17298 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.125+0000 7f820b7fe640 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1525f350 0x562c15692050 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1c97b481e5f8a2 server_cookie=44f09c518d8c315f in_seq=0 out_seq=0 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f81eaffd640 1 -- 172.21.15.83:0/3880171052 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c1557a860 con 0x562c1525f350 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f81eaffd640 10 client.17298 handle_command_reply: tid=0 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 client.17298 shutdown 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 msgr2=0x562c156165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f82040099a0 tx=0x7f820400fee0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 shutdown_connections 2024-09-21T03:27:48.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562c1525f350 0x562c15692050 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c1523e670 0x562c156165e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c15236490 0x562c156888c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f81eaffd640 0 client.17298 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 --2- 172.21.15.83:0/3880171052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c1521dcd0 0x562c15688300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 >> 172.21.15.83:0/3880171052 conn(0x562c15253740 msgr2=0x562c15610690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 shutdown_connections 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 1 -- 172.21.15.83:0/3880171052 wait complete. 2024-09-21T03:27:48.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.127+0000 7f82187c4740 20 client.17298 trim_cache size 0 max 16384 2024-09-21T03:27:48.133 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:27:48.133 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:27:48.133 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4", 2024-09-21T03:27:48.133 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:27:48.133 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 msgr2=0x7f81f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 0x7f81f40b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f82000e8a00 tx=0x7f8200004b40 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 msgr2=0x7f820c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8200009be0 tx=0x7f820000b690 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 shutdown_connections 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f81f40b6680 0x7f81f40b8b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c108cf0 0x7f820c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c1083a0 0x7f820c19aa20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 --2- 172.21.15.83:0/4093468397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c102a60 0x7f820c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 >> 172.21.15.83:0/4093468397 conn(0x7f820c079730 msgr2=0x7f820c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 shutdown_connections 2024-09-21T03:27:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:48.128+0000 7f8212250640 1 -- 172.21.15.83:0/4093468397 wait complete. 2024-09-21T03:27:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:48 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:48 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:48 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:48 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:48 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:48.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:48 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:48 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:48 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:48.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:48 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:48.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:27:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: pgmap v669: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:27:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:49 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: pgmap v669: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-21T03:27:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:27:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:49 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: pgmap v669: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:49 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 -- 172.21.15.83:0/693453802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd0102a60 msgr2=0x7f0fd0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/693453802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd0102a60 0x7f0fd0102e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009900 tx=0x7f0fc402a410 comp rx=0 tx=0).stop 2024-09-21T03:27:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 -- 172.21.15.83:0/693453802 shutdown_connections 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/693453802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 0x7f0fd010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/693453802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd01083a0 0x7f0fd01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/693453802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd0102a60 0x7f0fd0102e70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 -- 172.21.15.83:0/693453802 >> 172.21.15.83:0/693453802 conn(0x7f0fd0079730 msgr2=0x7f0fd0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 -- 172.21.15.83:0/693453802 shutdown_connections 2024-09-21T03:27:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.826+0000 7f0fd6de0640 1 -- 172.21.15.83:0/693453802 wait complete. 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.827+0000 7f0fd6de0640 1 Processor -- start 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.827+0000 7f0fd6de0640 1 -- start start 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd0102a60 0x7f0fd019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 0x7f0fd01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0112960 con 0x7f0fd01083a0 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01127e0 con 0x7f0fd0108cf0 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0112ae0 con 0x7f0fd0102a60 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40892/0 (socket says 172.21.15.83:40892) 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 -- 172.21.15.83:0/4196073200 learned_addr learned my addr 172.21.15.83:0/4196073200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd0102a60 0x7f0fd019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd537d640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 0x7f0fd01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 -- 172.21.15.83:0/4196073200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd0102a60 msgr2=0x7f0fd019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd0102a60 0x7f0fd019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 -- 172.21.15.83:0/4196073200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 msgr2=0x7f0fd01a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 0x7f0fd01a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 -- 172.21.15.83:0/4196073200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd01a2640 con 0x7f0fd01083a0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcffff640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e9f0 tx=0x7f0fbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fcdffb640 1 -- 172.21.15.83:0/4196073200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00ccc0 con 0x7f0fd01083a0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.828+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd01a2930 con 0x7f0fd01083a0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.829+0000 7f0fcdffb640 1 -- 172.21.15.83:0/4196073200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fbc0054d0 con 0x7f0fd01083a0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.829+0000 7f0fcdffb640 1 -- 172.21.15.83:0/4196073200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc010430 con 0x7f0fd01083a0 2024-09-21T03:27:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.829+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd01a9090 con 0x7f0fd01083a0 2024-09-21T03:27:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.833+0000 7f0fcdffb640 1 -- 172.21.15.83:0/4196073200 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0fbc0046c0 con 0x7f0fd01083a0 2024-09-21T03:27:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.834+0000 7f0fdd354740 1 Processor -- start 2024-09-21T03:27:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.834+0000 7f0fdd354740 1 -- start start 2024-09-21T03:27:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.834+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 0x560d38800270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.834+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d38800cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fdd354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d38776e50 con 0x560d38438390 2024-09-21T03:27:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fdd354740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d38766280 con 0x560d384301b0 2024-09-21T03:27:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fdd354740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d3876a090 con 0x560d38459070 2024-09-21T03:27:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fcdffb640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 0x7f0fc00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d38800cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd537d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 0x560d38800270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41620/0 (socket says 172.21.15.83:41620) 2024-09-21T03:27:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd537d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 0x560d38800270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38326/0 (socket says 172.21.15.83:38326) 2024-09-21T03:27:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/560697507 learned_addr learned my addr 172.21.15.83:0/560697507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fcdffb640 1 -- 172.21.15.83:0/4196073200 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f0fbc0e7ae0 con 0x7f0fd01083a0 2024-09-21T03:27:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/560697507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 msgr2=0x560d38800270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 0x560d38800270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/560697507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 msgr2=0x560d38800cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d38800cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/560697507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d384c4650 con 0x560d384301b0 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.835+0000 7f0fd537d640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 0x7f0fc00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.836+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009900 tx=0x7f0fc4032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c975c6e780f0b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.836+0000 7f0fd537d640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 0x7f0fc00b8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8005fd0 tx=0x7f0fc8015040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.836+0000 7f0fbaffd640 1 -- 172.21.15.83:0/560697507 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4040070 con 0x560d384301b0 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.836+0000 7f0fbaffd640 1 -- 172.21.15.83:0/560697507 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc4004030 con 0x560d384301b0 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.836+0000 7f0fbaffd640 1 -- 172.21.15.83:0/560697507 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4035040 con 0x560d384301b0 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 -- 172.21.15.83:0/560697507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 msgr2=0x560d388007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009900 tx=0x7f0fc4032b90 comp rx=0 tx=0).stop 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 -- 172.21.15.83:0/560697507 shutdown_connections 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d38800cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d388007b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 --2- 172.21.15.83:0/560697507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38459070 0x560d38800270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 -- 172.21.15.83:0/560697507 >> 172.21.15.83:0/560697507 conn(0x560d3844d460 msgr2=0x560d387fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 -- 172.21.15.83:0/560697507 shutdown_connections 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.837+0000 7f0fdd354740 1 -- 172.21.15.83:0/560697507 wait complete. 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fdd354740 1 -- start start 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fdd354740 1 Processor -- start 2024-09-21T03:27:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d3880bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d3880c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.838+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41624/0 (socket says 172.21.15.83:41624) 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/1637984445 learned_addr learned my addr 172.21.15.83:0/1637984445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d385c7700 con 0x560d38459070 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d38701110 con 0x560d384301b0 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d3880e9f0 con 0x560d38438390 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fdd354740 10 client.0 _ll_register_callbacks cb 0x560d3871a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fcffff640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d3880bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd537d640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d3880c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/1637984445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 msgr2=0x560d3880bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d3880bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/1637984445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 msgr2=0x560d3880c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d3880c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.839+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3872ec40 con 0x560d384301b0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.840+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4032f30 tx=0x7f0fc4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad0f0068b3c15e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.840+0000 7f0faffff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.840+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4040070 con 0x560d384301b0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.840+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f0fc4005460 con 0x560d384301b0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.840+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fc4004030 con 0x560d384301b0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.841+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0fc4035040 con 0x560d384301b0 2024-09-21T03:27:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.841+0000 7f0fdd354740 10 client.26834 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.841+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560d3881a6a0 con 0x560d384301b0 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.841+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc4004030 con 0x560d384301b0 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.841+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0fc4004cb0 con 0x560d384301b0 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 10 client.26834 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 20 client.26834 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d384179f0 0x560d3881da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 4 client.26834 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d38778da0 con 0x560d384179f0 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fcffff640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d384179f0 0x560d3881da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0fcffff640 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d384179f0 0x560d3881da00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb79b8f61af1dbe server_cookie=5cc47f1b5dd93634 in_seq=0 out_seq=0 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.842+0000 7f0faffff640 10 client.26834 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0faffff640 1 -- 172.21.15.83:0/1637984445 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d38778da0 con 0x560d384179f0 2024-09-21T03:27:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0faffff640 10 client.26834 handle_command_reply: tid=0 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 client.26834 shutdown 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 msgr2=0x560d3880b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4032f30 tx=0x7f0fc4005a20 comp rx=0 tx=0).stop 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 shutdown_connections 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d384179f0 0x560d3881da00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d3880c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d3880bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 --2- 172.21.15.83:0/1637984445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880b890 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0faffff640 0 client.26834 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.843+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 >> 172.21.15.83:0/1637984445 conn(0x560d3844d460 msgr2=0x560d38803400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.844+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 shutdown_connections 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.844+0000 7f0fdd354740 1 -- 172.21.15.83:0/1637984445 wait complete. 2024-09-21T03:27:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.844+0000 7f0fdd354740 20 client.26834 trim_cache size 0 max 16384 2024-09-21T03:27:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 Processor -- start 2024-09-21T03:27:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 -- start start 2024-09-21T03:27:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d387fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d387fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01b1780 con 0x560d38459070 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01b3320 con 0x560d384301b0 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fdd354740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01b1e50 con 0x560d38438390 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fd537d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d387fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fd4b7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41638/0 (socket says 172.21.15.83:41638) 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.868+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/2105817853 learned_addr learned my addr 172.21.15.83:0/2105817853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fcffff640 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d387fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/2105817853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 msgr2=0x560d387fd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d387fd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/2105817853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 msgr2=0x560d387fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d387fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 -- 172.21.15.83:0/2105817853 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3880a7f0 con 0x560d384301b0 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402a930 tx=0x7f0fc40045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e52e4f14f1e7520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fba7fc640 1 -- 172.21.15.83:0/2105817853 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc405a070 con 0x560d384301b0 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fba7fc640 1 -- 172.21.15.83:0/2105817853 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc404d070 con 0x560d384301b0 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.869+0000 7f0fba7fc640 1 -- 172.21.15.83:0/2105817853 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4040070 con 0x560d384301b0 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 -- 172.21.15.83:0/2105817853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 msgr2=0x560d3880bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402a930 tx=0x7f0fc40045b0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 -- 172.21.15.83:0/2105817853 shutdown_connections 2024-09-21T03:27:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38459070 0x560d387fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d38438390 0x560d387fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 --2- 172.21.15.83:0/2105817853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d384301b0 0x560d3880bd50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 -- 172.21.15.83:0/2105817853 >> 172.21.15.83:0/2105817853 conn(0x560d3844d460 msgr2=0x560d3880b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 -- 172.21.15.83:0/2105817853 shutdown_connections 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.870+0000 7f0fdd354740 1 -- 172.21.15.83:0/2105817853 wait complete. 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.871+0000 7f0fdd354740 1 -- start start 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.871+0000 7f0fdd354740 1 Processor -- start 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.871+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d384301b0 0x560d38814690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d3880fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01b2bf0 con 0x560d38438390 2024-09-21T03:27:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd537d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd537d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41640/0 (socket says 172.21.15.83:41640) 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd537d640 1 -- 172.21.15.83:0/23366484 learned_addr learned my addr 172.21.15.83:0/23366484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01b2520 con 0x560d38459070 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd01af780 con 0x560d384301b0 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fdd354740 10 client.0 _ll_register_callbacks cb 0x7f0fbc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d384301b0 0x560d38814690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fcffff640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d3880fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd537d640 1 -- 172.21.15.83:0/23366484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d384301b0 msgr2=0x560d38814690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.872+0000 7f0fd537d640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d384301b0 0x560d38814690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fd537d640 1 -- 172.21.15.83:0/23366484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 msgr2=0x560d3880fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fd537d640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d3880fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fd537d640 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d388871a0 con 0x560d38459070 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fd537d640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8013e60 tx=0x7f0fc8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fda07fae75b47d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:49.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0faffff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc802c420 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8441+0+0 (secure 0 0 0) 0x7f0fc802c5c0 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fc404d070 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0fc802c800 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fdd354740 10 client.26840 fetch_fsmap learned FSMap version 46 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.873+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560d38888390 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc404d070 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0fc801c040 con 0x560d38459070 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 10 client.26840 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 10 client.26840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 20 client.26840 populate_metadata read hostname 'smithi083' 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d383ecda0 0x560d3888ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 4 client.26840 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d38774850 con 0x560d383ecda0 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.874+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d383ecda0 0x560d3888ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.875+0000 7f0faffff640 10 client.26840 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.875+0000 7f0fd4b7c640 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d383ecda0 0x560d3888ac60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9138d99e15bfd server_cookie=1c268f8a5e99bc10 in_seq=0 out_seq=0 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0faffff640 1 -- 172.21.15.83:0/23366484 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x560d38774850 con 0x560d383ecda0 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0faffff640 10 client.26840 handle_command_reply: tid=0 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 client.26840 shutdown 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 msgr2=0x560d38810300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8013e60 tx=0x7f0fc8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 shutdown_connections 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x560d383ecda0 0x560d3888ac60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d38459070 0x560d38810300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d38438390 0x560d3880fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 --2- 172.21.15.83:0/23366484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d384301b0 0x560d38814690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 >> 172.21.15.83:0/23366484 conn(0x560d3844d460 msgr2=0x560d388012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.876+0000 7f0faffff640 0 client.26840 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.877+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 shutdown_connections 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.877+0000 7f0fdd354740 1 -- 172.21.15.83:0/23366484 wait complete. 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.877+0000 7f0fdd354740 20 client.26840 trim_cache size 0 max 16384 2024-09-21T03:27:49.887 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4", 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:27:49.889 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.890 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:27:49.891 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.892 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:27:49.893 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.895 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.896 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.897 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:27:49.898 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:27:49.899 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:27:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 msgr2=0x7f0fc00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 0x7f0fc00b8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8005fd0 tx=0x7f0fc8015040 comp rx=0 tx=0).stop 2024-09-21T03:27:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 msgr2=0x7f0fd019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e9f0 tx=0x7f0fbc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 shutdown_connections 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f0fc00b6700 0x7f0fc00b8c10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd0108cf0 0x7f0fd01a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fd01083a0 0x7f0fd019cc70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 --2- 172.21.15.83:0/4196073200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd0102a60 0x7f0fd019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 >> 172.21.15.83:0/4196073200 conn(0x7f0fd0079730 msgr2=0x7f0fd006af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 shutdown_connections 2024-09-21T03:27:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:27:49.878+0000 7f0fd6de0640 1 -- 172.21.15.83:0/4196073200 wait complete. 2024-09-21T03:27:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:50 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:50 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.607 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:50 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.607 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:50 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.640 INFO:tasks.cephfs.filesystem:scrub status for tag:18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4 - {'path': '/', 'tag': '18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4', 'options': 'recursive,force'} 2024-09-21T03:27:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:50 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:50 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:27:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 630.0 s, 1247.8 tps, lat 80.224 ms stddev 88.293 2024-09-21T03:27:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:51 smithi193 ceph-mon[35224]: pgmap v670: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-09-21T03:27:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:51 smithi083 ceph-mon[31219]: pgmap v670: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-09-21T03:27:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:51 smithi142 ceph-mon[34512]: pgmap v670: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-09-21T03:27:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:53 smithi193 ceph-mon[35224]: pgmap v671: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-21T03:27:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:53 smithi083 ceph-mon[31219]: pgmap v671: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-21T03:27:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:53 smithi142 ceph-mon[34512]: pgmap v671: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-21T03:27:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:55 smithi193 ceph-mon[35224]: pgmap v672: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-09-21T03:27:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:55 smithi083 ceph-mon[31219]: pgmap v672: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-09-21T03:27:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:55 smithi142 ceph-mon[34512]: pgmap v672: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-09-21T03:27:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 635.0 s, 1560.4 tps, lat 64.336 ms stddev 73.503 2024-09-21T03:27:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:57 smithi193 ceph-mon[35224]: pgmap v673: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-09-21T03:27:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:57 smithi083 ceph-mon[31219]: pgmap v673: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-09-21T03:27:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:57 smithi142 ceph-mon[34512]: pgmap v673: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: pgmap v674: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:59.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:27:59 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:27:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: pgmap v674: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-21T03:27:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:59.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:27:59 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: pgmap v674: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:27:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:27:59 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 640.0 s, 1918.6 tps, lat 52.414 ms stddev 60.982 2024-09-21T03:28:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:01 smithi083 ceph-mon[31219]: pgmap v675: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-21T03:28:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:01 smithi142 ceph-mon[34512]: pgmap v675: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-21T03:28:01.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:01 smithi193 ceph-mon[35224]: pgmap v675: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-21T03:28:03.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:03 smithi083 ceph-mon[31219]: pgmap v676: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-21T03:28:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:03 smithi142 ceph-mon[34512]: pgmap v676: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-21T03:28:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:03 smithi193 ceph-mon[35224]: pgmap v676: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-09-21T03:28:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:28:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:04 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:28:04.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:04 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:04.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:04 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:05 smithi083 ceph-mon[31219]: pgmap v678: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 720 op/s 2024-09-21T03:28:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:05.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:05 smithi142 ceph-mon[34512]: pgmap v678: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 720 op/s 2024-09-21T03:28:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:05 smithi193 ceph-mon[35224]: pgmap v678: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 720 op/s 2024-09-21T03:28:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:28:05.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 645.0 s, 349.4 tps, lat 279.074 ms stddev 430.748 2024-09-21T03:28:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:07 smithi083 ceph-mon[31219]: pgmap v679: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 977 op/s 2024-09-21T03:28:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:07 smithi142 ceph-mon[34512]: pgmap v679: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 977 op/s 2024-09-21T03:28:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:07 smithi193 ceph-mon[35224]: pgmap v679: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 22 MiB/s wr, 977 op/s 2024-09-21T03:28:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:09 smithi083 ceph-mon[31219]: pgmap v680: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 28 MiB/s wr, 1.35k op/s 2024-09-21T03:28:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:09 smithi142 ceph-mon[34512]: pgmap v680: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 28 MiB/s wr, 1.35k op/s 2024-09-21T03:28:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:09 smithi193 ceph-mon[35224]: pgmap v680: 97 pgs: 7 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 75 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 28 MiB/s wr, 1.35k op/s 2024-09-21T03:28:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 650.0 s, 1539.6 tps, lat 65.656 ms stddev 87.436 2024-09-21T03:28:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:11 smithi142 ceph-mon[34512]: pgmap v681: 97 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 92 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-09-21T03:28:11.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:11 smithi193 ceph-mon[35224]: pgmap v681: 97 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 92 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-09-21T03:28:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:11 smithi083 ceph-mon[31219]: pgmap v681: 97 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 92 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-09-21T03:28:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:13 smithi142 ceph-mon[34512]: pgmap v682: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2024-09-21T03:28:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:13 smithi193 ceph-mon[35224]: pgmap v682: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2024-09-21T03:28:13.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:13 smithi083 ceph-mon[31219]: pgmap v682: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2024-09-21T03:28:15.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:15 smithi193 ceph-mon[35224]: pgmap v683: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2024-09-21T03:28:15.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:15 smithi193 ceph-mon[35224]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:28:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:15 smithi083 ceph-mon[31219]: pgmap v683: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2024-09-21T03:28:15.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:15 smithi083 ceph-mon[31219]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:28:15.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:15 smithi142 ceph-mon[34512]: pgmap v683: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 37 MiB/s wr, 1.98k op/s 2024-09-21T03:28:15.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:15 smithi142 ceph-mon[34512]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:28:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 655.0 s, 1821.6 tps, lat 55.584 ms stddev 62.278 2024-09-21T03:28:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:17 smithi193 ceph-mon[35224]: pgmap v685: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-09-21T03:28:17.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:17 smithi083 ceph-mon[31219]: pgmap v685: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-09-21T03:28:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:17 smithi142 ceph-mon[34512]: pgmap v685: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-09-21T03:28:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:19 smithi193 ceph-mon[35224]: pgmap v686: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-21T03:28:19.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:19 smithi083 ceph-mon[31219]: pgmap v686: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-21T03:28:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:19 smithi142 ceph-mon[34512]: pgmap v686: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-21T03:28:20.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:28:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 660.0 s, 1416.2 tps, lat 69.975 ms stddev 107.526 2024-09-21T03:28:21.023 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:21.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:21 smithi083 ceph-mon[31219]: pgmap v687: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-09-21T03:28:21.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:21 smithi193 ceph-mon[35224]: pgmap v687: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-09-21T03:28:21.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:21 smithi142 ceph-mon[34512]: pgmap v687: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-09-21T03:28:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 -- 172.21.15.83:0/3119340926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f01083a0 msgr2=0x7f98f01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 --2- 172.21.15.83:0/3119340926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f01083a0 0x7f98f01087b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009860 tx=0x7f98d802a340 comp rx=0 tx=0).stop 2024-09-21T03:28:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 -- 172.21.15.83:0/3119340926 shutdown_connections 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 --2- 172.21.15.83:0/3119340926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f0108cf0 0x7f98f010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 --2- 172.21.15.83:0/3119340926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f01083a0 0x7f98f01087b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 --2- 172.21.15.83:0/3119340926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0102a60 0x7f98f0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 -- 172.21.15.83:0/3119340926 >> 172.21.15.83:0/3119340926 conn(0x7f98f0079730 msgr2=0x7f98f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 -- 172.21.15.83:0/3119340926 shutdown_connections 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.777+0000 7f98f64b2640 1 -- 172.21.15.83:0/3119340926 wait complete. 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.779+0000 7f98f64b2640 1 Processor -- start 2024-09-21T03:28:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.779+0000 7f98f64b2640 1 -- start start 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f0102a60 0x7f98f01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 0x7f98f0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0112960 con 0x7f98f0102a60 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01127e0 con 0x7f98f0108cf0 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f64b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0112ae0 con 0x7f98f01083a0 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44736/0 (socket says 172.21.15.83:44736) 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 -- 172.21.15.83:0/294518893 learned_addr learned my addr 172.21.15.83:0/294518893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98e7fff640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 0x7f98f0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 -- 172.21.15.83:0/294518893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 msgr2=0x7f98f0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 0x7f98f0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 -- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f0102a60 msgr2=0x7f98f01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f0102a60 0x7f98f01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98f4a4f640 1 -- 172.21.15.83:0/294518893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f019f360 con 0x7f98f0108cf0 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.780+0000 7f98e7fff640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 0x7f98f0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.781+0000 7f98f4a4f640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98e800efc0 tx=0x7f98e800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.781+0000 7f98edffb640 1 -- 172.21.15.83:0/294518893 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8003970 con 0x7f98f0108cf0 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.781+0000 7f98edffb640 1 -- 172.21.15.83:0/294518893 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e8004020 con 0x7f98f0108cf0 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.781+0000 7f98edffb640 1 -- 172.21.15.83:0/294518893 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800b940 con 0x7f98f0108cf0 2024-09-21T03:28:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.781+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f019f650 con 0x7f98f0108cf0 2024-09-21T03:28:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.782+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f006ae60 con 0x7f98f0108cf0 2024-09-21T03:28:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.784+0000 7f98fca26740 1 Processor -- start 2024-09-21T03:28:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.784+0000 7f98edffb640 1 -- 172.21.15.83:0/294518893 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f98e801d020 con 0x7f98f0108cf0 2024-09-21T03:28:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.784+0000 7f98fca26740 1 -- start start 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 0x555c60d44650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c60cbae50 con 0x555c609741b0 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555c60caa280 con 0x555c6099d070 2024-09-21T03:28:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98fca26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c60cae090 con 0x555c6097c390 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53908/0 (socket says 172.21.15.83:53908) 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98effff640 1 -- 172.21.15.83:0/827903489 learned_addr learned my addr 172.21.15.83:0/827903489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98e7fff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98e7fff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 0x555c60d44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.785+0000 7f98edffb640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 0x7f98dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 -- 172.21.15.83:0/827903489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 msgr2=0x555c60d450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 -- 172.21.15.83:0/827903489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 msgr2=0x555c60d44b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 -- 172.21.15.83:0/827903489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c60a08650 con 0x555c6099d070 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98effff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98f4a4f640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 0x7f98dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98edffb640 1 -- 172.21.15.83:0/294518893 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f98e80e7640 con 0x7f98f0108cf0 2024-09-21T03:28:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e7fff640 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 0x555c60d44650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f98d802a930 tx=0x7f98d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27be9c1e86fd6862 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e67fc640 1 -- 172.21.15.83:0/827903489 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8046070 con 0x555c6099d070 2024-09-21T03:28:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e67fc640 1 -- 172.21.15.83:0/827903489 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8030b50 con 0x555c6099d070 2024-09-21T03:28:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.786+0000 7f98e67fc640 1 -- 172.21.15.83:0/827903489 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8037040 con 0x555c6099d070 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98fca26740 1 -- 172.21.15.83:0/827903489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 msgr2=0x555c60d44650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98fca26740 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 0x555c60d44650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f98d802a930 tx=0x7f98d8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98f4a4f640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 0x7f98dc0b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f98e80e53e0 tx=0x7f98e80047a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98fca26740 1 -- 172.21.15.83:0/827903489 shutdown_connections 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98fca26740 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.787+0000 7f98fca26740 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.788+0000 7f98fca26740 1 --2- 172.21.15.83:0/827903489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6099d070 0x555c60d44650 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.788+0000 7f98fca26740 1 -- 172.21.15.83:0/827903489 >> 172.21.15.83:0/827903489 conn(0x555c60991460 msgr2=0x555c60d42540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.788+0000 7f98fca26740 1 -- 172.21.15.83:0/827903489 shutdown_connections 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.788+0000 7f98fca26740 1 -- 172.21.15.83:0/827903489 wait complete. 2024-09-21T03:28:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.788+0000 7f98fca26740 1 -- start start 2024-09-21T03:28:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 Processor -- start 2024-09-21T03:28:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d4fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6099d070 0x555c60d50520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c60b0b700 con 0x555c6099d070 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555c60c45110 con 0x555c609741b0 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c60d57180 con 0x555c6097c390 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98fca26740 10 client.0 _ll_register_callbacks cb 0x555c60c5e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.789+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44756/0 (socket says 172.21.15.83:44756) 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 -- 172.21.15.83:0/1710753823 learned_addr learned my addr 172.21.15.83:0/1710753823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98effff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d4fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 -- 172.21.15.83:0/1710753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 msgr2=0x555c60d4fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d4fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 -- 172.21.15.83:0/1710753823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6099d070 msgr2=0x555c60d50520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6099d070 0x555c60d50520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.790+0000 7f98e7fff640 1 -- 172.21.15.83:0/1710753823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c60c72c40 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98e7fff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98d8044040 tx=0x7f98d8005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d9ea0989c3ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d804f060 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f98d8030450 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d8030b50 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f98d8037050 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98fca26740 10 client.26852 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555c60d67340 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.791+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8030b50 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f98d8032890 con 0x555c609741b0 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 10 client.26852 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 20 client.26852 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6095b9f0 0x555c60d6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 4 client.26852 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c60cbcda0 con 0x555c6095b9f0 2024-09-21T03:28:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98effff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6095b9f0 0x555c60d6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98cf7fe640 10 client.26852 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.792+0000 7f98effff640 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6095b9f0 0x555c60d6a6a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed64f8ff4ac0daf server_cookie=d6c361998f6f2fa7 in_seq=0 out_seq=0 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.793+0000 7f98cf7fe640 1 -- 172.21.15.83:0/1710753823 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c60cbcda0 con 0x555c6095b9f0 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.793+0000 7f98cf7fe640 10 client.26852 handle_command_reply: tid=0 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 client.26852 shutdown 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 msgr2=0x555c60d568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98d8044040 tx=0x7f98d8005c90 comp rx=0 tx=0).stop 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 shutdown_connections 2024-09-21T03:28:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6095b9f0 0x555c60d6a6a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98cf7fe640 0 client.26852 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6099d070 0x555c60d50520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d4fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 --2- 172.21.15.83:0/1710753823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d568c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 >> 172.21.15.83:0/1710753823 conn(0x555c60991460 msgr2=0x555c60d40b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 shutdown_connections 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.794+0000 7f98fca26740 1 -- 172.21.15.83:0/1710753823 wait complete. 2024-09-21T03:28:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.795+0000 7f98fca26740 20 client.26852 trim_cache size 0 max 16384 2024-09-21T03:28:21.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 Processor -- start 2024-09-21T03:28:21.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 -- start start 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d4d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01ad500 con 0x555c6095b9f0 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01af0a0 con 0x555c609741b0 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98fca26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01adbd0 con 0x555c6097c390 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53932/0 (socket says 172.21.15.83:53932) 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.821+0000 7f98e7fff640 1 -- 172.21.15.83:0/194707864 learned_addr learned my addr 172.21.15.83:0/194707864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98effff640 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d4d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 -- 172.21.15.83:0/194707864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 msgr2=0x555c60d64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d64ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 -- 172.21.15.83:0/194707864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 msgr2=0x555c60d4d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d4d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 -- 172.21.15.83:0/194707864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c60d51030 con 0x555c6095b9f0 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.822+0000 7f98e7fff640 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005e30 tx=0x7f98d802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5dbccd34b991ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.823+0000 7f98e5ffb640 1 -- 172.21.15.83:0/194707864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d804f040 con 0x555c6095b9f0 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.823+0000 7f98e5ffb640 1 -- 172.21.15.83:0/194707864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8030900 con 0x555c6095b9f0 2024-09-21T03:28:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.823+0000 7f98e5ffb640 1 -- 172.21.15.83:0/194707864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8046040 con 0x555c6095b9f0 2024-09-21T03:28:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.824+0000 7f98fca26740 1 -- 172.21.15.83:0/194707864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 msgr2=0x555c60d4d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005e30 tx=0x7f98d802a340 comp rx=0 tx=0).stop 2024-09-21T03:28:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 -- 172.21.15.83:0/194707864 shutdown_connections 2024-09-21T03:28:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60d64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c609741b0 0x555c60d4d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 --2- 172.21.15.83:0/194707864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c6095b9f0 0x555c60d4d170 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 -- 172.21.15.83:0/194707864 >> 172.21.15.83:0/194707864 conn(0x555c60991460 msgr2=0x555c60d4e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 -- 172.21.15.83:0/194707864 shutdown_connections 2024-09-21T03:28:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.825+0000 7f98fca26740 1 -- 172.21.15.83:0/194707864 wait complete. 2024-09-21T03:28:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 -- start start 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 Processor -- start 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6095b9f0 0x555c60d630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60dd3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01ae970 con 0x555c609741b0 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01ae2a0 con 0x555c6095b9f0 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.826+0000 7f98fca26740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01ab500 con 0x555c6097c390 2024-09-21T03:28:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98fca26740 10 client.0 _ll_register_callbacks cb 0x7f98d8044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53936/0 (socket says 172.21.15.83:53936) 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 -- 172.21.15.83:0/3818214720 learned_addr learned my addr 172.21.15.83:0/3818214720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98e7fff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60dd3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98e7fff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6095b9f0 0x555c60d630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 -- 172.21.15.83:0/3818214720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 msgr2=0x555c60dd3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60dd3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 -- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6095b9f0 msgr2=0x555c60d630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6095b9f0 0x555c60d630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 -- 172.21.15.83:0/3818214720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c60dd4360 con 0x555c609741b0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98effff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x555c60b0b7c0 tx=0x7f98e0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bcc5257975f0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.827+0000 7f98cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004230 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f98e00043d0 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d8030900 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f98e00035f0 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 10 client.17328 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555c60dd54d0 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8030900 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f98e001c960 con 0x555c609741b0 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 10 client.17328 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:21.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.828+0000 7f98fca26740 20 client.17328 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6099d070 0x555c60dd7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98fca26740 4 client.17328 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c60cb8850 con 0x555c6099d070 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98e7fff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6099d070 0x555c60dd7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98cf7fe640 10 client.17328 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.829+0000 7f98e7fff640 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6099d070 0x555c60dd7d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9880950be3c8e8 server_cookie=6bc873e7bb42ae38 in_seq=0 out_seq=0 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.830+0000 7f98cf7fe640 1 -- 172.21.15.83:0/3818214720 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x555c60cb8850 con 0x555c6099d070 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.830+0000 7f98cf7fe640 10 client.17328 handle_command_reply: tid=0 2024-09-21T03:28:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.830+0000 7f98fca26740 1 client.17328 shutdown 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 msgr2=0x555c60d635e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x555c60b0b7c0 tx=0x7f98e0010800 comp rx=0 tx=0).stop 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 shutdown_connections 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x555c6099d070 0x555c60dd7d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c6097c390 0x555c60dd3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c609741b0 0x555c60d635e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 --2- 172.21.15.83:0/3818214720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c6095b9f0 0x555c60d630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98cf7fe640 0 client.17328 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.832+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 >> 172.21.15.83:0/3818214720 conn(0x555c60991460 msgr2=0x555c60d45660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.833+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 shutdown_connections 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.833+0000 7f98fca26740 1 -- 172.21.15.83:0/3818214720 wait complete. 2024-09-21T03:28:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.833+0000 7f98fca26740 20 client.17328 trim_cache size 0 max 16384 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:28:21.837 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.838 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.840 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.841 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:28:21.842 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:28:21.844 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 msgr2=0x7f98dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 0x7f98dc0b8c10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f98e80e53e0 tx=0x7f98e80047a0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 msgr2=0x7f98f019ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98e800efc0 tx=0x7f98e800c800 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 shutdown_connections 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98dc0b6700 0x7f98dc0b8c10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98f0108cf0 0x7f98f019ec30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f01083a0 0x7f98f0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 --2- 172.21.15.83:0/294518893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98f0102a60 0x7f98f01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 >> 172.21.15.83:0/294518893 conn(0x7f98f0079730 msgr2=0x7f98f0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 shutdown_connections 2024-09-21T03:28:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:21.834+0000 7f98f64b2640 1 -- 172.21.15.83:0/294518893 wait complete. 2024-09-21T03:28:22.742 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:28:22.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:23.106 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:23.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:23 smithi193 ceph-mon[35224]: pgmap v688: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-09-21T03:28:23.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:23 smithi083 ceph-mon[31219]: pgmap v688: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-09-21T03:28:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:23 smithi142 ceph-mon[34512]: pgmap v688: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.17k op/s 2024-09-21T03:28:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 -- 172.21.15.83:0/1214372618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010a190 msgr2=0x7fa3f010a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 --2- 172.21.15.83:0/1214372618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010a190 0x7fa3f010a5a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009900 tx=0x7fa3e002a440 comp rx=0 tx=0).stop 2024-09-21T03:28:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 -- 172.21.15.83:0/1214372618 shutdown_connections 2024-09-21T03:28:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 --2- 172.21.15.83:0/1214372618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f0110420 0x7fa3f0116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 --2- 172.21.15.83:0/1214372618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f010fad0 0x7fa3f010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 --2- 172.21.15.83:0/1214372618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010a190 0x7fa3f010a5a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.647+0000 7fa3f6894640 1 -- 172.21.15.83:0/1214372618 >> 172.21.15.83:0/1214372618 conn(0x7fa3f00780f0 msgr2=0x7fa3f01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:23.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.650+0000 7fa3f6894640 1 -- 172.21.15.83:0/1214372618 shutdown_connections 2024-09-21T03:28:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.653+0000 7fa3f6894640 1 -- 172.21.15.83:0/1214372618 wait complete. 2024-09-21T03:28:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.654+0000 7fa3f6894640 1 Processor -- start 2024-09-21T03:28:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.655+0000 7fa3f6894640 1 -- start start 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f010a190 0x7fa3f0083c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010fad0 0x7fa3f007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f011a210 con 0x7fa3f010fad0 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f011a090 con 0x7fa3f0110420 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f6894640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f011a390 con 0x7fa3f010a190 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f4e31640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f4e31640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44792/0 (socket says 172.21.15.83:44792) 2024-09-21T03:28:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.656+0000 7fa3f4e31640 1 -- 172.21.15.83:0/2472769052 learned_addr learned my addr 172.21.15.83:0/2472769052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 -- 172.21.15.83:0/2472769052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f010a190 msgr2=0x7fa3f0083c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3effff640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f010a190 0x7fa3f0083c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f010a190 0x7fa3f0083c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 -- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010fad0 msgr2=0x7fa3f007cf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010fad0 0x7fa3f007cf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 -- 172.21.15.83:0/2472769052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f007dd00 con 0x7fa3f0110420 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f4e31640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b990 tx=0x7fa3e800be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8019070 con 0x7fa3f0110420 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f00829c0 con 0x7fa3f0110420 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e800cba0 con 0x7fa3f0110420 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80075d0 con 0x7fa3f0110420 2024-09-21T03:28:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.657+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f0082f80 con 0x7fa3f0110420 2024-09-21T03:28:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.659+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f010a620 con 0x7fa3f0110420 2024-09-21T03:28:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.661+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa3e801c070 con 0x7fa3f0110420 2024-09-21T03:28:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.662+0000 7fa3ed7fa640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 0x7fa3d80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.662+0000 7fa3effff640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 0x7fa3d80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.662+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7fa3e80e6e40 con 0x7fa3f0110420 2024-09-21T03:28:23.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.662+0000 7fa3effff640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 0x7fa3d80b8c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00096c0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.664+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3e80ad3e0 con 0x7fa3f0110420 2024-09-21T03:28:23.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.981+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f002e640 con 0x7fa3f0110420 2024-09-21T03:28:23.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.982+0000 7fa3ed7fa640 1 -- 172.21.15.83:0/2472769052 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11985 (secure 0 0 0) 0x7fa3e80b17c0 con 0x7fa3f0110420 2024-09-21T03:28:23.984 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:28:23.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"btime":"2024-09-21T03:27:19:181587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:27:18.256865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:28:23.985 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2024-09-21T03:28:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 msgr2=0x7fa3d80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 0x7fa3d80b8c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00096c0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 msgr2=0x7fa3f007d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b990 tx=0x7fa3e800be50 comp rx=0 tx=0).stop 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 shutdown_connections 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3d80b6730 0x7fa3d80b8c40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0110420 0x7fa3f007d4e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f010fad0 0x7fa3f007cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 --2- 172.21.15.83:0/2472769052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f010a190 0x7fa3f0083c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.986+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 >> 172.21.15.83:0/2472769052 conn(0x7fa3f00780f0 msgr2=0x7fa3f010dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.987+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 shutdown_connections 2024-09-21T03:28:23.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:23.987+0000 7fa3f6894640 1 -- 172.21.15.83:0/2472769052 wait complete. 2024-09-21T03:28:24.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:24 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2472769052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:24.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:24 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2472769052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:24.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:24 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2472769052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:24.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:28:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:25.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:25 smithi083 ceph-mon[31219]: pgmap v689: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-09-21T03:28:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:25 smithi193 ceph-mon[35224]: pgmap v689: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-09-21T03:28:25.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:25 smithi142 ceph-mon[34512]: pgmap v689: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-09-21T03:28:25.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 -- 172.21.15.83:0/1325564902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10a190 msgr2=0x7f124c10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 --2- 172.21.15.83:0/1325564902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10a190 0x7f124c10a5a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f123c009900 tx=0x7f123c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 -- 172.21.15.83:0/1325564902 shutdown_connections 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 --2- 172.21.15.83:0/1325564902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c110420 0x7f124c116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 --2- 172.21.15.83:0/1325564902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c10fad0 0x7f124c10fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 --2- 172.21.15.83:0/1325564902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10a190 0x7f124c10a5a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 -- 172.21.15.83:0/1325564902 >> 172.21.15.83:0/1325564902 conn(0x7f124c0780f0 msgr2=0x7f124c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 -- 172.21.15.83:0/1325564902 shutdown_connections 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.697+0000 7f125166b640 1 -- 172.21.15.83:0/1325564902 wait complete. 2024-09-21T03:28:25.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 Processor -- start 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 -- start start 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 0x7f124c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 0x7f124c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 0x7f124c1a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f124c11a0f0 con 0x7f124c10fad0 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f124c119f70 con 0x7f124c10a190 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.698+0000 7f125166b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f124c11a270 con 0x7f124c110420 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f1243fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 0x7f124c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 0x7f124c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 0x7f124c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44812/0 (socket says 172.21.15.83:44812) 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124affd640 1 -- 172.21.15.83:0/1142585455 learned_addr learned my addr 172.21.15.83:0/1142585455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 0x7f124c1a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 -- 172.21.15.83:0/1142585455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 msgr2=0x7f124c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 0x7f124c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 -- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 msgr2=0x7f124c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 0x7f124c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 -- 172.21.15.83:0/1142585455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c1a69d0 con 0x7f124c110420 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f1243fff640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 0x7f124c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f124b7fe640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 0x7f124c1a62a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fd0 tx=0x7f124400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f1248ff9640 1 -- 172.21.15.83:0/1142585455 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124400ce20 con 0x7f124c110420 2024-09-21T03:28:25.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a6cc0 con 0x7f124c110420 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f1248ff9640 1 -- 172.21.15.83:0/1142585455 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1244005020 con 0x7f124c110420 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.699+0000 7f1248ff9640 1 -- 172.21.15.83:0/1142585455 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244010690 con 0x7f124c110420 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.700+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1ad420 con 0x7f124c110420 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.703+0000 7f1257bdf740 1 Processor -- start 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.704+0000 7f1248ff9640 1 -- 172.21.15.83:0/1142585455 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1244010830 con 0x7f124c110420 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.704+0000 7f1257bdf740 1 -- start start 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.704+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.704+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1257bdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cf2abbea0 con 0x560cf279e070 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1257bdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560cf2aab2d0 con 0x560cf27751b0 2024-09-21T03:28:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1257bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cf2aaf0e0 con 0x560cf277d390 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1243fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44822/0 (socket says 172.21.15.83:44822) 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 -- 172.21.15.83:0/4240002472 learned_addr learned my addr 172.21.15.83:0/4240002472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1243fff640 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 -- 172.21.15.83:0/4240002472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 msgr2=0x560cf2b46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 -- 172.21.15.83:0/4240002472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 msgr2=0x560cf2b456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 -- 172.21.15.83:0/4240002472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf2809650 con 0x560cf27751b0 2024-09-21T03:28:25.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1248ff9640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 0x7f12380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1243fff640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 0x7f12380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f124affd640 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f123c005880 tx=0x7f123c031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8d0f1b7b821f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.705+0000 7f1248ff9640 1 -- 172.21.15.83:0/1142585455 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f12440e7dd0 con 0x7f124c110420 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.706+0000 7f12417fa640 1 -- 172.21.15.83:0/4240002472 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c035020 con 0x560cf27751b0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.706+0000 7f12417fa640 1 -- 172.21.15.83:0/4240002472 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c041070 con 0x560cf27751b0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.706+0000 7f12417fa640 1 -- 172.21.15.83:0/4240002472 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c004b70 con 0x560cf27751b0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 -- 172.21.15.83:0/4240002472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 msgr2=0x560cf2b45be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1243fff640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 0x7f12380b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1234009710 tx=0x7f1234005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f123c005880 tx=0x7f123c031490 comp rx=0 tx=0).stop 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 -- 172.21.15.83:0/4240002472 shutdown_connections 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b45be0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 --2- 172.21.15.83:0/4240002472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 -- 172.21.15.83:0/4240002472 >> 172.21.15.83:0/4240002472 conn(0x560cf2792460 msgr2=0x560cf2b43590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 -- 172.21.15.83:0/4240002472 shutdown_connections 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.707+0000 7f1257bdf740 1 -- 172.21.15.83:0/4240002472 wait complete. 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.708+0000 7f1257bdf740 1 -- start start 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.708+0000 7f1257bdf740 1 Processor -- start 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf279e070 0x560cf2b51800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cf290c700 con 0x560cf27751b0 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560cf2a46160 con 0x560cf279e070 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cf2b53e20 con 0x560cf277d390 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53970/0 (socket says 172.21.15.83:53970) 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 -- 172.21.15.83:0/1399383693 learned_addr learned my addr 172.21.15.83:0/1399383693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1257bdf740 10 client.0 _ll_register_callbacks cb 0x560cf2a5f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f1243fff640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124b7fe640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf279e070 0x560cf2b51800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 -- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 msgr2=0x560cf2b51280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b51280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 -- 172.21.15.83:0/1399383693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf279e070 msgr2=0x560cf2b51800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf279e070 0x560cf2b51800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 -- 172.21.15.83:0/1399383693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf2a73c90 con 0x560cf27751b0 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f124affd640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1de689899d2f8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.709+0000 7f12327fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c054060 con 0x560cf27751b0 2024-09-21T03:28:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f123c004d40 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123c041070 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f123c035070 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 10 client.17343 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560cf2b5fb30 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c041070 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f123c04e030 con 0x560cf27751b0 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 10 client.17343 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 10 client.17343 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 20 client.17343 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf275c9f0 0x560cf2b62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 4 client.17343 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.710+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cf2abddf0 con 0x560cf275c9f0 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.711+0000 7f1243fff640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf275c9f0 0x560cf2b62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.711+0000 7f1243fff640 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf275c9f0 0x560cf2b62e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=958022f1c8ee54d9 server_cookie=945f8942c6dd5a1b in_seq=0 out_seq=0 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.711+0000 7f12327fc640 10 client.17343 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f12327fc640 1 -- 172.21.15.83:0/1399383693 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560cf2abddf0 con 0x560cf275c9f0 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f12327fc640 10 client.17343 handle_command_reply: tid=0 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f1257bdf740 1 client.17343 shutdown 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 msgr2=0x560cf2b50d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f123c006010 tx=0x7f123c005550 comp rx=0 tx=0).stop 2024-09-21T03:28:25.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 shutdown_connections 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.712+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf275c9f0 0x560cf2b62e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf279e070 0x560cf2b51800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf277d390 0x560cf2b51280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 --2- 172.21.15.83:0/1399383693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf27751b0 0x560cf2b50d40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 >> 172.21.15.83:0/1399383693 conn(0x560cf2792460 msgr2=0x560cf2b48830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f12327fc640 0 client.17343 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 shutdown_connections 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 1 -- 172.21.15.83:0/1399383693 wait complete. 2024-09-21T03:28:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.713+0000 7f1257bdf740 20 client.17343 trim_cache size 0 max 16384 2024-09-21T03:28:25.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.732+0000 7f1257bdf740 1 Processor -- start 2024-09-21T03:28:25.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.732+0000 7f1257bdf740 1 -- start start 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 0x560cf2b51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf277d390 0x560cf2b4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b5b10 con 0x560cf279e070 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b76b0 con 0x560cf277d390 2024-09-21T03:28:25.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1257bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b61e0 con 0x560cf27751b0 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 0x560cf2b51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53974/0 (socket says 172.21.15.83:53974) 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 -- 172.21.15.83:0/446119470 learned_addr learned my addr 172.21.15.83:0/446119470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124affd640 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 0x560cf2b51200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:56246/0 (socket says 172.21.15.83:56246) 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f1243fff640 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf277d390 0x560cf2b4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 -- 172.21.15.83:0/446119470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 msgr2=0x560cf2b51200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 0x560cf2b51200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 -- 172.21.15.83:0/446119470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf277d390 msgr2=0x560cf2b4e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf277d390 0x560cf2b4e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 -- 172.21.15.83:0/446119470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf2b51de0 con 0x560cf279e070 2024-09-21T03:28:25.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.733+0000 7f124b7fe640 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f12440aaee0 tx=0x7f12440aa370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d070e8b807cfcd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.734+0000 7f1240ff9640 1 -- 172.21.15.83:0/446119470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440950c0 con 0x560cf279e070 2024-09-21T03:28:25.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.734+0000 7f1240ff9640 1 -- 172.21.15.83:0/446119470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12440b2750 con 0x560cf279e070 2024-09-21T03:28:25.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.734+0000 7f1240ff9640 1 -- 172.21.15.83:0/446119470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244005690 con 0x560cf279e070 2024-09-21T03:28:25.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.735+0000 7f1257bdf740 1 -- 172.21.15.83:0/446119470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 msgr2=0x560cf2b4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.735+0000 7f1257bdf740 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f12440aaee0 tx=0x7f12440aa370 comp rx=0 tx=0).stop 2024-09-21T03:28:25.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 -- 172.21.15.83:0/446119470 shutdown_connections 2024-09-21T03:28:25.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf279e070 0x560cf2b4e680 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf277d390 0x560cf2b4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 --2- 172.21.15.83:0/446119470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf27751b0 0x560cf2b51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 -- 172.21.15.83:0/446119470 >> 172.21.15.83:0/446119470 conn(0x560cf2792460 msgr2=0x560cf2b50af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 -- 172.21.15.83:0/446119470 shutdown_connections 2024-09-21T03:28:25.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 -- 172.21.15.83:0/446119470 wait complete. 2024-09-21T03:28:25.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 -- start start 2024-09-21T03:28:25.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.736+0000 7f1257bdf740 1 Processor -- start 2024-09-21T03:28:25.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 0x560cf2bcc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b6f80 con 0x560cf277d390 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b68b0 con 0x560cf27751b0 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1b3b10 con 0x560cf279e070 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1243fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1257bdf740 10 client.0 _ll_register_callbacks cb 0x7f1234009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1243fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53986/0 (socket says 172.21.15.83:53986) 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f1243fff640 1 -- 172.21.15.83:0/2911164958 learned_addr learned my addr 172.21.15.83:0/2911164958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f124b7fe640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 0x560cf2bcc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.737+0000 7f124affd640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 -- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 msgr2=0x560cf2bcc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 0x560cf2bcc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 -- 172.21.15.83:0/2911164958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 msgr2=0x560cf2b646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 -- 172.21.15.83:0/2911164958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf2b64f70 con 0x560cf27751b0 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124b7fe640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 0x560cf2bcc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f1243fff640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f124affd640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b641b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f123c005d90 tx=0x7f123c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fec92bc8979e681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f12327fc640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c054040 con 0x560cf27751b0 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f123c046d30 con 0x560cf27751b0 2024-09-21T03:28:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12440b2750 con 0x560cf27751b0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f123c005df0 con 0x560cf27751b0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f1257bdf740 10 client.26879 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.738+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560cf2bcdb70 con 0x560cf27751b0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12440b2750 con 0x560cf27751b0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f123c061070 con 0x560cf27751b0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 10 client.26879 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 10 client.26879 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 20 client.26879 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf2731da0 0x560cf2bd0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 4 client.26879 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cf2ab98a0 con 0x560cf2731da0 2024-09-21T03:28:25.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.739+0000 7f1243fff640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf2731da0 0x560cf2bd0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:25.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.740+0000 7f12327fc640 10 client.26879 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1243fff640 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf2731da0 0x560cf2bd0300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10c4f765a388c6b1 server_cookie=76b8caa6e11fcb4c in_seq=0 out_seq=0 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f12327fc640 1 -- 172.21.15.83:0/2911164958 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cf2ab98a0 con 0x560cf2731da0 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f12327fc640 10 client.26879 handle_command_reply: tid=0 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 client.26879 shutdown 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 msgr2=0x560cf2b641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b641b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f123c005d90 tx=0x7f123c005a20 comp rx=0 tx=0).stop 2024-09-21T03:28:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 shutdown_connections 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560cf2731da0 0x560cf2bd0300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cf279e070 0x560cf2bcc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cf277d390 0x560cf2b646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 --2- 172.21.15.83:0/2911164958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cf27751b0 0x560cf2b641b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 >> 172.21.15.83:0/2911164958 conn(0x560cf2792460 msgr2=0x560cf2b487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f12327fc640 0 client.26879 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 shutdown_connections 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.741+0000 7f1257bdf740 1 -- 172.21.15.83:0/2911164958 wait complete. 2024-09-21T03:28:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f1257bdf740 20 client.26879 trim_cache size 0 max 16384 2024-09-21T03:28:25.749 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:28:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 msgr2=0x7f12380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 0x7f12380b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1234009710 tx=0x7f1234005850 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 msgr2=0x7f124c1a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 0x7f124c1a62a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fd0 tx=0x7f124400ef10 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 shutdown_connections 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12380b6680 0x7f12380b8b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f124c110420 0x7f124c1a62a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f124c10fad0 0x7f124c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 --2- 172.21.15.83:0/1142585455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f124c10a190 0x7f124c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 >> 172.21.15.83:0/1142585455 conn(0x7f124c0780f0 msgr2=0x7f124c10dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.742+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 shutdown_connections 2024-09-21T03:28:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:25.743+0000 7f125166b640 1 -- 172.21.15.83:0/1142585455 wait complete. 2024-09-21T03:28:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 665.0 s, 1833.6 tps, lat 55.094 ms stddev 62.297 2024-09-21T03:28:26.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:28:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:27.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:27 smithi083 ceph-mon[31219]: pgmap v690: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-09-21T03:28:27.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:27 smithi193 ceph-mon[35224]: pgmap v690: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-09-21T03:28:27.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:27 smithi142 ceph-mon[34512]: pgmap v690: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 -- 172.21.15.83:0/659651708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f901021f0 msgr2=0x7f7f90102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 --2- 172.21.15.83:0/659651708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f901021f0 0x7f7f90102600 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009900 tx=0x7f7f8002b630 comp rx=0 tx=0).stop 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 -- 172.21.15.83:0/659651708 shutdown_connections 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 --2- 172.21.15.83:0/659651708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f90108480 0x7f7f9010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 --2- 172.21.15.83:0/659651708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f90107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 --2- 172.21.15.83:0/659651708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f901021f0 0x7f7f90102600 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 -- 172.21.15.83:0/659651708 >> 172.21.15.83:0/659651708 conn(0x7f7f90077c60 msgr2=0x7f7f900780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 -- 172.21.15.83:0/659651708 shutdown_connections 2024-09-21T03:28:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.669+0000 7f7f960af640 1 -- 172.21.15.83:0/659651708 wait complete. 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 Processor -- start 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 -- start start 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 0x7f7f9010ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f9010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90111420 con 0x7f7f90108480 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901112a0 con 0x7f7f901021f0 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.670+0000 7f7f960af640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901115a0 con 0x7f7f90107b30 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54000/0 (socket says 172.21.15.83:54000) 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8ffff640 1 -- 172.21.15.83:0/3969909863 learned_addr learned my addr 172.21.15.83:0/3969909863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 0x7f7f9010ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f87fff640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f9010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3969909863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 msgr2=0x7f7f9010bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f9010bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 msgr2=0x7f7f901a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3969909863 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f901a6a70 con 0x7f7f901021f0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 0x7f7f9010ba30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009850 tx=0x7f7f80005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.671+0000 7f7f87fff640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f9010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.672+0000 7f7f8d7fa640 1 -- 172.21.15.83:0/3969909863 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8003f020 con 0x7f7f901021f0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.672+0000 7f7f8d7fa640 1 -- 172.21.15.83:0/3969909863 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f80004230 con 0x7f7f901021f0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.672+0000 7f7f8d7fa640 1 -- 172.21.15.83:0/3969909863 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80036070 con 0x7f7f901021f0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.672+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f901a6d00 con 0x7f7f901021f0 2024-09-21T03:28:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.672+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901ad4c0 con 0x7f7f901021f0 2024-09-21T03:28:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.677+0000 7f7f9c623740 1 Processor -- start 2024-09-21T03:28:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.677+0000 7f7f8d7fa640 1 -- 172.21.15.83:0/3969909863 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7f80042020 con 0x7f7f901021f0 2024-09-21T03:28:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.678+0000 7f7f8d7fa640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 0x7f7f7c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.678+0000 7f7f8d7fa640 1 -- 172.21.15.83:0/3969909863 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7f80107da0 con 0x7f7f901021f0 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.677+0000 7f7f9c623740 1 -- start start 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c87959f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c87959f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c879515ea0 con 0x55c8791f8070 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c8795052d0 con 0x55c8791cf1b0 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.679+0000 7f7f9c623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8795090e0 con 0x55c8791d7390 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.680+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 0x7f7f7c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.680+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.680+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:56266/0 (socket says 172.21.15.83:56266) 2024-09-21T03:28:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.680+0000 7f7f8ffff640 1 -- 172.21.15.83:0/3849143752 learned_addr learned my addr 172.21.15.83:0/3849143752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.681+0000 7f7f8ffff640 1 -- 172.21.15.83:0/3849143752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 msgr2=0x55c87959f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.681+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c87959f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.681+0000 7f7f8ffff640 1 -- 172.21.15.83:0/3849143752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 msgr2=0x55c87959f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.681+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c87959f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.681+0000 7f7f8ffff640 1 -- 172.21.15.83:0/3849143752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c879263650 con 0x55c8791d7390 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.682+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 0x7f7f7c0b8cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f88013fd0 tx=0x7f7f880136e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.682+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009fc0 tx=0x7f7f8801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c11f032cb1aecc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.688+0000 7f7f85ffb640 1 -- 172.21.15.83:0/3849143752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88026070 con 0x55c8791d7390 2024-09-21T03:28:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.689+0000 7f7f85ffb640 1 -- 172.21.15.83:0/3849143752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8800ed60 con 0x55c8791d7390 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 -- 172.21.15.83:0/3849143752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 msgr2=0x55c87959fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f85ffb640 1 -- 172.21.15.83:0/3849143752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88026070 con 0x55c8791d7390 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009fc0 tx=0x7f7f8801d000 comp rx=0 tx=0).stop 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 -- 172.21.15.83:0/3849143752 shutdown_connections 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c87959fd40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c87959f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3849143752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c87959f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.690+0000 7f7f9c623740 1 -- 172.21.15.83:0/3849143752 >> 172.21.15.83:0/3849143752 conn(0x55c8791ec460 msgr2=0x55c87959d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.700+0000 7f7f9c623740 1 -- 172.21.15.83:0/3849143752 shutdown_connections 2024-09-21T03:28:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.701+0000 7f7f9c623740 1 -- 172.21.15.83:0/3849143752 wait complete. 2024-09-21T03:28:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.701+0000 7f7f9c623740 1 -- start start 2024-09-21T03:28:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.702+0000 7f7f9c623740 1 Processor -- start 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.702+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.702+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c8795aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.702+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f9c623740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c879366700 con 0x55c8791f8070 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f9c623740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c8794a0160 con 0x55c8791cf1b0 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f9c623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8795beaa0 con 0x55c8791d7390 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f9c623740 10 client.0 _ll_register_callbacks cb 0x55c8794b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44856/0 (socket says 172.21.15.83:44856) 2024-09-21T03:28:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.703+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1322252535 learned_addr learned my addr 172.21.15.83:0/1322252535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.704+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c8795aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.706+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 msgr2=0x55c8795aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.706+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c8795aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.706+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1322252535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 msgr2=0x55c8795c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.706+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.706+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1322252535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8794cdc90 con 0x55c8791cf1b0 2024-09-21T03:28:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.707+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f800058c0 tx=0x7f7f800cd9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e561a8e547af976 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.708+0000 7f7f6effd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.708+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800c92c0 con 0x55c8791cf1b0 2024-09-21T03:28:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.708+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7f800d7170 con 0x55c8791cf1b0 2024-09-21T03:28:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.709+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f8800ed60 con 0x55c8791cf1b0 2024-09-21T03:28:27.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.719+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7f80105960 con 0x55c8791cf1b0 2024-09-21T03:28:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.720+0000 7f7f9c623740 10 client.26888 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.720+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c8795ca800 con 0x55c8791cf1b0 2024-09-21T03:28:27.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.725+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8800ed60 con 0x55c8791cf1b0 2024-09-21T03:28:27.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.725+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f7f800cbb30 con 0x55c8791cf1b0 2024-09-21T03:28:27.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 10 client.26888 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 10 client.26888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 20 client.26888 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c8791b69f0 0x55c8795cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 4 client.26888 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c879517df0 con 0x55c8791b69f0 2024-09-21T03:28:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.729+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c8791b69f0 0x55c8795cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.730+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c8791b69f0 0x55c8795cdda0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdb229e220f224cf server_cookie=a098ceb99df7600b in_seq=0 out_seq=0 2024-09-21T03:28:27.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.730+0000 7f7f6effd640 10 client.26888 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:28:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.731+0000 7f7f6effd640 1 -- 172.21.15.83:0/1322252535 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c879517df0 con 0x55c8791b69f0 2024-09-21T03:28:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.731+0000 7f7f6effd640 10 client.26888 handle_command_reply: tid=0 2024-09-21T03:28:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.733+0000 7f7f9c623740 1 client.26888 shutdown 2024-09-21T03:28:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 msgr2=0x55c8795aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f800058c0 tx=0x7f7f800cd9b0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 shutdown_connections 2024-09-21T03:28:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c8791b69f0 0x55c8795cdda0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791d7390 0x55c8795aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1322252535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795aa950 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 >> 172.21.15.83:0/1322252535 conn(0x55c8791ec460 msgr2=0x55c8795a02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f6effd640 0 client.26888 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 shutdown_connections 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 1 -- 172.21.15.83:0/1322252535 wait complete. 2024-09-21T03:28:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.735+0000 7f7f9c623740 20 client.26888 trim_cache size 0 max 16384 2024-09-21T03:28:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.762+0000 7f7f9c623740 1 Processor -- start 2024-09-21T03:28:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f9c623740 1 -- start start 2024-09-21T03:28:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 0x55c8795a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.763+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:56282/0 (socket says 172.21.15.83:56282) 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 0x55c8795a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 0x55c8795a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44864/0 (socket says 172.21.15.83:44864) 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3510890702 learned_addr learned my addr 172.21.15.83:0/3510890702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ad690 con 0x55c8791f8070 2024-09-21T03:28:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901af0f0 con 0x55c8791d7390 2024-09-21T03:28:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901adc20 con 0x55c8791cf1b0 2024-09-21T03:28:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3510890702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 msgr2=0x55c8795a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 0x55c8795a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3510890702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 msgr2=0x55c8795c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.764+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/3510890702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8795a15b0 con 0x55c8791cf1b0 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.765+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f800cbef0 tx=0x7f7f80003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51472508668567ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.765+0000 7f7f857fa640 1 -- 172.21.15.83:0/3510890702 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8002be30 con 0x55c8791cf1b0 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.765+0000 7f7f857fa640 1 -- 172.21.15.83:0/3510890702 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f800cd6f0 con 0x55c8791cf1b0 2024-09-21T03:28:27.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.765+0000 7f7f857fa640 1 -- 172.21.15.83:0/3510890702 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80105760 con 0x55c8791cf1b0 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 msgr2=0x55c8795ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f800cbef0 tx=0x7f7f80003d60 comp rx=0 tx=0).stop 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 shutdown_connections 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791f8070 0x55c8795c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791d7390 0x55c8795a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 --2- 172.21.15.83:0/3510890702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791cf1b0 0x55c8795ac830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 >> 172.21.15.83:0/3510890702 conn(0x55c8791ec460 msgr2=0x55c8795a02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 shutdown_connections 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.767+0000 7f7f9c623740 1 -- 172.21.15.83:0/3510890702 wait complete. 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.768+0000 7f7f9c623740 1 -- start start 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.768+0000 7f7f9c623740 1 Processor -- start 2024-09-21T03:28:27.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.768+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f9c623740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 0x55c8795a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44866/0 (socket says 172.21.15.83:44866) 2024-09-21T03:28:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1641961377 learned_addr learned my addr 172.21.15.83:0/1641961377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791f8070 0x55c8795a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ae9c0 con 0x55c8791d7390 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ae2f0 con 0x55c8791cf1b0 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010cb40 con 0x55c8791f8070 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8ffff640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791f8070 0x55c8795a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 0x55c8795a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791f8070 msgr2=0x55c8795a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.769+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791f8070 0x55c8795a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1641961377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 msgr2=0x55c8795a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 0x55c8795a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f8f7fe640 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8796378a0 con 0x55c8791cf1b0 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f9c623740 10 client.0 _ll_register_callbacks cb 0x7f7f80005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f800cd6f0 con 0x55c8791cf1b0 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 0x55c8795a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f8f7fe640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7f800cadc0 tx=0x7f7f800cd9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2acb88e089b3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f6effd640 10 client.26894 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f6effd640 1 -- 172.21.15.83:0/1641961377 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800e8060 con 0x55c8791cf1b0 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f6effd640 1 -- 172.21.15.83:0/1641961377 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7f80105760 con 0x55c8791cf1b0 2024-09-21T03:28:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.770+0000 7f7f6effd640 1 -- 172.21.15.83:0/1641961377 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7f8002be30 con 0x55c8791cf1b0 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 10 client.26894 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55c8796378a0 con 0x55c8791cf1b0 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f6effd640 1 -- 172.21.15.83:0/1641961377 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f7f800cce40 con 0x55c8791cf1b0 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 10 client.26894 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 10 client.26894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:28:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 20 client.26894 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c87918bda0 0x55c87963aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 4 client.26894 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8795138a0 con 0x55c87918bda0 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.771+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c87918bda0 0x55c87963aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f6effd640 10 client.26894 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f87fff640 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c87918bda0 0x55c87963aca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97536f8cf035cc86 server_cookie=5f3e1f02d1d4e829 in_seq=0 out_seq=0 2024-09-21T03:28:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f6effd640 1 -- 172.21.15.83:0/1641961377 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8795138a0 con 0x55c87918bda0 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f6effd640 10 client.26894 handle_command_reply: tid=0 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 client.26894 shutdown 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 msgr2=0x55c8795a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7f800cadc0 tx=0x7f7f800cd9b0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 shutdown_connections 2024-09-21T03:28:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55c87918bda0 0x55c87963aca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8791f8070 0x55c8795a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8791d7390 0x55c8795a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 --2- 172.21.15.83:0/1641961377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8791cf1b0 0x55c8795a5c00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f6effd640 0 client.26894 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.772+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 >> 172.21.15.83:0/1641961377 conn(0x55c8791ec460 msgr2=0x55c8795ac520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.773+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 shutdown_connections 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.773+0000 7f7f9c623740 1 -- 172.21.15.83:0/1641961377 wait complete. 2024-09-21T03:28:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.773+0000 7f7f9c623740 20 client.26894 trim_cache size 0 max 16384 2024-09-21T03:28:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:28:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 msgr2=0x7f7f7c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 0x7f7f7c0b8cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f88013fd0 tx=0x7f7f880136e0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 msgr2=0x7f7f9010ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 0x7f7f9010ba30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009850 tx=0x7f7f80005410 comp rx=0 tx=0).stop 2024-09-21T03:28:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 shutdown_connections 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7f7c0b67b0 0x7f7f7c0b8cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f90108480 0x7f7f901a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f90107b30 0x7f7f9010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 --2- 172.21.15.83:0/3969909863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f901021f0 0x7f7f9010ba30 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009850 tx=0x7f7f80005410 comp rx=0 tx=0).stop 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.774+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 >> 172.21.15.83:0/3969909863 conn(0x7f7f90077c60 msgr2=0x7f7f90100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.775+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 shutdown_connections 2024-09-21T03:28:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:27.775+0000 7f7f960af640 1 -- 172.21.15.83:0/3969909863 wait complete. 2024-09-21T03:28:28.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:28:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:29.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:29 smithi083 ceph-mon[31219]: pgmap v691: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-09-21T03:28:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:29 smithi193 ceph-mon[35224]: pgmap v691: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-09-21T03:28:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:29 smithi142 ceph-mon[34512]: pgmap v691: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-09-21T03:28:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 -- 172.21.15.83:0/1666412817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c1083c0 msgr2=0x7fb40c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 --2- 172.21.15.83:0/1666412817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c1083c0 0x7fb40c1087d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009ca0 tx=0x7fb3fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 -- 172.21.15.83:0/1666412817 shutdown_connections 2024-09-21T03:28:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 --2- 172.21.15.83:0/1666412817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c108d10 0x7fb40c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 --2- 172.21.15.83:0/1666412817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c1083c0 0x7fb40c1087d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 --2- 172.21.15.83:0/1666412817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c102a80 0x7fb40c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 -- 172.21.15.83:0/1666412817 >> 172.21.15.83:0/1666412817 conn(0x7fb40c079740 msgr2=0x7fb40c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 -- 172.21.15.83:0/1666412817 shutdown_connections 2024-09-21T03:28:29.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.561+0000 7fb411a44640 1 -- 172.21.15.83:0/1666412817 wait complete. 2024-09-21T03:28:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.563+0000 7fb411a44640 1 Processor -- start 2024-09-21T03:28:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 -- start start 2024-09-21T03:28:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c102a80 0x7fb40c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 0x7fb40c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113a80 con 0x7fb40c108d10 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113900 con 0x7fb40c1083c0 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb411a44640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c113c00 con 0x7fb40c102a80 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb40a7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 0x7fb40c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37538/0 (socket says 172.21.15.83:37538) 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb40b7fe640 1 -- 172.21.15.83:0/499079458 learned_addr learned my addr 172.21.15.83:0/499079458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.564+0000 7fb40affd640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c102a80 0x7fb40c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 -- 172.21.15.83:0/499079458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c102a80 msgr2=0x7fb40c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c102a80 0x7fb40c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 -- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 msgr2=0x7fb40c1ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 -- 172.21.15.83:0/499079458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1acd00 con 0x7fb40c1083c0 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 0x7fb40c1a1c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06aaa0 tx=0x7fb3fc0317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb410a42640 1 -- 172.21.15.83:0/499079458 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc003b70 con 0x7fb40c1083c0 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1aced0 con 0x7fb40c1083c0 2024-09-21T03:28:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb410a42640 1 -- 172.21.15.83:0/499079458 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc003d10 con 0x7fb40c1083c0 2024-09-21T03:28:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb410a42640 1 -- 172.21.15.83:0/499079458 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc02fa50 con 0x7fb40c1083c0 2024-09-21T03:28:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.565+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1ad270 con 0x7fb40c1083c0 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 Processor -- start 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 -- start start 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x55677002c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc64390 0x55677002d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55676ffa2ea0 con 0x55676fc5c1b0 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55676ff922d0 con 0x55676fc64390 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb417fb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55676ff960e0 con 0x55676fc85070 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.570+0000 7fb40affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37554/0 (socket says 172.21.15.83:37554) 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40affd640 1 -- 172.21.15.83:0/3377972168 learned_addr learned my addr 172.21.15.83:0/3377972168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb410a42640 1 -- 172.21.15.83:0/499079458 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb3fc02fbf0 con 0x7fb40c1083c0 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x55677002c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc64390 0x55677002d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3377972168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc64390 msgr2=0x55677002d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc64390 0x55677002d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3377972168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 msgr2=0x55677002cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3377972168 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55676fcf0650 con 0x55676fc85070 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40affd640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x55677002c6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb400010400 tx=0x7fb4000108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec78860c0772a82a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb3f1ffb640 1 -- 172.21.15.83:0/3377972168 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400011820 con 0x55676fc85070 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb3f1ffb640 1 -- 172.21.15.83:0/3377972168 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40001a380 con 0x55676fc85070 2024-09-21T03:28:29.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.571+0000 7fb3f1ffb640 1 -- 172.21.15.83:0/3377972168 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40001b640 con 0x55676fc85070 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.572+0000 7fb410a42640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 0x7fb3f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.572+0000 7fb410a42640 1 -- 172.21.15.83:0/499079458 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7fb3fc108720 con 0x7fb40c1083c0 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.572+0000 7fb40affd640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 0x7fb3f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.572+0000 7fb417fb8740 1 -- 172.21.15.83:0/3377972168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 msgr2=0x55677002c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.572+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x55677002c6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb400010400 tx=0x7fb4000108c0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 -- 172.21.15.83:0/3377972168 shutdown_connections 2024-09-21T03:28:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc64390 0x55677002d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x55677002cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3377972168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x55677002c6a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 -- 172.21.15.83:0/3377972168 >> 172.21.15.83:0/3377972168 conn(0x55676fc79460 msgr2=0x55677002a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 -- 172.21.15.83:0/3377972168 shutdown_connections 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 -- 172.21.15.83:0/3377972168 wait complete. 2024-09-21T03:28:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb40affd640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 0x7fb3f40b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55677002b320 tx=0x7fb404005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.573+0000 7fb417fb8740 1 -- start start 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.574+0000 7fb417fb8740 1 Processor -- start 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x556770044170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55676fdf3700 con 0x55676fc64390 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55676ff2d160 con 0x55676fc5c1b0 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55677003ae80 con 0x55676fc85070 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb40affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb417fb8740 10 client.0 _ll_register_callbacks cb 0x55676ff469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb40affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42816/0 (socket says 172.21.15.83:42816) 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb40affd640 1 -- 172.21.15.83:0/692890611 learned_addr learned my addr 172.21.15.83:0/692890611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.575+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x556770044170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 -- 172.21.15.83:0/692890611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 msgr2=0x556770044170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x556770044170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 -- 172.21.15.83:0/692890611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 msgr2=0x5567700382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 -- 172.21.15.83:0/692890611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55676ff5ac90 con 0x55676fc5c1b0 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x556770044170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb40affd640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb40400fa00 tx=0x7fb404004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22294bc001b3cbd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb3eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40402d060 con 0x55676fc5c1b0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7fb404018030 con 0x55676fc5c1b0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40001a380 con 0x55676fc5c1b0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.576+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb40401d620 con 0x55676fc5c1b0 2024-09-21T03:28:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 10 client.26906 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556770046c50 con 0x55676fc5c1b0 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40001a380 con 0x55676fc5c1b0 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fb40401c840 con 0x55676fc5c1b0 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 10 client.26906 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 10 client.26906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 20 client.26906 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc439f0 0x55677004a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 4 client.26906 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55676ffa4df0 con 0x55676fc439f0 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.577+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc439f0 0x55677004a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.578+0000 7fb3eaffd640 10 client.26906 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.578+0000 7fb40a7fc640 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc439f0 0x55677004a220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37bd75678fd8a2e4 server_cookie=bdd87147a28b66bd in_seq=0 out_seq=0 2024-09-21T03:28:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.579+0000 7fb3eaffd640 1 -- 172.21.15.83:0/692890611 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55676ffa4df0 con 0x55676fc439f0 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.579+0000 7fb3eaffd640 10 client.26906 handle_command_reply: tid=0 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.579+0000 7fb417fb8740 1 client.26906 shutdown 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 msgr2=0x556770037da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb40400fa00 tx=0x7fb404004730 comp rx=0 tx=0).stop 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 shutdown_connections 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc439f0 0x55677004a220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb3eaffd640 0 client.26906 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:28:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc85070 0x556770044170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 --2- 172.21.15.83:0/692890611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc5c1b0 0x556770037da0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 >> 172.21.15.83:0/692890611 conn(0x55676fc79460 msgr2=0x55677002f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 shutdown_connections 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 1 -- 172.21.15.83:0/692890611 wait complete. 2024-09-21T03:28:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.580+0000 7fb417fb8740 20 client.26906 trim_cache size 0 max 16384 2024-09-21T03:28:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 Processor -- start 2024-09-21T03:28:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 -- start start 2024-09-21T03:28:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x5567700399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc64390 0x556770034f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b5bc0 con 0x55676fc5c1b0 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b7760 con 0x55676fc85070 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb417fb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b6290 con 0x55676fc64390 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42826/0 (socket says 172.21.15.83:42826) 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.597+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3419957537 learned_addr learned my addr 172.21.15.83:0/3419957537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3419957537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc64390 msgr2=0x556770034f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc64390 0x556770034f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3419957537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 msgr2=0x5567700399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x5567700399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 -- 172.21.15.83:0/3419957537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677002ebe0 con 0x55676fc85070 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.598+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb40000bcd0 tx=0x7fb40000bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7b3f00e2e11a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.602+0000 7fb3f17fa640 1 -- 172.21.15.83:0/3419957537 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40002b070 con 0x55676fc85070 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.602+0000 7fb3f17fa640 1 -- 172.21.15.83:0/3419957537 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4000054f0 con 0x55676fc85070 2024-09-21T03:28:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.602+0000 7fb3f17fa640 1 -- 172.21.15.83:0/3419957537 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400004660 con 0x55676fc85070 2024-09-21T03:28:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.608+0000 7fb417fb8740 1 -- 172.21.15.83:0/3419957537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 msgr2=0x5567700354d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.608+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb40000bcd0 tx=0x7fb40000bd00 comp rx=0 tx=0).stop 2024-09-21T03:28:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.614+0000 7fb417fb8740 1 -- 172.21.15.83:0/3419957537 shutdown_connections 2024-09-21T03:28:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.614+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700354d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.614+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc64390 0x556770034f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.614+0000 7fb417fb8740 1 --2- 172.21.15.83:0/3419957537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc5c1b0 0x5567700399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.614+0000 7fb417fb8740 1 -- 172.21.15.83:0/3419957537 >> 172.21.15.83:0/3419957537 conn(0x55676fc79460 msgr2=0x5567700392c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.621+0000 7fb417fb8740 1 -- 172.21.15.83:0/3419957537 shutdown_connections 2024-09-21T03:28:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.623+0000 7fb417fb8740 1 -- 172.21.15.83:0/3419957537 wait complete. 2024-09-21T03:28:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.624+0000 7fb417fb8740 1 -- start start 2024-09-21T03:28:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.624+0000 7fb417fb8740 1 Processor -- start 2024-09-21T03:28:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc5c1b0 0x556770040860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b7030 con 0x55676fc64390 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b6960 con 0x55676fc85070 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b3bc0 con 0x55676fc5c1b0 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42838/0 (socket says 172.21.15.83:42838) 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb40b7fe640 1 -- 172.21.15.83:0/653130177 learned_addr learned my addr 172.21.15.83:0/653130177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.625+0000 7fb417fb8740 10 client.0 _ll_register_callbacks cb 0x7fb404009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.626+0000 7fb40b7fe640 1 -- 172.21.15.83:0/653130177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc5c1b0 msgr2=0x556770040860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:28:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.626+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc5c1b0 0x556770040860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.626+0000 7fb40b7fe640 1 -- 172.21.15.83:0/653130177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 msgr2=0x5567700afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.626+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.626+0000 7fb40b7fe640 1 -- 172.21.15.83:0/653130177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567700b0850 con 0x55676fc85070 2024-09-21T03:28:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.627+0000 7fb40b7fe640 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb40001ab00 tx=0x7fb40001ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa6340b00ad6ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.627+0000 7fb3eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.627+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400034060 con 0x55676fc85070 2024-09-21T03:28:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.627+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7fb40001a580 con 0x55676fc85070 2024-09-21T03:28:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.627+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4000054f0 con 0x55676fc85070 2024-09-21T03:28:29.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.628+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb40002b040 con 0x55676fc85070 2024-09-21T03:28:29.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.628+0000 7fb417fb8740 10 client.26912 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:29.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.628+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5567700d6c00 con 0x55676fc85070 2024-09-21T03:28:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb400011ad0 con 0x55676fc85070 2024-09-21T03:28:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fb40002e030 con 0x55676fc85070 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 10 client.26912 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 10 client.26912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 20 client.26912 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc18da0 0x5567700d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 4 client.26912 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:28:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.632+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb4000054f0 con 0x55676fc18da0 2024-09-21T03:28:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.635+0000 7fb40affd640 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc18da0 0x5567700d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.643+0000 7fb3eaffd640 10 client.26912 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:28:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.643+0000 7fb40affd640 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc18da0 0x5567700d8ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4ade08ef42f80fb server_cookie=655dda31d3ca1707 in_seq=0 out_seq=0 2024-09-21T03:28:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.645+0000 7fb3eaffd640 1 -- 172.21.15.83:0/653130177 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb4000054f0 con 0x55676fc18da0 2024-09-21T03:28:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.645+0000 7fb3eaffd640 10 client.26912 handle_command_reply: tid=0 2024-09-21T03:28:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.645+0000 7fb417fb8740 1 client.26912 shutdown 2024-09-21T03:28:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.645+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 msgr2=0x5567700b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.645+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb40001ab00 tx=0x7fb40001ab30 comp rx=0 tx=0).stop 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 shutdown_connections 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55676fc18da0 0x5567700d8ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55676fc85070 0x5567700b0110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55676fc64390 0x5567700afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 --2- 172.21.15.83:0/653130177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55676fc5c1b0 0x556770040860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 >> 172.21.15.83:0/653130177 conn(0x55676fc79460 msgr2=0x556770039210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb3eaffd640 0 client.26912 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:28:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 shutdown_connections 2024-09-21T03:28:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 1 -- 172.21.15.83:0/653130177 wait complete. 2024-09-21T03:28:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.648+0000 7fb417fb8740 20 client.26912 trim_cache size 0 max 16384 2024-09-21T03:28:29.652 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.649+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 msgr2=0x7fb3f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.649+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 0x7fb3f40b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55677002b320 tx=0x7fb404005f90 comp rx=0 tx=0).stop 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.649+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 msgr2=0x7fb40c1a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.649+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 0x7fb40c1a1c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06aaa0 tx=0x7fb3fc0317c0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.649+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 shutdown_connections 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb3f40b6680 0x7fb3f40b8b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40c108d10 0x7fb40c1ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c1083c0 0x7fb40c1a1c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 --2- 172.21.15.83:0/499079458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c102a80 0x7fb40c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 >> 172.21.15.83:0/499079458 conn(0x7fb40c079740 msgr2=0x7fb40c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 shutdown_connections 2024-09-21T03:28:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:29.650+0000 7fb411a44640 1 -- 172.21.15.83:0/499079458 wait complete. 2024-09-21T03:28:30.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:28:30.824 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 670.0 s, 1661.8 tps, lat 59.073 ms stddev 66.524 2024-09-21T03:28:31.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:31 smithi083 ceph-mon[31219]: pgmap v692: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-21T03:28:31.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:31 smithi193 ceph-mon[35224]: pgmap v692: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-21T03:28:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:31 smithi142 ceph-mon[34512]: pgmap v692: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-21T03:28:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 -- 172.21.15.83:0/2420961167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc8111cd0 msgr2=0x7f9cc810d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2420961167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc8111cd0 0x7f9cc810d6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009900 tx=0x7f9cc002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 -- 172.21.15.83:0/2420961167 shutdown_connections 2024-09-21T03:28:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2420961167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc811cba0 0x7f9cc811efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2420961167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc810dbf0 0x7f9cc810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2420961167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc8111cd0 0x7f9cc810d6b0 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009900 tx=0x7f9cc002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.538+0000 7f9ccd641640 1 -- 172.21.15.83:0/2420961167 >> 172.21.15.83:0/2420961167 conn(0x7f9cc806e9f0 msgr2=0x7f9cc806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.543+0000 7f9ccd641640 1 -- 172.21.15.83:0/2420961167 shutdown_connections 2024-09-21T03:28:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.543+0000 7f9ccd641640 1 -- 172.21.15.83:0/2420961167 wait complete. 2024-09-21T03:28:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.544+0000 7f9ccd641640 1 Processor -- start 2024-09-21T03:28:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.544+0000 7f9ccd641640 1 -- start start 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.544+0000 7f9ccd641640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9ccd641640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc811cba0 0x7f9cc8115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9ccd641640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc8116300 0x7f9cc811b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9ccd641640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81226d0 con 0x7f9cc811cba0 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9ccd641640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8122550 con 0x7f9cc810dbf0 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9ccd641640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8122850 con 0x7f9cc8116300 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42854/0 (socket says 172.21.15.83:42854) 2024-09-21T03:28:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.545+0000 7f9cc7fff640 1 -- 172.21.15.83:0/2890261053 learned_addr learned my addr 172.21.15.83:0/2890261053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc8116300 0x7f9cc811b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc811cba0 0x7f9cc8115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 -- 172.21.15.83:0/2890261053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc8116300 msgr2=0x7f9cc811b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc8116300 0x7f9cc811b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 -- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc811cba0 msgr2=0x7f9cc8115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc811cba0 0x7f9cc8115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 -- 172.21.15.83:0/2890261053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc811b760 con 0x7f9cc810dbf0 2024-09-21T03:28:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.546+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006010 tx=0x7f9cc0005370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.547+0000 7f9cc57fa640 1 -- 172.21.15.83:0/2890261053 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0040030 con 0x7f9cc810dbf0 2024-09-21T03:28:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.547+0000 7f9cc57fa640 1 -- 172.21.15.83:0/2890261053 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc003c8f0 con 0x7f9cc810dbf0 2024-09-21T03:28:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.547+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc811b9f0 con 0x7f9cc810dbf0 2024-09-21T03:28:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.547+0000 7f9cc57fa640 1 -- 172.21.15.83:0/2890261053 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0031840 con 0x7f9cc810dbf0 2024-09-21T03:28:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.547+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc811be10 con 0x7f9cc810dbf0 2024-09-21T03:28:31.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.553+0000 7f9cd4e17740 1 Processor -- start 2024-09-21T03:28:31.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.554+0000 7f9cc57fa640 1 -- 172.21.15.83:0/2890261053 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9cc0043030 con 0x7f9cc810dbf0 2024-09-21T03:28:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 -- start start 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea7b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea7bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ff6e9f1ea0 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ff6e9e12d0 con 0x55ff6e6d4070 2024-09-21T03:28:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ff6e9e50e0 con 0x55ff6e6b3390 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9ccce40640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9ccce40640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42858/0 (socket says 172.21.15.83:42858) 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.555+0000 7f9ccce40640 1 -- 172.21.15.83:0/665789197 learned_addr learned my addr 172.21.15.83:0/665789197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea7b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9cc57fa640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 0x7f9cb00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea7bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 -- 172.21.15.83:0/665789197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 msgr2=0x55ff6ea7bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea7bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9cc57fa640 1 -- 172.21.15.83:0/2890261053 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9cc0108030 con 0x7f9cc810dbf0 2024-09-21T03:28:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 -- 172.21.15.83:0/665789197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 msgr2=0x55ff6ea7b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea7b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 -- 172.21.15.83:0/665789197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6e73f650 con 0x55ff6e6d4070 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.556+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 0x7f9cb00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.557+0000 7f9ccce40640 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009cb0 tx=0x7f9cbc0137d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94227dd6785f86ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.557+0000 7f9cae7fc640 1 -- 172.21.15.83:0/665789197 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc01b4d0 con 0x55ff6e6d4070 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.557+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 0x7f9cb00b8c90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc022000 tx=0x7f9cbc01f000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.557+0000 7f9cae7fc640 1 -- 172.21.15.83:0/665789197 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cbc010880 con 0x55ff6e6d4070 2024-09-21T03:28:31.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.557+0000 7f9cae7fc640 1 -- 172.21.15.83:0/665789197 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00e6b0 con 0x55ff6e6d4070 2024-09-21T03:28:31.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 -- 172.21.15.83:0/665789197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 msgr2=0x55ff6ea7b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009cb0 tx=0x7f9cbc0137d0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 -- 172.21.15.83:0/665789197 shutdown_connections 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea7bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea7b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/665789197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea7b460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.559+0000 7f9cd4e17740 1 -- 172.21.15.83:0/665789197 >> 172.21.15.83:0/665789197 conn(0x55ff6e6c8460 msgr2=0x55ff6ea79350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.560+0000 7f9cd4e17740 1 -- 172.21.15.83:0/665789197 shutdown_connections 2024-09-21T03:28:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.560+0000 7f9cd4e17740 1 -- 172.21.15.83:0/665789197 wait complete. 2024-09-21T03:28:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.560+0000 7f9cd4e17740 1 -- start start 2024-09-21T03:28:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.561+0000 7f9cd4e17740 1 Processor -- start 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.561+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.561+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37624/0 (socket says 172.21.15.83:37624) 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:32966/0 (socket says 172.21.15.83:32966) 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cc7fff640 1 -- 172.21.15.83:0/2484862409 learned_addr learned my addr 172.21.15.83:0/2484862409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea92f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ff6e842700 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ff6e97c160 con 0x55ff6e6d4070 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ff6ea89c40 con 0x55ff6e6b3390 2024-09-21T03:28:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.562+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9cd4e17740 10 client.0 _ll_register_callbacks cb 0x55ff6e9959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 -- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 msgr2=0x55ff6ea870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 -- 172.21.15.83:0/2484862409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 msgr2=0x55ff6ea86b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6e9a9c90 con 0x55ff6e6d4070 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea92f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc011020 tx=0x7f9cbc00f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a970a1d7a08e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc010880 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2484862409 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0105e0 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.563+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2484862409 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9cbc01b4d0 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2484862409 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9cbc01b710 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 10 client.26924 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55ff6ea95790 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2484862409 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9cbc01b8b0 con 0x55ff6e6d4070 2024-09-21T03:28:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 10 client.26924 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 10 client.26924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 20 client.26924 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6929f0 0x55ff6ea98af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 4 client.26924 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff6e9f3df0 con 0x55ff6e6929f0 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.564+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6929f0 0x55ff6ea98af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.565+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6929f0 0x55ff6ea98af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9963a45c53eb411f server_cookie=569fe107da5f83c4 in_seq=0 out_seq=0 2024-09-21T03:28:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.565+0000 7f9ca77fe640 10 client.26924 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.567+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2484862409 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff6e9f3df0 con 0x55ff6e6929f0 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.567+0000 7f9ca77fe640 10 client.26924 handle_command_reply: tid=0 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.567+0000 7f9cd4e17740 1 client.26924 shutdown 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.568+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 msgr2=0x55ff6ea92f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.568+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea92f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc011020 tx=0x7f9cbc00f420 comp rx=0 tx=0).stop 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.568+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 shutdown_connections 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.568+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6929f0 0x55ff6ea98af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6d4070 0x55ff6ea92f30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9ca77fe640 0 client.26924 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2484862409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 >> 172.21.15.83:0/2484862409 conn(0x55ff6e6c8460 msgr2=0x55ff6ea7e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 shutdown_connections 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2484862409 wait complete. 2024-09-21T03:28:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.569+0000 7f9cd4e17740 20 client.26924 trim_cache size 0 max 16384 2024-09-21T03:28:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.586+0000 7f9cd4e17740 1 Processor -- start 2024-09-21T03:28:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 -- start start 2024-09-21T03:28:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea92e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c70f0 con 0x55ff6e6929f0 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c8c90 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c77c0 con 0x55ff6e6b3390 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea92e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37640/0 (socket says 172.21.15.83:37640) 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9cc7fff640 1 -- 172.21.15.83:0/1399216141 learned_addr learned my addr 172.21.15.83:0/1399216141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.587+0000 7f9ccce40640 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 -- 172.21.15.83:0/1399216141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 msgr2=0x55ff6ea78980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea78980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 -- 172.21.15.83:0/1399216141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 msgr2=0x55ff6ea92e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea92e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 -- 172.21.15.83:0/1399216141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6ea81790 con 0x55ff6e6929f0 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea92e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ca440 tx=0x7f9cc00f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a5e1e7b0907f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cadffb640 1 -- 172.21.15.83:0/1399216141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc004a040 con 0x55ff6e6929f0 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cadffb640 1 -- 172.21.15.83:0/1399216141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00cd980 con 0x55ff6e6929f0 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.588+0000 7f9cadffb640 1 -- 172.21.15.83:0/1399216141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b50c0 con 0x55ff6e6929f0 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 -- 172.21.15.83:0/1399216141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 msgr2=0x55ff6ea87020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ca440 tx=0x7f9cc00f1040 comp rx=0 tx=0).stop 2024-09-21T03:28:31.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 -- 172.21.15.83:0/1399216141 shutdown_connections 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6b3390 0x55ff6ea78980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea92e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/1399216141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6929f0 0x55ff6ea87020 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.589+0000 7f9cd4e17740 1 -- 172.21.15.83:0/1399216141 >> 172.21.15.83:0/1399216141 conn(0x55ff6e6c8460 msgr2=0x55ff6ea86910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.590+0000 7f9cd4e17740 1 -- 172.21.15.83:0/1399216141 shutdown_connections 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.590+0000 7f9cd4e17740 1 -- 172.21.15.83:0/1399216141 wait complete. 2024-09-21T03:28:31.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.590+0000 7f9cd4e17740 1 -- start start 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.590+0000 7f9cd4e17740 1 Processor -- start 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 0x55ff6ea8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6b3390 0x55ff6ea8a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 0x55ff6ea8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 0x55ff6ea8f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:32984/0 (socket says 172.21.15.83:32984) 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c8560 con 0x55ff6e6b3390 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c7e90 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cc77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42872/0 (socket says 172.21.15.83:42872) 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cc77fe640 1 -- 172.21.15.83:0/2973802146 learned_addr learned my addr 172.21.15.83:0/2973802146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81c50f0 con 0x55ff6e6929f0 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.591+0000 7f9cd4e17740 10 client.0 _ll_register_callbacks cb 0x7f9cbc009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9ccce40640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6b3390 0x55ff6ea8a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 -- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 msgr2=0x55ff6ea8f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 0x55ff6ea8f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 -- 172.21.15.83:0/2973802146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6b3390 msgr2=0x55ff6ea8a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6b3390 0x55ff6ea8a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 -- 172.21.15.83:0/2973802146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6ea8b130 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.592+0000 7f9cc77fe640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005fd0 tx=0x7f9cb8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b4570a78ae40b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.593+0000 7f9ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8021060 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9cb8010440 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc00cd980 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9cb8010640 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9cd4e17740 10 client.26930 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:31.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.594+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ff6eb03020 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb8010c90 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9cb801b030 con 0x55ff6e6ab1b0 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 10 client.26930 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 10 client.26930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 20 client.26930 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6d4070 0x55ff6eb05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 4 client.26930 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9cc00cd980 con 0x55ff6e6d4070 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.595+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6d4070 0x55ff6eb05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9cc7fff640 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6d4070 0x55ff6eb05790 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8562cd709db24862 server_cookie=7852afcc17ddf8a2 in_seq=0 out_seq=0 2024-09-21T03:28:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9ca77fe640 10 client.26930 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9ca77fe640 1 -- 172.21.15.83:0/2973802146 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9cc00cd980 con 0x55ff6e6d4070 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9ca77fe640 10 client.26930 handle_command_reply: tid=0 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9cd4e17740 1 client.26930 shutdown 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 msgr2=0x55ff6ea8f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.596+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005fd0 tx=0x7f9cb8011440 comp rx=0 tx=0).stop 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 shutdown_connections 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55ff6e6d4070 0x55ff6eb05790 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff6e6b3390 0x55ff6ea8a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ff6e6ab1b0 0x55ff6ea8f8b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 --2- 172.21.15.83:0/2973802146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ff6e6929f0 0x55ff6ea8f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 >> 172.21.15.83:0/2973802146 conn(0x55ff6e6c8460 msgr2=0x55ff6ea7c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9ca77fe640 0 client.26930 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:28:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 shutdown_connections 2024-09-21T03:28:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 1 -- 172.21.15.83:0/2973802146 wait complete. 2024-09-21T03:28:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.597+0000 7f9cd4e17740 20 client.26930 trim_cache size 0 max 16384 2024-09-21T03:28:31.602 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 msgr2=0x7f9cb00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 0x7f9cb00b8c90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc022000 tx=0x7f9cbc01f000 comp rx=0 tx=0).stop 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 msgr2=0x7f9cc8115880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006010 tx=0x7f9cc0005370 comp rx=0 tx=0).stop 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 shutdown_connections 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9cb00b6780 0x7f9cb00b8c90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc8116300 0x7f9cc811b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc811cba0 0x7f9cc8115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 --2- 172.21.15.83:0/2890261053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc810dbf0 0x7f9cc8115880 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 >> 172.21.15.83:0/2890261053 conn(0x7f9cc806e9f0 msgr2=0x7f9cc8110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 shutdown_connections 2024-09-21T03:28:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:31.598+0000 7f9ccd641640 1 -- 172.21.15.83:0/2890261053 wait complete. 2024-09-21T03:28:32.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:28:33.024 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:33.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:33 smithi193 ceph-mon[35224]: pgmap v693: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2024-09-21T03:28:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:33 smithi083 ceph-mon[31219]: pgmap v693: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2024-09-21T03:28:33.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:33 smithi142 ceph-mon[34512]: pgmap v693: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.858+0000 7f7b34956640 1 -- 172.21.15.83:0/4282088993 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00a0d0 con 0x7f7b30110210 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 -- 172.21.15.83:0/4282088993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 msgr2=0x7f7b3007b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4282088993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b3007b690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c008570 tx=0x7f7b2c02e3b0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 -- 172.21.15.83:0/4282088993 shutdown_connections 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4282088993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b3007b690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4282088993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 0x7f7b3010fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4282088993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b3010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.859+0000 7f7b3695a640 1 -- 172.21.15.83:0/4282088993 >> 172.21.15.83:0/4282088993 conn(0x7f7b3006e9f0 msgr2=0x7f7b3006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.860+0000 7f7b3695a640 1 -- 172.21.15.83:0/4282088993 shutdown_connections 2024-09-21T03:28:33.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.860+0000 7f7b3695a640 1 -- 172.21.15.83:0/4282088993 wait complete. 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 Processor -- start 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 -- start start 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b300805c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 0x7f7b30080b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b30081040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3007ecc0 con 0x7f7b3010b8e0 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3007eb40 con 0x7f7b30110210 2024-09-21T03:28:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b3695a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3007ee40 con 0x7f7b3010f8c0 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b36159640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b30081040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b35958640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b300805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b35958640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b300805c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37674/0 (socket says 172.21.15.83:37674) 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.861+0000 7f7b35958640 1 -- 172.21.15.83:0/4202098118 learned_addr learned my addr 172.21.15.83:0/4202098118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b35157640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 0x7f7b30080b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 -- 172.21.15.83:0/4202098118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 msgr2=0x7f7b30080b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 0x7f7b30080b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 -- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 msgr2=0x7f7b300805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b300805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 -- 172.21.15.83:0/4202098118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b301c6130 con 0x7f7b30110210 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.862+0000 7f7b36159640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b30081040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0084c0 tx=0x7f7b2c004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.863+0000 7f7b26ffd640 1 -- 172.21.15.83:0/4202098118 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00f290 con 0x7f7b30110210 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.863+0000 7f7b26ffd640 1 -- 172.21.15.83:0/4202098118 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2c0035f0 con 0x7f7b30110210 2024-09-21T03:28:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.864+0000 7f7b26ffd640 1 -- 172.21.15.83:0/4202098118 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00a330 con 0x7f7b30110210 2024-09-21T03:28:33.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.864+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b301c63c0 con 0x7f7b30110210 2024-09-21T03:28:33.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.864+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301c6900 con 0x7f7b30110210 2024-09-21T03:28:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.868+0000 7f7b3e130740 1 Processor -- start 2024-09-21T03:28:33.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 -- start start 2024-09-21T03:28:33.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b26ffd640 1 -- 172.21.15.83:0/4202098118 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7b2c005350 con 0x7f7b30110210 2024-09-21T03:28:33.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f87070 0x56312432e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312432eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631242a4ea0 con 0x563123f5e1b0 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5631242942d0 con 0x563123f87070 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.870+0000 7f7b3e130740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631242980e0 con 0x563123f66390 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b35958640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b35958640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37684/0 (socket says 172.21.15.83:37684) 2024-09-21T03:28:33.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b35958640 1 -- 172.21.15.83:0/2836274901 learned_addr learned my addr 172.21.15.83:0/2836274901 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:33.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b35157640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312432eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b35157640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f87070 0x56312432e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.871+0000 7f7b26ffd640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 0x7f7b1c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b36159640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 0x7f7b1c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 -- 172.21.15.83:0/2836274901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f87070 msgr2=0x56312432e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f87070 0x56312432e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 -- 172.21.15.83:0/2836274901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 msgr2=0x56312432e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 -- 172.21.15.83:0/2836274901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563123ff2650 con 0x563123f66390 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35958640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b35157640 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312432eee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006c680 tx=0x7f7b28009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a31216c165525e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.872+0000 7f7b26ffd640 1 -- 172.21.15.83:0/4202098118 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7b2c1069e0 con 0x7f7b30110210 2024-09-21T03:28:33.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.873+0000 7f7b36159640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 0x7f7b1c0b8ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0cc7b0 tx=0x7f7b2c00a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.873+0000 7f7b1bfff640 1 -- 172.21.15.83:0/2836274901 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28014040 con 0x563123f66390 2024-09-21T03:28:33.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.873+0000 7f7b1bfff640 1 -- 172.21.15.83:0/2836274901 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2801b430 con 0x563123f66390 2024-09-21T03:28:33.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.873+0000 7f7b1bfff640 1 -- 172.21.15.83:0/2836274901 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28014040 con 0x563123f66390 2024-09-21T03:28:33.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 -- 172.21.15.83:0/2836274901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 msgr2=0x56312432eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312432eee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006c680 tx=0x7f7b28009b20 comp rx=0 tx=0).stop 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 -- 172.21.15.83:0/2836274901 shutdown_connections 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312432eee0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f5e1b0 0x56312432e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 --2- 172.21.15.83:0/2836274901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f87070 0x56312432e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 -- 172.21.15.83:0/2836274901 >> 172.21.15.83:0/2836274901 conn(0x563123f7b460 msgr2=0x56312432c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 -- 172.21.15.83:0/2836274901 shutdown_connections 2024-09-21T03:28:33.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.875+0000 7f7b3e130740 1 -- 172.21.15.83:0/2836274901 wait complete. 2024-09-21T03:28:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.876+0000 7f7b3e130740 1 -- start start 2024-09-21T03:28:33.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.877+0000 7f7b3e130740 1 Processor -- start 2024-09-21T03:28:33.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.878+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.878+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312433a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.878+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x563124345ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.878+0000 7f7b35157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.878+0000 7f7b35157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42920/0 (socket says 172.21.15.83:42920) 2024-09-21T03:28:33.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.879+0000 7f7b35157640 1 -- 172.21.15.83:0/1772353613 learned_addr learned my addr 172.21.15.83:0/1772353613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:33.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.879+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631240f5700 con 0x563123f87070 2024-09-21T03:28:33.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.879+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56312422f160 con 0x563123f5e1b0 2024-09-21T03:28:33.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.879+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56312433cbe0 con 0x563123f66390 2024-09-21T03:28:33.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.879+0000 7f7b3e130740 10 client.0 _ll_register_callbacks cb 0x5631242489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:33.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x563124345ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 -- 172.21.15.83:0/1772353613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 msgr2=0x56312433a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312433a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 -- 172.21.15.83:0/1772353613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 msgr2=0x563124345ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x563124345ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.880+0000 7f7b35157640 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312425cc90 con 0x563123f5e1b0 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.881+0000 7f7b35157640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b2801be40 tx=0x7f7b2800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87770eb914f542f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.881+0000 7f7b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:28:33.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.884+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28014040 con 0x563123f5e1b0 2024-09-21T03:28:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.884+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2801b430 con 0x563123f5e1b0 2024-09-21T03:28:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.884+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7b2800bce0 con 0x563123f5e1b0 2024-09-21T03:28:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.884+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7b28038050 con 0x563123f5e1b0 2024-09-21T03:28:33.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.885+0000 7f7b3e130740 10 client.26942 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:33.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.885+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5631243488f0 con 0x563123f5e1b0 2024-09-21T03:28:33.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.886+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2801b430 con 0x563123f5e1b0 2024-09-21T03:28:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.886+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f7b2802a900 con 0x563123f5e1b0 2024-09-21T03:28:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.886+0000 7f7b3e130740 10 client.26942 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.886+0000 7f7b3e130740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.886+0000 7f7b3e130740 20 client.26942 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.887+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563123f459f0 0x56312434c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.887+0000 7f7b3e130740 4 client.26942 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.887+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631242a6df0 con 0x563123f459f0 2024-09-21T03:28:33.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.890+0000 7f7b35958640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563123f459f0 0x56312434c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.898+0000 7f7b35958640 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563123f459f0 0x56312434c2f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b045b2889e9772a server_cookie=6611acf88ca0da46 in_seq=0 out_seq=0 2024-09-21T03:28:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.898+0000 7f7b197fa640 10 client.26942 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.901+0000 7f7b197fa640 1 -- 172.21.15.83:0/1772353613 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631242a6df0 con 0x563123f459f0 2024-09-21T03:28:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.901+0000 7f7b197fa640 10 client.26942 handle_command_reply: tid=0 2024-09-21T03:28:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.901+0000 7f7b3e130740 1 client.26942 shutdown 2024-09-21T03:28:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.906+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 msgr2=0x563124339b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.906+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b2801be40 tx=0x7f7b2800d040 comp rx=0 tx=0).stop 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 shutdown_connections 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563123f459f0 0x56312434c2f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x563124345ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f66390 0x56312433a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1772353613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f5e1b0 0x563124339b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 >> 172.21.15.83:0/1772353613 conn(0x563123f7b460 msgr2=0x563124331640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.907+0000 7f7b197fa640 0 client.26942 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.911+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 shutdown_connections 2024-09-21T03:28:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.912+0000 7f7b3e130740 1 -- 172.21.15.83:0/1772353613 wait complete. 2024-09-21T03:28:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.916+0000 7f7b3e130740 20 client.26942 trim_cache size 0 max 16384 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.942+0000 7f7b3e130740 1 Processor -- start 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 -- start start 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x563124339930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312432b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301cf020 con 0x563123f87070 2024-09-21T03:28:33.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301d0bc0 con 0x563123f66390 2024-09-21T03:28:33.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.943+0000 7f7b3e130740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301cf6f0 con 0x563123f5e1b0 2024-09-21T03:28:33.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.944+0000 7f7b35958640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.944+0000 7f7b35958640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42934/0 (socket says 172.21.15.83:42934) 2024-09-21T03:28:33.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.944+0000 7f7b35958640 1 -- 172.21.15.83:0/988521943 learned_addr learned my addr 172.21.15.83:0/988521943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35157640 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x563124339930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35958640 1 -- 172.21.15.83:0/988521943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 msgr2=0x563124339930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35958640 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x563124339930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35958640 1 -- 172.21.15.83:0/988521943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 msgr2=0x56312432b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35958640 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312432b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35958640 1 -- 172.21.15.83:0/988521943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631243312c0 con 0x563123f66390 2024-09-21T03:28:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.947+0000 7f7b35157640 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x563124339930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.956+0000 7f7b35958640 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000be80 tx=0x7f7b200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df63de9cadd5616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.957+0000 7f7b1b7fe640 1 -- 172.21.15.83:0/988521943 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20023030 con 0x563123f66390 2024-09-21T03:28:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.958+0000 7f7b1b7fe640 1 -- 172.21.15.83:0/988521943 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20005020 con 0x563123f66390 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 -- 172.21.15.83:0/988521943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 msgr2=0x563124339e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000be80 tx=0x7f7b200046c0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 -- 172.21.15.83:0/988521943 shutdown_connections 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312432b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f66390 0x563124339e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 --2- 172.21.15.83:0/988521943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x563124339930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.959+0000 7f7b3e130740 1 -- 172.21.15.83:0/988521943 >> 172.21.15.83:0/988521943 conn(0x563123f7b460 msgr2=0x563124338400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.960+0000 7f7b3e130740 1 -- 172.21.15.83:0/988521943 shutdown_connections 2024-09-21T03:28:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.960+0000 7f7b3e130740 1 -- 172.21.15.83:0/988521943 wait complete. 2024-09-21T03:28:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.960+0000 7f7b3e130740 1 -- start start 2024-09-21T03:28:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.960+0000 7f7b3e130740 1 Processor -- start 2024-09-21T03:28:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x5631243426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312433cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 0x56312433d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301d0490 con 0x563123f87070 2024-09-21T03:28:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301cfdc0 con 0x563123f1ada0 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301cd020 con 0x563123f5e1b0 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.961+0000 7f7b3e130740 10 client.0 _ll_register_callbacks cb 0x7f7b2800b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.962+0000 7f7b35157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 0x56312433d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.962+0000 7f7b35157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 0x56312433d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42938/0 (socket says 172.21.15.83:42938) 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.962+0000 7f7b35157640 1 -- 172.21.15.83:0/1511277315 learned_addr learned my addr 172.21.15.83:0/1511277315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.962+0000 7f7b35157640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x5631243426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.962+0000 7f7b35958640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312433cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 -- 172.21.15.83:0/1511277315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 msgr2=0x5631243426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x5631243426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 -- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 msgr2=0x56312433d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 0x56312433d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 -- 172.21.15.83:0/1511277315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312433db80 con 0x563123f87070 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b35958640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312433cdf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7b200047b0 tx=0x7f7b20004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e961e1cc8eb38c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.963+0000 7f7b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200044e0 con 0x563123f87070 2024-09-21T03:28:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631242a28a0 con 0x563123f87070 2024-09-21T03:28:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f7b20005020 con 0x563123f87070 2024-09-21T03:28:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7b20005240 con 0x563123f87070 2024-09-21T03:28:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b3e130740 10 client.17400 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.964+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5631243c6cc0 con 0x563123f87070 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5631242a28a0 con 0x563123f87070 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f7b2003b480 con 0x563123f87070 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 10 client.17400 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 20 client.17400 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631243c73b0 0x5631243c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 4 client.17400 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:28:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.967+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56312429fcd0 con 0x5631243c73b0 2024-09-21T03:28:33.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.973+0000 7f7b35157640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631243c73b0 0x5631243c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.991+0000 7f7b197fa640 10 client.17400 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:33.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.991+0000 7f7b35157640 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631243c73b0 0x5631243c98c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec17297e18f13cba server_cookie=f9c0fd92738f162b in_seq=0 out_seq=0 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.993+0000 7f7b197fa640 1 -- 172.21.15.83:0/1511277315 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56312429fcd0 con 0x5631243c73b0 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.993+0000 7f7b197fa640 10 client.17400 handle_command_reply: tid=0 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.993+0000 7f7b3e130740 1 client.17400 shutdown 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.993+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 msgr2=0x56312433cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.993+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312433cdf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7b200047b0 tx=0x7f7b20004ed0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 shutdown_connections 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5631243c73b0 0x5631243c98c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563123f1ada0 0x56312433d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563123f87070 0x56312433cdf0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 --2- 172.21.15.83:0/1511277315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563123f5e1b0 0x5631243426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 >> 172.21.15.83:0/1511277315 conn(0x563123f7b460 msgr2=0x563124337fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.994+0000 7f7b197fa640 0 client.17400 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:33.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:33.998+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 shutdown_connections 2024-09-21T03:28:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.000+0000 7f7b3e130740 1 -- 172.21.15.83:0/1511277315 wait complete. 2024-09-21T03:28:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.000+0000 7f7b3e130740 20 client.17400 trim_cache size 0 max 16384 2024-09-21T03:28:34.003 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:28:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.002+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 msgr2=0x7f7b1c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.002+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 0x7f7b1c0b8ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0cc7b0 tx=0x7f7b2c00a040 comp rx=0 tx=0).stop 2024-09-21T03:28:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.003+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 msgr2=0x7f7b30081040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.003+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b30081040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0084c0 tx=0x7f7b2c004dd0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 shutdown_connections 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7b1c0b65a0 0x7f7b1c0b8ab0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b30110210 0x7f7b30081040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b3010f8c0 0x7f7b30080b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 --2- 172.21.15.83:0/4202098118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3010b8e0 0x7f7b300805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 >> 172.21.15.83:0/4202098118 conn(0x7f7b3006e9f0 msgr2=0x7f7b3010d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 shutdown_connections 2024-09-21T03:28:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:34.004+0000 7f7b3695a640 1 -- 172.21.15.83:0/4202098118 wait complete. 2024-09-21T03:28:35.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:35 smithi193 ceph-mon[35224]: pgmap v694: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-21T03:28:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:35 smithi083 ceph-mon[31219]: pgmap v694: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-21T03:28:35.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:35 smithi142 ceph-mon[34512]: pgmap v694: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-21T03:28:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 675.0 s, 1493.8 tps, lat 67.974 ms stddev 127.632 2024-09-21T03:28:36.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:28:36.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:28:36.508 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:37.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 -- 172.21.15.83:0/334602336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 msgr2=0x7f923010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 --2- 172.21.15.83:0/334602336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923010d910 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9220009900 tx=0x7f922002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 -- 172.21.15.83:0/334602336 shutdown_connections 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 --2- 172.21.15.83:0/334602336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f923010de50 0x7f9230110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 --2- 172.21.15.83:0/334602336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923010d910 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 --2- 172.21.15.83:0/334602336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92301048a0 0x7f9230104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 -- 172.21.15.83:0/334602336 >> 172.21.15.83:0/334602336 conn(0x7f9230079790 msgr2=0x7f9230079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 -- 172.21.15.83:0/334602336 shutdown_connections 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.122+0000 7f923808a640 1 -- 172.21.15.83:0/334602336 wait complete. 2024-09-21T03:28:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.123+0000 7f923808a640 1 Processor -- start 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 -- start start 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 0x7f923019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113ae0 con 0x7f9230105280 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113960 con 0x7f92301048a0 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f923808a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113c60 con 0x7f923010de50 2024-09-21T03:28:37.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f9235e26640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f9235e26640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42958/0 (socket says 172.21.15.83:42958) 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f9235e26640 1 -- 172.21.15.83:0/214840429 learned_addr learned my addr 172.21.15.83:0/214840429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.124+0000 7f9235625640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9236627640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 0x7f923019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 -- 172.21.15.83:0/214840429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 msgr2=0x7f923019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 0x7f923019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 -- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 msgr2=0x7f923019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 -- 172.21.15.83:0/214840429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301a0400 con 0x7f92301048a0 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9236627640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 0x7f923019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.125+0000 7f9235e26640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f922c00e9f0 tx=0x7f922c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.126+0000 7f9226ffd640 1 -- 172.21.15.83:0/214840429 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c00cd80 con 0x7f92301048a0 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.126+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301a06f0 con 0x7f92301048a0 2024-09-21T03:28:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.126+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a6e50 con 0x7f92301048a0 2024-09-21T03:28:37.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.127+0000 7f9226ffd640 1 -- 172.21.15.83:0/214840429 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f922c0054d0 con 0x7f92301048a0 2024-09-21T03:28:37.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.127+0000 7f9226ffd640 1 -- 172.21.15.83:0/214840429 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c010430 con 0x7f92301048a0 2024-09-21T03:28:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.129+0000 7f9226ffd640 1 -- 172.21.15.83:0/214840429 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f922c0106b0 con 0x7f92301048a0 2024-09-21T03:28:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 Processor -- start 2024-09-21T03:28:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 -- start start 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 0x559f122dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f12255e60 con 0x559f11f0f490 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559f12245290 con 0x559f11f17670 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f923e5fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f122490a0 con 0x559f11f38350 2024-09-21T03:28:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f9235e26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f9236627640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f9236627640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:33084/0 (socket says 172.21.15.83:33084) 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.130+0000 7f9236627640 1 -- 172.21.15.83:0/3952816904 learned_addr learned my addr 172.21.15.83:0/3952816904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9235625640 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 0x559f122dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 -- 172.21.15.83:0/3952816904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 msgr2=0x559f122dfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 0x559f122dfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 -- 172.21.15.83:0/3952816904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 msgr2=0x559f122df970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122df970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 -- 172.21.15.83:0/3952816904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f11fa3930 con 0x559f11f38350 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9235625640 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 0x559f122dfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9236627640 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9228009f90 tx=0x7f922800f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d1d82be8490fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9226ffd640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 0x7f921c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f921bfff640 1 -- 172.21.15.83:0/3952816904 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228003b70 con 0x559f11f38350 2024-09-21T03:28:37.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f921bfff640 1 -- 172.21.15.83:0/3952816904 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9228003d10 con 0x559f11f38350 2024-09-21T03:28:37.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f921bfff640 1 -- 172.21.15.83:0/3952816904 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92280118b0 con 0x559f11f38350 2024-09-21T03:28:37.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.131+0000 7f9235e26640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 0x7f921c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.132+0000 7f9226ffd640 1 -- 172.21.15.83:0/214840429 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f922c0e7910 con 0x7f92301048a0 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.133+0000 7f923e5fe740 1 -- 172.21.15.83:0/3952816904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 msgr2=0x559f122df430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.133+0000 7f923e5fe740 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9228009f90 tx=0x7f922800f7f0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.133+0000 7f9235e26640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 0x7f921c0b8d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x559f122de0b0 tx=0x7f922c0e9040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 -- 172.21.15.83:0/3952816904 shutdown_connections 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f17670 0x559f122dfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 --2- 172.21.15.83:0/3952816904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122df430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 -- 172.21.15.83:0/3952816904 >> 172.21.15.83:0/3952816904 conn(0x559f11f2c740 msgr2=0x559f122dd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 -- 172.21.15.83:0/3952816904 shutdown_connections 2024-09-21T03:28:37.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.134+0000 7f923e5fe740 1 -- 172.21.15.83:0/3952816904 wait complete. 2024-09-21T03:28:37.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.135+0000 7f923e5fe740 1 -- start start 2024-09-21T03:28:37.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.135+0000 7f923e5fe740 1 Processor -- start 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f38350 0x559f122fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f120a6890 con 0x559f11f38350 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559f121dfd40 con 0x559f11f0f490 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f122f1f70 con 0x559f11f17670 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9236627640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f923e5fe740 10 client.0 _ll_register_callbacks cb 0x559f12243c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9236627640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42990/0 (socket says 172.21.15.83:42990) 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:33100/0 (socket says 172.21.15.83:33100) 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9236627640 1 -- 172.21.15.83:0/2797038260 learned_addr learned my addr 172.21.15.83:0/2797038260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9236627640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f38350 0x559f122fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 -- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 msgr2=0x559f122ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 -- 172.21.15.83:0/2797038260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f38350 msgr2=0x559f122fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f38350 0x559f122fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9235625640 1 -- 172.21.15.83:0/2797038260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f122fb870 con 0x559f11f17670 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.136+0000 7f9236627640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f9235625640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9220009230 tx=0x7f92200043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f4eb2343202e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f92197fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:28:37.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220035040 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9220005450 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9228003d10 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9220005650 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f923e5fe740 10 client.26908 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.137+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559f122fe0c0 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9228003d10 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9220003b30 con 0x559f11f17670 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 10 client.26908 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 20 client.26908 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ef6cd0 0x559f12301580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 4 client.26908 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f12257db0 con 0x559f11ef6cd0 2024-09-21T03:28:37.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.138+0000 7f9236627640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ef6cd0 0x559f12301580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.139+0000 7f92197fa640 10 client.26908 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.139+0000 7f9236627640 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ef6cd0 0x559f12301580 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fa6e6fe2e7f3f7 server_cookie=fd0171bcdfa26d4a in_seq=0 out_seq=0 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f92197fa640 1 -- 172.21.15.83:0/2797038260 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f12257db0 con 0x559f11ef6cd0 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f92197fa640 10 client.26908 handle_command_reply: tid=0 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 client.26908 shutdown 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 msgr2=0x559f122ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9220009230 tx=0x7f92200043b0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 shutdown_connections 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ef6cd0 0x559f12301580 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f38350 0x559f122fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122ef940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 --2- 172.21.15.83:0/2797038260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 >> 172.21.15.83:0/2797038260 conn(0x559f11f2c740 msgr2=0x559f122e0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f92197fa640 0 client.26908 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 shutdown_connections 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 1 -- 172.21.15.83:0/2797038260 wait complete. 2024-09-21T03:28:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.140+0000 7f923e5fe740 20 client.26908 trim_cache size 0 max 16384 2024-09-21T03:28:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.165+0000 7f923e5fe740 1 Processor -- start 2024-09-21T03:28:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.165+0000 7f923e5fe740 1 -- start start 2024-09-21T03:28:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f17670 0x559f122eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921400e660 con 0x559f11f17670 2024-09-21T03:28:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9214010200 con 0x559f11f0f490 2024-09-21T03:28:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f923e5fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f921400ed30 con 0x559f11f38350 2024-09-21T03:28:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:42994/0 (socket says 172.21.15.83:42994) 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 -- 172.21.15.83:0/274715722 learned_addr learned my addr 172.21.15.83:0/274715722 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 -- 172.21.15.83:0/274715722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 msgr2=0x559f122ec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122ec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 -- 172.21.15.83:0/274715722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f17670 msgr2=0x559f122eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f17670 0x559f122eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.166+0000 7f9236627640 1 -- 172.21.15.83:0/274715722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f122dd600 con 0x559f11f0f490 2024-09-21T03:28:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.167+0000 7f9236627640 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f922800fcf0 tx=0x7f9228003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb5c305ed573662 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.167+0000 7f921b7fe640 1 -- 172.21.15.83:0/274715722 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228003c60 con 0x559f11f0f490 2024-09-21T03:28:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.167+0000 7f921b7fe640 1 -- 172.21.15.83:0/274715722 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9228004450 con 0x559f11f0f490 2024-09-21T03:28:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.167+0000 7f921b7fe640 1 -- 172.21.15.83:0/274715722 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228011bc0 con 0x559f11f0f490 2024-09-21T03:28:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- 172.21.15.83:0/274715722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 msgr2=0x559f122fb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f922800fcf0 tx=0x7f9228003ec0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- 172.21.15.83:0/274715722 shutdown_connections 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f38350 0x559f122ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f17670 0x559f122eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 --2- 172.21.15.83:0/274715722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f0f490 0x559f122fb230 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- 172.21.15.83:0/274715722 >> 172.21.15.83:0/274715722 conn(0x559f11f2c740 msgr2=0x559f122ea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- 172.21.15.83:0/274715722 shutdown_connections 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- 172.21.15.83:0/274715722 wait complete. 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 -- start start 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.168+0000 7f923e5fe740 1 Processor -- start 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f38350 0x559f1236afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921400fad0 con 0x559f11f0f490 2024-09-21T03:28:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f921400f400 con 0x559f11f38350 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f921400c660 con 0x559f11f17670 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f9235625640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f9235625640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:33106/0 (socket says 172.21.15.83:33106) 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f9235625640 1 -- 172.21.15.83:0/215219203 learned_addr learned my addr 172.21.15.83:0/215219203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f9236627640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f923e5fe740 10 client.0 _ll_register_callbacks cb 0x7f922c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.169+0000 7f9236627640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f38350 0x559f1236afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.170+0000 7f9235625640 1 -- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f38350 msgr2=0x559f1236afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.170+0000 7f9235625640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f38350 0x559f1236afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.170+0000 7f9235625640 1 -- 172.21.15.83:0/215219203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 msgr2=0x559f122f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.170+0000 7f9235625640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.170+0000 7f9235625640 1 -- 172.21.15.83:0/215219203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1236b4f0 con 0x559f11f17670 2024-09-21T03:28:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.171+0000 7f9235625640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9220006010 tx=0x7f9220005580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8c7bad4de6decc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.171+0000 7f92197fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.172+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220035070 con 0x559f11f17670 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.172+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9220031da0 con 0x559f11f17670 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.172+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9228004450 con 0x559f11f17670 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9220009780 con 0x559f11f17670 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 10 client.26914 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559f1236c660 con 0x559f11f17670 2024-09-21T03:28:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9220003aa0 con 0x559f11f17670 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9220056030 con 0x559f11f17670 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 10 client.26914 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 20 client.26914 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ecc080 0x559f1236ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 4 client.26914 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.173+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9228004450 con 0x559f11ecc080 2024-09-21T03:28:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.174+0000 7f9236627640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ecc080 0x559f1236ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.174+0000 7f92197fa640 10 client.26914 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.174+0000 7f9236627640 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ecc080 0x559f1236ee00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704c7d711a4ed119 server_cookie=afddbee3ceca9b7e in_seq=0 out_seq=0 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f92197fa640 1 -- 172.21.15.83:0/215219203 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9228004450 con 0x559f11ecc080 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f92197fa640 10 client.26914 handle_command_reply: tid=0 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 client.26914 shutdown 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 msgr2=0x559f122f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9220006010 tx=0x7f9220005580 comp rx=0 tx=0).stop 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 shutdown_connections 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559f11ecc080 0x559f1236ee00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f92197fa640 0 client.26914 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f11f38350 0x559f1236afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f11f17670 0x559f122f3be0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.176+0000 7f923e5fe740 1 --2- 172.21.15.83:0/215219203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f11f0f490 0x559f122f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 >> 172.21.15.83:0/215219203 conn(0x559f11f2c740 msgr2=0x559f122e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 shutdown_connections 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923e5fe740 1 -- 172.21.15.83:0/215219203 wait complete. 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923e5fe740 20 client.26914 trim_cache size 0 max 16384 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea", 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:28:37.181 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 msgr2=0x7f921c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.177+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 0x7f921c0b8d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x559f122de0b0 tx=0x7f922c0e9040 comp rx=0 tx=0).stop 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 msgr2=0x7f923019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f922c00e9f0 tx=0x7f922c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 shutdown_connections 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f921c0b6850 0x7f921c0b8d60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f923010de50 0x7f923019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9230105280 0x7f923019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 --2- 172.21.15.83:0/214840429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92301048a0 0x7f923019a490 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 >> 172.21.15.83:0/214840429 conn(0x7f9230079790 msgr2=0x7f92301008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 shutdown_connections 2024-09-21T03:28:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:37.178+0000 7f923808a640 1 -- 172.21.15.83:0/214840429 wait complete. 2024-09-21T03:28:37.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:37 smithi193 ceph-mon[35224]: pgmap v695: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T03:28:37.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:37 smithi083 ceph-mon[31219]: pgmap v695: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T03:28:37.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:37 smithi142 ceph-mon[34512]: pgmap v695: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-21T03:28:38.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:28:38.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:38.375 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.375 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:38 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:28:38.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:38 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:38 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:38.910 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:39 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:28:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:39 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:39 smithi193 ceph-mon[35224]: pgmap v696: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T03:28:39.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:39 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:39 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:39 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:28:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:39 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:39 smithi083 ceph-mon[31219]: pgmap v696: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T03:28:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:39 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:39 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:39 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:28:39.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:39 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:39 smithi142 ceph-mon[34512]: pgmap v696: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T03:28:39.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:39 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:39 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.885+0000 7f9a427fc640 1 -- 172.21.15.83:0/3607972487 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c030710 con 0x7f9a44107b30 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 -- 172.21.15.83:0/3607972487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44107b30 msgr2=0x7f9a44107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 --2- 172.21.15.83:0/3607972487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44107b30 0x7f9a44107f40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00a970 tx=0x7f9a2c02b670 comp rx=0 tx=0).stop 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 -- 172.21.15.83:0/3607972487 shutdown_connections 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 --2- 172.21.15.83:0/3607972487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44108480 0x7f9a4410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 --2- 172.21.15.83:0/3607972487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44107b30 0x7f9a44107f40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 --2- 172.21.15.83:0/3607972487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a441021f0 0x7f9a44102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 -- 172.21.15.83:0/3607972487 >> 172.21.15.83:0/3607972487 conn(0x7f9a44077c60 msgr2=0x7f9a440780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 -- 172.21.15.83:0/3607972487 shutdown_connections 2024-09-21T03:28:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.886+0000 7f9a49e74640 1 -- 172.21.15.83:0/3607972487 wait complete. 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 Processor -- start 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 -- start start 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 0x7f9a44198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 0x7f9a44198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44108480 0x7f9a4419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44111930 con 0x7f9a441021f0 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441117b0 con 0x7f9a44108480 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a49e74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44111ab0 con 0x7f9a44107b30 2024-09-21T03:28:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 0x7f9a44198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 0x7f9a44198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44274/0 (socket says 172.21.15.83:44274) 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 0x7f9a44198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a437fe640 1 -- 172.21.15.83:0/903586342 learned_addr learned my addr 172.21.15.83:0/903586342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.887+0000 7f9a43fff640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44108480 0x7f9a4419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 -- 172.21.15.83:0/903586342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44108480 msgr2=0x7f9a4419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44108480 0x7f9a4419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 -- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 msgr2=0x7f9a44198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 0x7f9a44198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 -- 172.21.15.83:0/903586342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4419e2b0 con 0x7f9a44107b30 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 0x7f9a44198910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9a44075820 tx=0x7f9a2c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a40ff9640 1 -- 172.21.15.83:0/903586342 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c040070 con 0x7f9a44107b30 2024-09-21T03:28:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.888+0000 7f9a40ff9640 1 -- 172.21.15.83:0/903586342 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2c005640 con 0x7f9a44107b30 2024-09-21T03:28:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.889+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4419e540 con 0x7f9a44107b30 2024-09-21T03:28:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.889+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a4d30 con 0x7f9a44107b30 2024-09-21T03:28:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.889+0000 7f9a40ff9640 1 -- 172.21.15.83:0/903586342 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c045460 con 0x7f9a44107b30 2024-09-21T03:28:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.891+0000 7f9a503e8740 1 Processor -- start 2024-09-21T03:28:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 -- start start 2024-09-21T03:28:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a40ff9640 1 -- 172.21.15.83:0/903586342 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9a2c0046c0 con 0x7f9a44107b30 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d311127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d31112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d31088e50 con 0x557d30d4a390 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557d31078280 con 0x557d30d421b0 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a503e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d3107c090 con 0x557d30d6b070 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:49388/0 (socket says 172.21.15.83:49388) 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.892+0000 7f9a43fff640 1 -- 172.21.15.83:0/1250618292 learned_addr learned my addr 172.21.15.83:0/1250618292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d31112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a437fe640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d311127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 -- 172.21.15.83:0/1250618292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 msgr2=0x557d311127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d311127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 -- 172.21.15.83:0/1250618292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 msgr2=0x557d31112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d31112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 -- 172.21.15.83:0/1250618292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d30dd6650 con 0x557d30d6b070 2024-09-21T03:28:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d31112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a43fff640 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0099e0 tx=0x7f9a3c009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7143c06a6946c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a40ff9640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 0x7f9a300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a39ffb640 1 -- 172.21.15.83:0/1250618292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c017070 con 0x557d30d6b070 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a39ffb640 1 -- 172.21.15.83:0/1250618292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c013470 con 0x557d30d6b070 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.893+0000 7f9a39ffb640 1 -- 172.21.15.83:0/1250618292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c012610 con 0x557d30d6b070 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.894+0000 7f9a40ff9640 1 -- 172.21.15.83:0/903586342 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9a2c1087d0 con 0x7f9a44107b30 2024-09-21T03:28:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.894+0000 7f9a437fe640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 0x7f9a300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.897+0000 7f9a437fe640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 0x7f9a300b8c10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557d31110ef0 tx=0x7f9a34010040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.897+0000 7f9a503e8740 1 -- 172.21.15.83:0/1250618292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 msgr2=0x557d31112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.897+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0099e0 tx=0x7f9a3c009ea0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 -- 172.21.15.83:0/1250618292 shutdown_connections 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d31112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d311127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1250618292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31112270 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 -- 172.21.15.83:0/1250618292 >> 172.21.15.83:0/1250618292 conn(0x557d30d5f460 msgr2=0x557d31110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 -- 172.21.15.83:0/1250618292 shutdown_connections 2024-09-21T03:28:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 -- 172.21.15.83:0/1250618292 wait complete. 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.898+0000 7f9a503e8740 1 -- start start 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 Processor -- start 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31129d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d30ed9700 con 0x557d30d4a390 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557d31013110 con 0x557d30d421b0 2024-09-21T03:28:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41384/0 (socket says 172.21.15.83:41384) 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a437fe640 1 -- 172.21.15.83:0/1961960237 learned_addr learned my addr 172.21.15.83:0/1961960237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a43fff640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31129d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d31120a50 con 0x557d30d6b070 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.899+0000 7f9a503e8740 10 client.0 _ll_register_callbacks cb 0x557d3102c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 -- 172.21.15.83:0/1961960237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 msgr2=0x557d31129d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31129d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 -- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 msgr2=0x557d3111d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 -- 172.21.15.83:0/1961960237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d31040c40 con 0x557d30d4a390 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a437fe640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111de10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c105f70 tx=0x7f9a2c10e320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4697ac1384e70725 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0ce440 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9a2c0cd8d0 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3c013470 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9a2c0cdde0 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.900+0000 7f9a503e8740 10 client.17424 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557d3112c6c0 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c013470 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9a2c04d030 con 0x557d30d4a390 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 10 client.17424 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 20 client.17424 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d299f0 0x557d3112fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 4 client.17424 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3108ada0 con 0x557d30d299f0 2024-09-21T03:28:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.901+0000 7f9a437fe640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d299f0 0x557d3112fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.902+0000 7f9a437fe640 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d299f0 0x557d3112fc10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a547e321903602 server_cookie=9a2ce63c0acdab08 in_seq=0 out_seq=0 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.902+0000 7f9a22ffd640 10 client.17424 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a22ffd640 1 -- 172.21.15.83:0/1961960237 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d3108ada0 con 0x557d30d299f0 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a22ffd640 10 client.17424 handle_command_reply: tid=0 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 client.17424 shutdown 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 msgr2=0x557d3111de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111de10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c105f70 tx=0x7f9a2c10e320 comp rx=0 tx=0).stop 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 shutdown_connections 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d299f0 0x557d3112fc10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a22ffd640 0 client.17424 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d6b070 0x557d31129d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111de10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 --2- 172.21.15.83:0/1961960237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d3111d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 >> 172.21.15.83:0/1961960237 conn(0x557d30d5f460 msgr2=0x557d31115400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 shutdown_connections 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.903+0000 7f9a503e8740 1 -- 172.21.15.83:0/1961960237 wait complete. 2024-09-21T03:28:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.904+0000 7f9a503e8740 20 client.17424 trim_cache size 0 max 16384 2024-09-21T03:28:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.926+0000 7f9a503e8740 1 Processor -- start 2024-09-21T03:28:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.926+0000 7f9a503e8740 1 -- start start 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.926+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d31129c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:49410/0 (socket says 172.21.15.83:49410) 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d31129c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d31129c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:41388/0 (socket says 172.21.15.83:41388) 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a437fe640 1 -- 172.21.15.83:0/931480151 learned_addr learned my addr 172.21.15.83:0/931480151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a503e8740 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441b17a0 con 0x557d30d4a390 2024-09-21T03:28:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441b3340 con 0x557d30d421b0 2024-09-21T03:28:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441b1e70 con 0x557d30d299f0 2024-09-21T03:28:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.927+0000 7f9a43fff640 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 -- 172.21.15.83:0/931480151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 msgr2=0x557d31129c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d31129c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 -- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 msgr2=0x557d3111aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 -- 172.21.15.83:0/931480151 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d311150d0 con 0x557d30d299f0 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a43fff640 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a437fe640 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400edc0 tx=0x7f9a3400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea793e6951ff87ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.928+0000 7f9a397fa640 1 -- 172.21.15.83:0/931480151 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3402e650 con 0x557d30d299f0 2024-09-21T03:28:39.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.929+0000 7f9a397fa640 1 -- 172.21.15.83:0/931480151 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a34033070 con 0x557d30d299f0 2024-09-21T03:28:39.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.929+0000 7f9a397fa640 1 -- 172.21.15.83:0/931480151 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a340268f0 con 0x557d30d299f0 2024-09-21T03:28:39.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 msgr2=0x557d3111dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400edc0 tx=0x7f9a3400ef90 comp rx=0 tx=0).stop 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 shutdown_connections 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d4a390 0x557d3111aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d421b0 0x557d31129c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 --2- 172.21.15.83:0/931480151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d3111dd90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 >> 172.21.15.83:0/931480151 conn(0x557d30d5f460 msgr2=0x557d3111d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 shutdown_connections 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.930+0000 7f9a503e8740 1 -- 172.21.15.83:0/931480151 wait complete. 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 -- start start 2024-09-21T03:28:39.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 Processor -- start 2024-09-21T03:28:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 0x557d31122af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 0x557d311935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441b2c10 con 0x557d30d421b0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441b2540 con 0x557d30d4a390 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.931+0000 7f9a503e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441af7a0 con 0x557d30d299f0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a503e8740 10 client.0 _ll_register_callbacks cb 0x7f9a2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 0x557d31122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a43fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 0x557d311935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 0x557d31122af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44308/0 (socket says 172.21.15.83:44308) 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:49414/0 (socket says 172.21.15.83:49414) 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a42ffd640 1 -- 172.21.15.83:0/2778412006 learned_addr learned my addr 172.21.15.83:0/2778412006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 -- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 msgr2=0x557d311935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 0x557d311935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 -- 172.21.15.83:0/2778412006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 msgr2=0x557d31122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 0x557d31122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 -- 172.21.15.83:0/2778412006 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d31193c50 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a43fff640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 0x557d311935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a437fe640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400dac0 tx=0x7f9a340095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd0f36f2af8dd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34026980 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a34033070 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.932+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8441+0+0 (secure 0 0 0) 0x7f9a34026b20 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9a34047050 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 10 client.26932 fetch_fsmap learned FSMap version 46 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557d3119af60 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a34033070 con 0x557d30d299f0 2024-09-21T03:28:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9a3400b040 con 0x557d30d299f0 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 10 client.26932 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 20 client.26932 populate_metadata read hostname 'smithi083' 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d6b070 0x557d3119d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 4 client.26932 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d31086850 con 0x557d30d6b070 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.933+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d6b070 0x557d3119d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a22ffd640 10 client.26932 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:39.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a42ffd640 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d6b070 0x557d3119d750 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa676da22f1f0b7 server_cookie=76a53ee35af45a35 in_seq=0 out_seq=0 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a22ffd640 1 -- 172.21.15.83:0/2778412006 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x557d31086850 con 0x557d30d6b070 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a22ffd640 10 client.26932 handle_command_reply: tid=0 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a503e8740 1 client.26932 shutdown 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 msgr2=0x557d31122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.934+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400dac0 tx=0x7f9a340095c0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 shutdown_connections 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557d30d6b070 0x557d3119d750 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d30d4a390 0x557d311935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d30d421b0 0x557d31122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 --2- 172.21.15.83:0/2778412006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d30d299f0 0x557d31122530 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a22ffd640 0 client.26932 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 >> 172.21.15.83:0/2778412006 conn(0x557d30d5f460 msgr2=0x557d311132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 shutdown_connections 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 1 -- 172.21.15.83:0/2778412006 wait complete. 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.935+0000 7f9a503e8740 20 client.26932 trim_cache size 0 max 16384 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:28:39.941 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea", 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.942 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:28:39.943 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.945 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:28:39.946 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.947 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.948 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.949 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:28:39.950 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:28:39.951 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.938+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 msgr2=0x7f9a300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.938+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 0x7f9a300b8c10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557d31110ef0 tx=0x7f9a34010040 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 msgr2=0x7f9a44198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 0x7f9a44198910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9a44075820 tx=0x7f9a2c005040 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 shutdown_connections 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f9a300b6700 0x7f9a300b8c10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a44108480 0x7f9a4419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a44107b30 0x7f9a44198910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 --2- 172.21.15.83:0/903586342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441021f0 0x7f9a44198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 >> 172.21.15.83:0/903586342 conn(0x7f9a44077c60 msgr2=0x7f9a44105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 shutdown_connections 2024-09-21T03:28:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:28:39.939+0000 7f9a49e74640 1 -- 172.21.15.83:0/903586342 wait complete. 2024-09-21T03:28:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 680.0 s, 1714.0 tps, lat 58.003 ms stddev 71.979 2024-09-21T03:28:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:41 smithi083 ceph-mon[31219]: pgmap v697: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2024-09-21T03:28:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:41 smithi142 ceph-mon[34512]: pgmap v697: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2024-09-21T03:28:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:41 smithi193 ceph-mon[35224]: pgmap v697: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2024-09-21T03:28:41.808 INFO:tasks.cephfs.filesystem:scrub status for tag:1d7afd16-69f4-4d76-9a37-cc9fcfa820ea - {'path': '/', 'tag': '1d7afd16-69f4-4d76-9a37-cc9fcfa820ea', 'options': 'recursive,force'} 2024-09-21T03:28:43.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:43 smithi083 ceph-mon[31219]: pgmap v698: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-21T03:28:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:43 smithi142 ceph-mon[34512]: pgmap v698: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-21T03:28:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:43 smithi193 ceph-mon[35224]: pgmap v698: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 21 MiB/s wr, 1.69k op/s 2024-09-21T03:28:44.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:44 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:28:44.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:44 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:28:44.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:44 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:28:44.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:44 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:28:44.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:44 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:28:44.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:44 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:28:44.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:44 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:28:44.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:44 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:28:44.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:44 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:44.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:44 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:44.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:44 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:44.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:44 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:45 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:28:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:45 smithi083 ceph-mon[31219]: pgmap v699: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-09-21T03:28:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:45 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:45 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:28:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:45 smithi142 ceph-mon[34512]: pgmap v699: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-09-21T03:28:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:45 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:45.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:45 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:28:45.466 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:45 smithi193 ceph-mon[35224]: pgmap v699: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-09-21T03:28:45.466 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:45 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 685.0 s, 1874.2 tps, lat 53.604 ms stddev 63.015 2024-09-21T03:28:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:47 smithi083 ceph-mon[31219]: pgmap v700: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T03:28:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:47 smithi142 ceph-mon[34512]: pgmap v700: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T03:28:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:47 smithi193 ceph-mon[35224]: pgmap v700: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T03:28:49.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:49 smithi083 ceph-mon[31219]: pgmap v701: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-09-21T03:28:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:49 smithi142 ceph-mon[34512]: pgmap v701: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-09-21T03:28:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:49 smithi193 ceph-mon[35224]: pgmap v701: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-09-21T03:28:50.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:50 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:50 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:50 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:28:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 690.0 s, 1784.0 tps, lat 56.019 ms stddev 65.201 2024-09-21T03:28:51.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:51 smithi083 ceph-mon[31219]: pgmap v702: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T03:28:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:51 smithi142 ceph-mon[34512]: pgmap v702: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T03:28:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:51 smithi193 ceph-mon[35224]: pgmap v702: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T03:28:53.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:53 smithi083 ceph-mon[31219]: pgmap v703: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-09-21T03:28:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:53 smithi142 ceph-mon[34512]: pgmap v703: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-09-21T03:28:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:53 smithi193 ceph-mon[35224]: pgmap v703: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-09-21T03:28:55.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:55 smithi083 ceph-mon[31219]: pgmap v704: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.3 MiB/s wr, 637 op/s 2024-09-21T03:28:55.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:55 smithi142 ceph-mon[34512]: pgmap v704: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.3 MiB/s wr, 637 op/s 2024-09-21T03:28:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:55 smithi193 ceph-mon[35224]: pgmap v704: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.3 MiB/s wr, 637 op/s 2024-09-21T03:28:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 695.0 s, 1849.0 tps, lat 54.111 ms stddev 62.887 2024-09-21T03:28:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:57 smithi142 ceph-mon[34512]: pgmap v705: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-21T03:28:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:57 smithi193 ceph-mon[35224]: pgmap v705: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-21T03:28:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:57 smithi083 ceph-mon[31219]: pgmap v705: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 623 op/s 2024-09-21T03:28:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:28:59 smithi142 ceph-mon[34512]: pgmap v706: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 599 op/s 2024-09-21T03:28:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:28:59 smithi193 ceph-mon[35224]: pgmap v706: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 599 op/s 2024-09-21T03:28:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:28:59 smithi083 ceph-mon[31219]: pgmap v706: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 599 op/s 2024-09-21T03:29:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 700.0 s, 2035.4 tps, lat 49.425 ms stddev 59.420 2024-09-21T03:29:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:01 smithi142 ceph-mon[34512]: pgmap v707: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.2 MiB/s wr, 614 op/s 2024-09-21T03:29:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:01 smithi193 ceph-mon[35224]: pgmap v707: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.2 MiB/s wr, 614 op/s 2024-09-21T03:29:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:01 smithi083 ceph-mon[31219]: pgmap v707: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.2 MiB/s wr, 614 op/s 2024-09-21T03:29:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:03 smithi142 ceph-mon[34512]: pgmap v708: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-09-21T03:29:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:03 smithi193 ceph-mon[35224]: pgmap v708: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-09-21T03:29:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:03 smithi083 ceph-mon[31219]: pgmap v708: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 670 op/s 2024-09-21T03:29:05.425 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:05 smithi083 ceph-mon[31219]: pgmap v709: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-09-21T03:29:05.426 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:05 smithi083 ceph-mon[31219]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:29:05.426 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:05 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:05.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:05 smithi142 ceph-mon[34512]: pgmap v709: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-09-21T03:29:05.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:05 smithi142 ceph-mon[34512]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:29:05.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:05 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:05.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:05 smithi193 ceph-mon[35224]: pgmap v709: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-09-21T03:29:05.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:05 smithi193 ceph-mon[35224]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:29:05.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:05 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 705.0 s, 993.0 tps, lat 99.682 ms stddev 255.446 2024-09-21T03:29:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:06.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:06.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:29:06.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:06.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:06.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:29:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:29:07.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:07 smithi142 ceph-mon[34512]: pgmap v711: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-09-21T03:29:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:07 smithi193 ceph-mon[35224]: pgmap v711: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-09-21T03:29:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:07 smithi083 ceph-mon[31219]: pgmap v711: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-09-21T03:29:09.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:09 smithi142 ceph-mon[34512]: pgmap v712: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2024-09-21T03:29:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:09 smithi193 ceph-mon[35224]: pgmap v712: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2024-09-21T03:29:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:09 smithi083 ceph-mon[31219]: pgmap v712: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 25 MiB/s wr, 1.94k op/s 2024-09-21T03:29:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 710.0 s, 1394.6 tps, lat 72.067 ms stddev 76.431 2024-09-21T03:29:11.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:11 smithi142 ceph-mon[34512]: pgmap v713: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.97k op/s 2024-09-21T03:29:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:11 smithi193 ceph-mon[35224]: pgmap v713: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.97k op/s 2024-09-21T03:29:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:11 smithi083 ceph-mon[31219]: pgmap v713: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 25 MiB/s wr, 1.97k op/s 2024-09-21T03:29:11.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:29:12.197 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:12.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.765+0000 7fa3fc9cf640 1 -- 172.21.15.83:0/1896634847 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40307d0 con 0x7fa3f810a590 2024-09-21T03:29:12.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.766+0000 7fa3ffc35640 1 -- 172.21.15.83:0/1896634847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f810a590 msgr2=0x7fa3f810a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.766+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/1896634847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f810a590 0x7fa3f810a9a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009ca0 tx=0x7fa3f402a8d0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 -- 172.21.15.83:0/1896634847 shutdown_connections 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/1896634847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f8116640 0x7fa3f8118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/1896634847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 0x7fa3f8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/1896634847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f810a590 0x7fa3f810a9a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 -- 172.21.15.83:0/1896634847 >> 172.21.15.83:0/1896634847 conn(0x7fa3f8070ec0 msgr2=0x7fa3f80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 -- 172.21.15.83:0/1896634847 shutdown_connections 2024-09-21T03:29:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.767+0000 7fa3ffc35640 1 -- 172.21.15.83:0/1896634847 wait complete. 2024-09-21T03:29:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.768+0000 7fa3ffc35640 1 Processor -- start 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.770+0000 7fa3ffc35640 1 -- start start 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.770+0000 7fa3ffc35640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3ffc35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 0x7fa3f811c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3ffc35640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3ffc35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a1e0 con 0x7fa3f810aee0 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3ffc35640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a060 con 0x7fa3f8116640 2024-09-21T03:29:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fe1d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fe1d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:43984/0 (socket says 172.21.15.83:43984) 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/3979962914 learned_addr learned my addr 172.21.15.83:0/3979962914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 0x7fa3f811c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f811a360 con 0x7fa3f810a590 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.771+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46018/0 (socket says 172.21.15.83:46018) 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3979962914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 msgr2=0x7fa3f81216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 msgr2=0x7fa3f811c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 0x7fa3f811c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3979962914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f8121d10 con 0x7fa3f810a590 2024-09-21T03:29:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.772+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.774+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009bf0 tx=0x7fa3f4031c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.774+0000 7fa3eeffd640 1 -- 172.21.15.83:0/3979962914 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004040 con 0x7fa3f810a590 2024-09-21T03:29:12.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.775+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f8121fa0 con 0x7fa3f810a590 2024-09-21T03:29:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.775+0000 7fa3eeffd640 1 -- 172.21.15.83:0/3979962914 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f4004690 con 0x7fa3f810a590 2024-09-21T03:29:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.775+0000 7fa3eeffd640 1 -- 172.21.15.83:0/3979962914 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004040 con 0x7fa3f810a590 2024-09-21T03:29:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.775+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f81224e0 con 0x7fa3f810a590 2024-09-21T03:29:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.778+0000 7fa4061a9740 1 Processor -- start 2024-09-21T03:29:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.779+0000 7fa4061a9740 1 -- start start 2024-09-21T03:29:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.780+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51cf4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.780+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 0x561a51cf4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.780+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.780+0000 7fa4061a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a51c6ae50 con 0x561a5192c390 2024-09-21T03:29:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.780+0000 7fa3fd1d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.781+0000 7fa3fd1d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44938/0 (socket says 172.21.15.83:44938) 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.781+0000 7fa3fd1d0640 1 -- 172.21.15.83:0/1685624025 learned_addr learned my addr 172.21.15.83:0/1685624025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.781+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a51c5a280 con 0x561a5194d070 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.781+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561a51c5e090 con 0x561a519241b0 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.781+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51cf4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.782+0000 7fa3eeffd640 1 -- 172.21.15.83:0/3979962914 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa3f40305d0 con 0x7fa3f810a590 2024-09-21T03:29:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.782+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 0x561a51cf4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.783+0000 7fa3eeffd640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 0x7fa3e40b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.784+0000 7fa3eeffd640 1 -- 172.21.15.83:0/3979962914 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8485+0+0 (secure 0 0 0) 0x7fa3f4003d20 con 0x7fa3f810a590 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/1685624025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 msgr2=0x561a51cf4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 0x561a51cf4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/1685624025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 msgr2=0x561a51cf4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/1685624025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a519b8650 con 0x561a5194d070 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 0x7fa3e40b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.786+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 0x561a51cf4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.787+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51cf4240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009f50 tx=0x7fa3f000e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f635bd8aab95c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.787+0000 7fa3e3fff640 1 -- 172.21.15.83:0/1685624025 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001e440 con 0x561a5194d070 2024-09-21T03:29:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.787+0000 7fa3e3fff640 1 -- 172.21.15.83:0/1685624025 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f00148a0 con 0x561a5194d070 2024-09-21T03:29:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.787+0000 7fa3e3fff640 1 -- 172.21.15.83:0/1685624025 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001e440 con 0x561a5194d070 2024-09-21T03:29:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.788+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 msgr2=0x561a51cf4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.788+0000 7fa4061a9740 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51cf4240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009f50 tx=0x7fa3f000e660 comp rx=0 tx=0).stop 2024-09-21T03:29:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 0x7fa3e40b9010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000efd0 tx=0x7fa3f001e9b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 shutdown_connections 2024-09-21T03:29:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa4061a9740 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5192c390 0x561a51cf4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa4061a9740 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a519241b0 0x561a51cf4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa4061a9740 1 --2- 172.21.15.83:0/1685624025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51cf4240 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.791+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 >> 172.21.15.83:0/1685624025 conn(0x561a51941460 msgr2=0x561a51cf2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.793+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 shutdown_connections 2024-09-21T03:29:12.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.794+0000 7fa4061a9740 1 -- 172.21.15.83:0/1685624025 wait complete. 2024-09-21T03:29:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.794+0000 7fa4061a9740 1 -- start start 2024-09-21T03:29:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.795+0000 7fa4061a9740 1 Processor -- start 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51d002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a51abb700 con 0x561a519241b0 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa3fd1d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa3fd1d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46032/0 (socket says 172.21.15.83:46032) 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa3fd1d0640 1 -- 172.21.15.83:0/2669317405 learned_addr learned my addr 172.21.15.83:0/2669317405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:12.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a51bf5110 con 0x561a5194d070 2024-09-21T03:29:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561a51d02a20 con 0x561a5192c390 2024-09-21T03:29:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.796+0000 7fa4061a9740 10 client.0 _ll_register_callbacks cb 0x561a51c0e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51d002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 -- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 msgr2=0x561a51d002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51d002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 -- 172.21.15.83:0/2669317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 msgr2=0x561a51cff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a51c22c40 con 0x561a5192c390 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51d002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.797+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80119d0 tx=0x7fa3e8011e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2efa7f5dc3c33d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.798+0000 7fa3e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.798+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e801f060 con 0x561a5192c390 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.798+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8485+0+0 (secure 0 0 0) 0x7fa3e800fd50 con 0x561a5192c390 2024-09-21T03:29:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.798+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f00148a0 con 0x561a5192c390 2024-09-21T03:29:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.799+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa3e8012450 con 0x561a5192c390 2024-09-21T03:29:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.799+0000 7fa4061a9740 10 client.26938 fetch_fsmap learned FSMap version 48 2024-09-21T03:29:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.799+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561a51d0e590 con 0x561a5192c390 2024-09-21T03:29:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.799+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e8012aa0 con 0x561a5192c390 2024-09-21T03:29:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.799+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa3e801a7e0 con 0x561a5192c390 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 10 client.26938 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 20 client.26938 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a5190b9f0 0x561a51d11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 4 client.26938 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa3f00148a0 con 0x561a5190b9f0 2024-09-21T03:29:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.800+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a5190b9f0 0x561a51d11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.801+0000 7fa3e17fa640 10 client.26938 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:12.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.801+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a5190b9f0 0x561a51d11f80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df2c81f6498652a server_cookie=a8650894e09b73fe in_seq=0 out_seq=0 2024-09-21T03:29:12.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.802+0000 7fa3e17fa640 1 -- 172.21.15.83:0/2669317405 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa3f00148a0 con 0x561a5190b9f0 2024-09-21T03:29:12.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.802+0000 7fa3e17fa640 10 client.26938 handle_command_reply: tid=0 2024-09-21T03:29:12.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.802+0000 7fa4061a9740 1 client.26938 shutdown 2024-09-21T03:29:12.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 msgr2=0x561a51cffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80119d0 tx=0x7fa3e8011e90 comp rx=0 tx=0).stop 2024-09-21T03:29:12.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 shutdown_connections 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a5190b9f0 0x561a51d11f80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5194d070 0x561a51d002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51cffda0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa3e17fa640 0 client.26938 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 --2- 172.21.15.83:0/2669317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.803+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 >> 172.21.15.83:0/2669317405 conn(0x561a51941460 msgr2=0x561a51cf7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.804+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 shutdown_connections 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.804+0000 7fa4061a9740 1 -- 172.21.15.83:0/2669317405 wait complete. 2024-09-21T03:29:12.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.804+0000 7fa4061a9740 20 client.26938 trim_cache size 0 max 16384 2024-09-21T03:29:12.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.830+0000 7fa4061a9740 1 Processor -- start 2024-09-21T03:29:12.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.830+0000 7fa4061a9740 1 -- start start 2024-09-21T03:29:12.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cf5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5192c390 0x561a51cf16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cf5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 0x561a51cf1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c6d60 con 0x561a519241b0 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c8900 con 0x561a5192c390 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa4061a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c7430 con 0x561a5194d070 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cf5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44958/0 (socket says 172.21.15.83:44958) 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.831+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/670450986 learned_addr learned my addr 172.21.15.83:0/670450986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 0x561a51cf1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5192c390 0x561a51cf16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/670450986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5192c390 msgr2=0x561a51cf16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5192c390 0x561a51cf16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/670450986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 msgr2=0x561a51cf5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cf5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/670450986 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a51cf2a20 con 0x561a5194d070 2024-09-21T03:29:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.832+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 0x561a51cf1c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa3f001efd0 tx=0x7fa3f0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed48764cab98c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.833+0000 7fa3e37fe640 1 -- 172.21.15.83:0/670450986 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001e440 con 0x561a5194d070 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.833+0000 7fa3e37fe640 1 -- 172.21.15.83:0/670450986 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f0004040 con 0x561a5194d070 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.833+0000 7fa3e37fe640 1 -- 172.21.15.83:0/670450986 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00038f0 con 0x561a5194d070 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- 172.21.15.83:0/670450986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 msgr2=0x561a51cf1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 0x561a51cf1c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa3f001efd0 tx=0x7fa3f0024000 comp rx=0 tx=0).stop 2024-09-21T03:29:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- 172.21.15.83:0/670450986 shutdown_connections 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5194d070 0x561a51cf1c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a5192c390 0x561a51cf16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 --2- 172.21.15.83:0/670450986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a519241b0 0x561a51cf5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- 172.21.15.83:0/670450986 >> 172.21.15.83:0/670450986 conn(0x561a51941460 msgr2=0x561a51cf5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- 172.21.15.83:0/670450986 shutdown_connections 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- 172.21.15.83:0/670450986 wait complete. 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.834+0000 7fa4061a9740 1 -- start start 2024-09-21T03:29:12.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.835+0000 7fa4061a9740 1 Processor -- start 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51d039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c81d0 con 0x561a5194d070 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c7b00 con 0x561a519241b0 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81c4d60 con 0x561a5192c390 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa4061a9740 10 client.0 _ll_register_callbacks cb 0x7fa3e800a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fe1d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fe1d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44968/0 (socket says 172.21.15.83:44968) 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fd9d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44014/0 (socket says 172.21.15.83:44014) 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fe1d2640 1 -- 172.21.15.83:0/3986905723 learned_addr learned my addr 172.21.15.83:0/3986905723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.836+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51d039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3986905723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 msgr2=0x561a51d039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51d039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3986905723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 msgr2=0x561a51d03fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 -- 172.21.15.83:0/3986905723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a51d04700 con 0x561a519241b0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fe1d2640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3fd9d1640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4031cd0 tx=0x7fa3f40ca810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91906f677cc48b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40f5070 con 0x561a519241b0 2024-09-21T03:29:12.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3f0004040 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8485+0+0 (secure 0 0 0) 0x7fa3f40d7b20 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa3f4106120 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa4061a9740 10 client.26996 fetch_fsmap learned FSMap version 48 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.837+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561a51d7c960 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f0004040 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa3f4030b90 con 0x561a519241b0 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 10 client.26996 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 20 client.26996 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a518e0da0 0x561a51d7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 4 client.26996 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:29:12.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a51c6cda0 con 0x561a518e0da0 2024-09-21T03:29:12.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.838+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a518e0da0 0x561a51d7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.839+0000 7fa3e17fa640 10 client.26996 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.839+0000 7fa3fd1d0640 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a518e0da0 0x561a51d7f180 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6ad64da28fb19a server_cookie=24077ffb40a917cf in_seq=0 out_seq=0 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.839+0000 7fa3e17fa640 1 -- 172.21.15.83:0/3986905723 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x561a51c6cda0 con 0x561a518e0da0 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.839+0000 7fa3e17fa640 10 client.26996 handle_command_reply: tid=0 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.839+0000 7fa4061a9740 1 client.26996 shutdown 2024-09-21T03:29:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 msgr2=0x561a51d084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4031cd0 tx=0x7fa3f40ca810 comp rx=0 tx=0).stop 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 shutdown_connections 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x561a518e0da0 0x561a51d7f180 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a5194d070 0x561a51d03fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a5192c390 0x561a51d039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa3e17fa640 0 client.26996 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 --2- 172.21.15.83:0/3986905723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a519241b0 0x561a51d084f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.840+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 >> 172.21.15.83:0/3986905723 conn(0x561a51941460 msgr2=0x561a51cff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.841+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 shutdown_connections 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.841+0000 7fa4061a9740 1 -- 172.21.15.83:0/3986905723 wait complete. 2024-09-21T03:29:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.841+0000 7fa4061a9740 20 client.26996 trim_cache size 0 max 16384 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:29:12.856 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 msgr2=0x7fa3e40b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 0x7fa3e40b9010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000efd0 tx=0x7fa3f001e9b0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 msgr2=0x7fa3f811be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009bf0 tx=0x7fa3f4031c40 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 shutdown_connections 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa3e40b6b00 0x7fa3e40b9010 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8116640 0x7fa3f81216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f810aee0 0x7fa3f811c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 --2- 172.21.15.83:0/3979962914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f810a590 0x7fa3f811be90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 >> 172.21.15.83:0/3979962914 conn(0x7fa3f8070ec0 msgr2=0x7fa3f8072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 shutdown_connections 2024-09-21T03:29:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:12.842+0000 7fa3ffc35640 1 -- 172.21.15.83:0/3979962914 wait complete. 2024-09-21T03:29:13.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:13 smithi142 ceph-mon[34512]: pgmap v714: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 2.04k op/s 2024-09-21T03:29:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:13 smithi193 ceph-mon[35224]: pgmap v714: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 2.04k op/s 2024-09-21T03:29:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:13 smithi083 ceph-mon[31219]: pgmap v714: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 2.04k op/s 2024-09-21T03:29:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:15 smithi193 ceph-mon[35224]: pgmap v715: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T03:29:15.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:15 smithi193 ceph-mon[35224]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:29:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:15 smithi083 ceph-mon[31219]: pgmap v715: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T03:29:15.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:15 smithi083 ceph-mon[31219]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:29:15.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:15 smithi142 ceph-mon[34512]: pgmap v715: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T03:29:15.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:15 smithi142 ceph-mon[34512]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:29:15.595 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:29:15.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 715.0 s, 1578.6 tps, lat 62.920 ms stddev 77.500 2024-09-21T03:29:15.962 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 -- 172.21.15.83:0/1057654447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe834108d30 msgr2=0x7fe83410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/1057654447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe834108d30 0x7fe83410f560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009be0 tx=0x7fe82c02a6e0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 -- 172.21.15.83:0/1057654447 shutdown_connections 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/1057654447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe834108d30 0x7fe83410f560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/1057654447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8341083e0 0x7fe8341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/1057654447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834102aa0 0x7fe834102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.528+0000 7fe83c1ea640 1 -- 172.21.15.83:0/1057654447 >> 172.21.15.83:0/1057654447 conn(0x7fe834079750 msgr2=0x7fe834079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.530+0000 7fe83c1ea640 1 -- 172.21.15.83:0/1057654447 shutdown_connections 2024-09-21T03:29:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.530+0000 7fe83c1ea640 1 -- 172.21.15.83:0/1057654447 wait complete. 2024-09-21T03:29:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.531+0000 7fe83c1ea640 1 Processor -- start 2024-09-21T03:29:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.532+0000 7fe83c1ea640 1 -- start start 2024-09-21T03:29:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.532+0000 7fe83c1ea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.532+0000 7fe83c1ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe83c1ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 0x7fe83406f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe83c1ea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe83406d6c0 con 0x7fe834108d30 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe83c1ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe83406d540 con 0x7fe8341083e0 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe83c1ea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe83406d840 con 0x7fe834102aa0 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839f86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839785640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839785640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:44024/0 (socket says 172.21.15.83:44024) 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839785640 1 -- 172.21.15.83:0/163667024 learned_addr learned my addr 172.21.15.83:0/163667024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839f86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:46090/0 (socket says 172.21.15.83:46090) 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe83a787640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 0x7fe83406f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839f86640 1 -- 172.21.15.83:0/163667024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 msgr2=0x7fe83406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839f86640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.533+0000 7fe839f86640 1 -- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 msgr2=0x7fe83406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:16.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.534+0000 7fe839f86640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 0x7fe83406f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.534+0000 7fe839f86640 1 -- 172.21.15.83:0/163667024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83406fb40 con 0x7fe834102aa0 2024-09-21T03:29:16.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.534+0000 7fe839785640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:16.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.534+0000 7fe83a787640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 0x7fe83406f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.535+0000 7fe839f86640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe83000b3f0 tx=0x7fe83000b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.535+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830015020 con 0x7fe834102aa0 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.535+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83406b820 con 0x7fe834102aa0 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.536+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83406be30 con 0x7fe834102aa0 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.536+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe830011ce0 con 0x7fe834102aa0 2024-09-21T03:29:16.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.536+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830010d30 con 0x7fe834102aa0 2024-09-21T03:29:16.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.537+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818005290 con 0x7fe834102aa0 2024-09-21T03:29:16.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.540+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe830018020 con 0x7fe834102aa0 2024-09-21T03:29:16.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.541+0000 7fe82affd640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 0x7fe8200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:16.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.541+0000 7fe839785640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 0x7fe8200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:16.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.541+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fe8300e70c0 con 0x7fe834102aa0 2024-09-21T03:29:16.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.542+0000 7fe839785640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 0x7fe8200b8c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe834105ea0 tx=0x7fe824009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.550+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe83001f020 con 0x7fe834102aa0 2024-09-21T03:29:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.850+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe818005580 con 0x7fe834102aa0 2024-09-21T03:29:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.851+0000 7fe82affd640 1 -- 172.21.15.83:0/163667024 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11992 (secure 0 0 0) 0x7fe8300ad660 con 0x7fe834102aa0 2024-09-21T03:29:16.854 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:29:16.854 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"btime":"2024-09-21T03:28:49:090221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:28:48.902462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:29:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 48 2024-09-21T03:29:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 msgr2=0x7fe8200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 0x7fe8200b8c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe834105ea0 tx=0x7fe824009260 comp rx=0 tx=0).stop 2024-09-21T03:29:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 msgr2=0x7fe834074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe83000b3f0 tx=0x7fe83000b7f0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 shutdown_connections 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8200b6700 0x7fe8200b8c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe834108d30 0x7fe83406f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8341083e0 0x7fe83406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 --2- 172.21.15.83:0/163667024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe834102aa0 0x7fe834074ca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.855+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 >> 172.21.15.83:0/163667024 conn(0x7fe834079750 msgr2=0x7fe8341061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.856+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 shutdown_connections 2024-09-21T03:29:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:16.856+0000 7fe83c1ea640 1 -- 172.21.15.83:0/163667024 wait complete. 2024-09-21T03:29:17.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:17 smithi083 ceph-mon[31219]: pgmap v717: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-21T03:29:17.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:17 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/163667024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:17 smithi193 ceph-mon[35224]: pgmap v717: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-21T03:29:17.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:17 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/163667024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:17 smithi142 ceph-mon[34512]: pgmap v717: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-21T03:29:17.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:17 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/163667024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:18.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:29:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:19.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.293+0000 7f95acb7c640 1 -- 172.21.15.83:0/2474813380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8111f40 msgr2=0x7f95a810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.293+0000 7f95acb7c640 1 --2- 172.21.15.83:0/2474813380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8111f40 0x7f95a810a3c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9598009900 tx=0x7f959802a440 comp rx=0 tx=0).stop 2024-09-21T03:29:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.301+0000 7f95acb7c640 1 -- 172.21.15.83:0/2474813380 shutdown_connections 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.301+0000 7f95acb7c640 1 --2- 172.21.15.83:0/2474813380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.301+0000 7f95acb7c640 1 --2- 172.21.15.83:0/2474813380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a810a9e0 0x7f95a810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.301+0000 7f95acb7c640 1 --2- 172.21.15.83:0/2474813380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8111f40 0x7f95a810a3c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.301+0000 7f95acb7c640 1 -- 172.21.15.83:0/2474813380 >> 172.21.15.83:0/2474813380 conn(0x7f95a8070ec0 msgr2=0x7f95a80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.303+0000 7f95acb7c640 1 -- 172.21.15.83:0/2474813380 shutdown_connections 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.306+0000 7f95acb7c640 1 -- 172.21.15.83:0/2474813380 wait complete. 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.307+0000 7f95acb7c640 1 Processor -- start 2024-09-21T03:29:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.307+0000 7f95acb7c640 1 -- start start 2024-09-21T03:29:19.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:19 smithi083 ceph-mon[31219]: pgmap v718: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 716 op/s 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a810a9e0 0x7f95a80758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a8111f40 0x7f95a8075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8122a00 con 0x7f95a810a9e0 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8122880 con 0x7f95a8111f40 2024-09-21T03:29:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.308+0000 7f95acb7c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8122b80 con 0x7f95a811cc30 2024-09-21T03:29:19.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:19 smithi193 ceph-mon[35224]: pgmap v718: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 716 op/s 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.309+0000 7f95a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.309+0000 7f95a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54450/0 (socket says 172.21.15.83:54450) 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.309+0000 7f95a6d9d640 1 -- 172.21.15.83:0/1742524030 learned_addr learned my addr 172.21.15.83:0/1742524030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.309+0000 7f95a659c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a810a9e0 0x7f95a80758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 -- 172.21.15.83:0/1742524030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a8111f40 msgr2=0x7f95a8075e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a8111f40 0x7f95a8075e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 -- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a810a9e0 msgr2=0x7f95a80758e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a810a9e0 0x7f95a80758e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 -- 172.21.15.83:0/1742524030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a807b850 con 0x7f95a811cc30 2024-09-21T03:29:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.310+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f95a000b990 tx=0x7f95a000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.314+0000 7f95977fe640 1 -- 172.21.15.83:0/1742524030 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0019070 con 0x7f95a811cc30 2024-09-21T03:29:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.314+0000 7f95977fe640 1 -- 172.21.15.83:0/1742524030 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a000cba0 con 0x7f95a811cc30 2024-09-21T03:29:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.314+0000 7f95977fe640 1 -- 172.21.15.83:0/1742524030 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a00075d0 con 0x7f95a811cc30 2024-09-21T03:29:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.314+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a807bae0 con 0x7f95a811cc30 2024-09-21T03:29:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.314+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a80822c0 con 0x7f95a811cc30 2024-09-21T03:29:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.318+0000 7f95b30f0740 1 Processor -- start 2024-09-21T03:29:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.318+0000 7f95b30f0740 1 -- start start 2024-09-21T03:29:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18612be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18613120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de18588ea0 con 0x55de182421b0 2024-09-21T03:29:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55de185782d0 con 0x55de1824a390 2024-09-21T03:29:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.319+0000 7f95b30f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55de1857c0e0 con 0x55de1826b070 2024-09-21T03:29:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.321+0000 7f95a5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.321+0000 7f95a5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54466/0 (socket says 172.21.15.83:54466) 2024-09-21T03:29:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.321+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3148401119 learned_addr learned my addr 172.21.15.83:0/3148401119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:19.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.324+0000 7f95977fe640 1 -- 172.21.15.83:0/1742524030 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f95a0007830 con 0x7f95a811cc30 2024-09-21T03:29:19.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.324+0000 7f95a659c640 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18612be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.325+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3148401119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 msgr2=0x55de18613120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.325+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18613120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.325+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3148401119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 msgr2=0x55de18612be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.329+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18612be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.329+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3148401119 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de182d6650 con 0x55de1826b070 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.329+0000 7f95a659c640 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18612be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.329+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f959c00b810 tx=0x7f959c00bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d28976ef339c1688 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.330+0000 7f95977fe640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 0x7f95900b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.330+0000 7f95977fe640 1 -- 172.21.15.83:0/1742524030 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f95a00e6b30 con 0x7f95a811cc30 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.330+0000 7f9594ff9640 1 -- 172.21.15.83:0/3148401119 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c018070 con 0x55de1826b070 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.331+0000 7f9594ff9640 1 -- 172.21.15.83:0/3148401119 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c013430 con 0x55de1826b070 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.331+0000 7f9594ff9640 1 -- 172.21.15.83:0/3148401119 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00f070 con 0x55de1826b070 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.332+0000 7f95b30f0740 1 -- 172.21.15.83:0/3148401119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 msgr2=0x55de186126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.332+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f959c00b810 tx=0x7f959c00bcd0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.336+0000 7f95b30f0740 1 -- 172.21.15.83:0/3148401119 shutdown_connections 2024-09-21T03:29:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.336+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18613120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.336+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18612be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.336+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3148401119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de186126a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.336+0000 7f95b30f0740 1 -- 172.21.15.83:0/3148401119 >> 172.21.15.83:0/3148401119 conn(0x55de1825f460 msgr2=0x55de18610590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.339+0000 7f95a659c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 0x7f95900b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.339+0000 7f95b30f0740 1 -- 172.21.15.83:0/3148401119 shutdown_connections 2024-09-21T03:29:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.339+0000 7f95b30f0740 1 -- 172.21.15.83:0/3148401119 wait complete. 2024-09-21T03:29:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.340+0000 7f95b30f0740 1 -- start start 2024-09-21T03:29:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.340+0000 7f95b30f0740 1 Processor -- start 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de1862ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1863b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de183d9700 con 0x55de182421b0 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55de18513160 con 0x55de1824a390 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55de18631de0 con 0x55de1826b070 2024-09-21T03:29:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95b30f0740 10 client.0 _ll_register_callbacks cb 0x55de1852c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:19.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95a5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95a5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35620/0 (socket says 172.21.15.83:35620) 2024-09-21T03:29:19.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.341+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3456459982 learned_addr learned my addr 172.21.15.83:0/3456459982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.344+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 msgr2=0x55de1863b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.344+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1863b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.344+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3456459982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 msgr2=0x55de1862ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.344+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de1862ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.344+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3456459982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de18540c90 con 0x55de1824a390 2024-09-21T03:29:19.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.345+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f959c006d10 tx=0x7f959c00b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2820eeb273e5bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.345+0000 7f958dffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.349+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00f050 con 0x55de1824a390 2024-09-21T03:29:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.349+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f959c00da90 con 0x55de1824a390 2024-09-21T03:29:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.350+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c013430 con 0x55de1824a390 2024-09-21T03:29:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.351+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f959c018040 con 0x55de1824a390 2024-09-21T03:29:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.353+0000 7f95b30f0740 10 client.27005 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.353+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55de1863dc00 con 0x55de1824a390 2024-09-21T03:29:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.354+0000 7f95a659c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 0x7f95900b8d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55de18611320 tx=0x7f9598000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c021070 con 0x55de1824a390 2024-09-21T03:29:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f959c01b070 con 0x55de1824a390 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 10 client.27005 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 10 client.27005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 20 client.27005 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de182299f0 0x55de18640f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 4 client.27005 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.358+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f959c013430 con 0x55de182299f0 2024-09-21T03:29:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.362+0000 7f95a659c640 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de182299f0 0x55de18640f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.372+0000 7f95a659c640 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de182299f0 0x55de18640f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23efe359a4af2c88 server_cookie=d3ad2029c78433c2 in_seq=0 out_seq=0 2024-09-21T03:29:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.372+0000 7f958dffb640 10 client.27005 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:29:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.373+0000 7f958dffb640 1 -- 172.21.15.83:0/3456459982 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f959c013430 con 0x55de182299f0 2024-09-21T03:29:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.373+0000 7f958dffb640 10 client.27005 handle_command_reply: tid=0 2024-09-21T03:29:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.374+0000 7f95b30f0740 1 client.27005 shutdown 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 msgr2=0x55de1862f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f959c006d10 tx=0x7f959c00b5d0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 shutdown_connections 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de182299f0 0x55de18640f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1863b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862f3c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3456459982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de1862ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 >> 172.21.15.83:0/3456459982 conn(0x55de1825f460 msgr2=0x55de18614090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.375+0000 7f958dffb640 0 client.27005 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:29:19.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.379+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 shutdown_connections 2024-09-21T03:29:19.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.379+0000 7f95b30f0740 1 -- 172.21.15.83:0/3456459982 wait complete. 2024-09-21T03:29:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.379+0000 7f95b30f0740 20 client.27005 trim_cache size 0 max 16384 2024-09-21T03:29:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.405+0000 7f95b30f0740 1 Processor -- start 2024-09-21T03:29:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.406+0000 7f95b30f0740 1 -- start start 2024-09-21T03:29:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.406+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18613de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1862c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18613de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95a659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18613de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37846/0 (socket says 172.21.15.83:37846) 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95a659c640 1 -- 172.21.15.83:0/3955961459 learned_addr learned my addr 172.21.15.83:0/3955961459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95a81166c0 con 0x55de182421b0 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8118230 con 0x55de1824a390 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.407+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8116d90 con 0x55de1826b070 2024-09-21T03:29:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.408+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1862c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.409+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3955961459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 msgr2=0x55de1862c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.409+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1862c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.409+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3955961459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 msgr2=0x55de18613de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.409+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18613de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.409+0000 7f95a5d9b640 1 -- 172.21.15.83:0/3955961459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de18630140 con 0x55de1824a390 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.410+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862c070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c00b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9310e8bda9fea1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.410+0000 7f958ffff640 1 -- 172.21.15.83:0/3955961459 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00f070 con 0x55de1824a390 2024-09-21T03:29:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.411+0000 7f958ffff640 1 -- 172.21.15.83:0/3955961459 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c021040 con 0x55de1824a390 2024-09-21T03:29:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.411+0000 7f958ffff640 1 -- 172.21.15.83:0/3955961459 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c0043a0 con 0x55de1824a390 2024-09-21T03:29:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 msgr2=0x55de1862c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862c070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c00b6e0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 shutdown_connections 2024-09-21T03:29:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de1826b070 0x55de1862c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de1862c070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.412+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3955961459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de182421b0 0x55de18613de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.413+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 >> 172.21.15.83:0/3955961459 conn(0x55de1825f460 msgr2=0x55de186136d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.418+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 shutdown_connections 2024-09-21T03:29:19.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.419+0000 7f95b30f0740 1 -- 172.21.15.83:0/3955961459 wait complete. 2024-09-21T03:29:19.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.419+0000 7f95b30f0740 1 -- start start 2024-09-21T03:29:19.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.420+0000 7f95b30f0740 1 Processor -- start 2024-09-21T03:29:19.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.420+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.421+0000 7f95b30f0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18635040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.421+0000 7f95a659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.421+0000 7f95a659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54496/0 (socket says 172.21.15.83:54496) 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.421+0000 7f95a659c640 1 -- 172.21.15.83:0/3857755606 learned_addr learned my addr 172.21.15.83:0/3857755606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.422+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18635040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.422+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 0x55de186a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.422+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8117b30 con 0x55de1826b070 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.422+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8117460 con 0x55de1824a390 2024-09-21T03:29:19.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.422+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95a81146e0 con 0x55de182421b0 2024-09-21T03:29:19.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.423+0000 7f95b30f0740 10 client.0 _ll_register_callbacks cb 0x7f959c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:19.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.423+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 0x55de186a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 -- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 msgr2=0x55de18634a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 -- 172.21.15.83:0/3857755606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 msgr2=0x55de18635040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:19.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18635040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de186a3ee0 con 0x55de1826b070 2024-09-21T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:19 smithi142 ceph-mon[34512]: pgmap v718: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 716 op/s 2024-09-21T03:29:19.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f95a6d9d640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 0x55de186a3870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a00e4270 tx=0x7f95a00ac3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b769721df9fdcb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:19.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.424+0000 7f958dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:19.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.425+0000 7f95a659c640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:19.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.425+0000 7f95a5d9b640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18635040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:19.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.426+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a00e46f0 con 0x55de1826b070 2024-09-21T03:29:19.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.426+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f95a00abbd0 con 0x55de1826b070 2024-09-21T03:29:19.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.426+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c021040 con 0x55de1826b070 2024-09-21T03:29:19.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.427+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f95a00b1450 con 0x55de1826b070 2024-09-21T03:29:19.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.427+0000 7f95b30f0740 10 client.17457 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:19.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.427+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55de186ab2b0 con 0x55de1826b070 2024-09-21T03:29:19.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.428+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a00b6390 con 0x55de1826b070 2024-09-21T03:29:19.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.428+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f95a00d1060 con 0x55de1826b070 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 10 client.17457 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 10 client.17457 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 20 client.17457 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de181feda0 0x55de186ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 4 client.17457 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f959c021040 con 0x55de181feda0 2024-09-21T03:29:19.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.429+0000 7f95a659c640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de181feda0 0x55de186ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:19.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.434+0000 7f958dffb640 10 client.17457 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:29:19.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.434+0000 7f95a659c640 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de181feda0 0x55de186ada70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3f181b199311e2b server_cookie=c0ea7350bccadd24 in_seq=0 out_seq=0 2024-09-21T03:29:19.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.436+0000 7f958dffb640 1 -- 172.21.15.83:0/3857755606 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f959c021040 con 0x55de181feda0 2024-09-21T03:29:19.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.436+0000 7f958dffb640 10 client.17457 handle_command_reply: tid=0 2024-09-21T03:29:19.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.437+0000 7f95b30f0740 1 client.17457 shutdown 2024-09-21T03:29:19.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.437+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 msgr2=0x55de186a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.437+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 0x55de186a3870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a00e4270 tx=0x7f95a00ac3d0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.439+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 shutdown_connections 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.439+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x55de181feda0 0x55de186ada70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.439+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de1826b070 0x55de186a3870 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.439+0000 7f958dffb640 0 client.17457 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.439+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de1824a390 0x55de18635040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.440+0000 7f95b30f0740 1 --2- 172.21.15.83:0/3857755606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de182421b0 0x55de18634a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.440+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 >> 172.21.15.83:0/3857755606 conn(0x55de1825f460 msgr2=0x55de18630310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.441+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 shutdown_connections 2024-09-21T03:29:19.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.441+0000 7f95b30f0740 1 -- 172.21.15.83:0/3857755606 wait complete. 2024-09-21T03:29:19.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.442+0000 7f95b30f0740 20 client.17457 trim_cache size 0 max 16384 2024-09-21T03:29:19.446 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:29:19.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.443+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 msgr2=0x7f95900b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.443+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 0x7f95900b8d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55de18611320 tx=0x7f9598000bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.443+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 msgr2=0x7f95a807b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.443+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f95a000b990 tx=0x7f95a000be50 comp rx=0 tx=0).stop 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 shutdown_connections 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95900b6880 0x7f95900b8d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a811cc30 0x7f95a807b120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a8111f40 0x7f95a8075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 --2- 172.21.15.83:0/1742524030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a810a9e0 0x7f95a80758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 >> 172.21.15.83:0/1742524030 conn(0x7f95a8070ec0 msgr2=0x7f95a811e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 shutdown_connections 2024-09-21T03:29:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:19.444+0000 7f95acb7c640 1 -- 172.21.15.83:0/1742524030 wait complete. 2024-09-21T03:29:20.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:20 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:20.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:20 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:20.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:20 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:20.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:29:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 720.0 s, 1522.8 tps, lat 66.036 ms stddev 105.305 2024-09-21T03:29:20.980 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:21.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:21 smithi083 ceph-mon[31219]: pgmap v719: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.6 MiB/s wr, 710 op/s 2024-09-21T03:29:21.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:21 smithi193 ceph-mon[35224]: pgmap v719: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.6 MiB/s wr, 710 op/s 2024-09-21T03:29:21.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:21 smithi142 ceph-mon[34512]: pgmap v719: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.6 MiB/s wr, 710 op/s 2024-09-21T03:29:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 -- 172.21.15.83:0/176803193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 msgr2=0x7faddc10b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 --2- 172.21.15.83:0/176803193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 0x7faddc10b2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 -- 172.21.15.83:0/176803193 shutdown_connections 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 --2- 172.21.15.83:0/176803193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10ead0 0x7faddc1174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 --2- 172.21.15.83:0/176803193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10e180 0x7faddc10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 --2- 172.21.15.83:0/176803193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 0x7faddc10b2f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 -- 172.21.15.83:0/176803193 >> 172.21.15.83:0/176803193 conn(0x7faddc070ec0 msgr2=0x7faddc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 -- 172.21.15.83:0/176803193 shutdown_connections 2024-09-21T03:29:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.594+0000 7fade3017640 1 -- 172.21.15.83:0/176803193 wait complete. 2024-09-21T03:29:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.595+0000 7fade3017640 1 Processor -- start 2024-09-21T03:29:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.595+0000 7fade3017640 1 -- start start 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 0x7faddc071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 0x7faddc072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faddc11a920 con 0x7faddc10aee0 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faddc11a7a0 con 0x7faddc10ead0 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7fade3017640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faddc11aaa0 con 0x7faddc10e180 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.596+0000 7faddbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 0x7faddc072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7faddbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 0x7faddc072480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54510/0 (socket says 172.21.15.83:54510) 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35650/0 (socket says 172.21.15.83:35650) 2024-09-21T03:29:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7faddbfff640 1 -- 172.21.15.83:0/119877739 learned_addr learned my addr 172.21.15.83:0/119877739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 -- 172.21.15.83:0/119877739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 msgr2=0x7faddc072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 0x7faddc072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 -- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 msgr2=0x7faddc071f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 0x7faddc071f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.597+0000 7fade15b4640 1 -- 172.21.15.83:0/119877739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1b0f40 con 0x7faddc10ead0 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fade15b4640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009fd0 tx=0x7fadd400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fadd9ffb640 1 -- 172.21.15.83:0/119877739 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd400ce20 con 0x7faddc10ead0 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1b1230 con 0x7faddc10ead0 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fadd9ffb640 1 -- 172.21.15.83:0/119877739 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4005020 con 0x7faddc10ead0 2024-09-21T03:29:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fadd9ffb640 1 -- 172.21.15.83:0/119877739 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4010690 con 0x7faddc10ead0 2024-09-21T03:29:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.599+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1b17c0 con 0x7faddc10ead0 2024-09-21T03:29:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.602+0000 7fadd9ffb640 1 -- 172.21.15.83:0/119877739 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fadd4004b20 con 0x7faddc10ead0 2024-09-21T03:29:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.603+0000 7fade958b740 1 Processor -- start 2024-09-21T03:29:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.603+0000 7fadd9ffb640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 0x7fadcc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 -- start start 2024-09-21T03:29:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7faddbfff640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 0x7fadcc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fadd9ffb640 1 -- 172.21.15.83:0/119877739 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fadd40e7ed0 con 0x7faddc10ead0 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 0x55fe53678be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 0x55fe53679120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7faddbfff640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 0x7fadcc0b8c10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faddc074b60 tx=0x7fadc8005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe535eeea0 con 0x55fe532d1070 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fe535de2d0 con 0x55fe532a81b0 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.604+0000 7fade958b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe535e20e0 con 0x55fe532b0390 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade0db3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade0db3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37870/0 (socket says 172.21.15.83:37870) 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade0db3640 1 -- 172.21.15.83:0/3563298700 learned_addr learned my addr 172.21.15.83:0/3563298700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7faddbfff640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 0x55fe53678be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 0x55fe53679120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 -- 172.21.15.83:0/3563298700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 msgr2=0x55fe53678be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 0x55fe53678be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 -- 172.21.15.83:0/3563298700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 msgr2=0x55fe536786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 -- 172.21.15.83:0/3563298700 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5333c650 con 0x55fe532b0390 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade0db3640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7faddbfff640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 0x55fe53678be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fade15b4640 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 0x55fe53679120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fadd40e5610 tx=0x7fadd4004e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83ed9308b51f769 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.605+0000 7fadc6ffd640 1 -- 172.21.15.83:0/3563298700 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b27f0 con 0x55fe532b0390 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.606+0000 7fadc6ffd640 1 -- 172.21.15.83:0/3563298700 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40b7730 con 0x55fe532b0390 2024-09-21T03:29:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.606+0000 7fadc6ffd640 1 -- 172.21.15.83:0/3563298700 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd400b2c0 con 0x55fe532b0390 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.606+0000 7fade958b740 1 -- 172.21.15.83:0/3563298700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 msgr2=0x55fe53679120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.606+0000 7fade958b740 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 0x55fe53679120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fadd40e5610 tx=0x7fadd4004e10 comp rx=0 tx=0).stop 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 -- 172.21.15.83:0/3563298700 shutdown_connections 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532b0390 0x55fe53679120 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532a81b0 0x55fe53678be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 --2- 172.21.15.83:0/3563298700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe536786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 -- 172.21.15.83:0/3563298700 >> 172.21.15.83:0/3563298700 conn(0x55fe532c5460 msgr2=0x55fe53676590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 -- 172.21.15.83:0/3563298700 shutdown_connections 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 -- 172.21.15.83:0/3563298700 wait complete. 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 -- start start 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.607+0000 7fade958b740 1 Processor -- start 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53683d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe53690170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe5343f700 con 0x55fe532d1070 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fe53579160 con 0x55fe532b0390 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe53686e80 con 0x55fe532a81b0 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade958b740 10 client.0 _ll_register_callbacks cb 0x55fe535929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35676/0 (socket says 172.21.15.83:35676) 2024-09-21T03:29:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade0db3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53683d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 -- 172.21.15.83:0/172157548 learned_addr learned my addr 172.21.15.83:0/172157548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade15b4640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe53690170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 -- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 msgr2=0x55fe53683d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53683d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 -- 172.21.15.83:0/172157548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 msgr2=0x55fe53690170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe53690170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7faddbfff640 1 -- 172.21.15.83:0/172157548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe535a6c90 con 0x55fe532b0390 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade15b4640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe53690170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.608+0000 7fade0db3640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53683d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.611+0000 7faddbfff640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadc8012c40 tx=0x7fadc8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc8fb01b48b5741 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.611+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.611+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8029640 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fadc80283e0 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd40b7730 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fadc80297e0 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 10 client.27020 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fe53692bb0 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40b7730 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fadc8027070 con 0x55fe532b0390 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 10 client.27020 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 10 client.27020 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 20 client.27020 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe5328f9f0 0x55fe536961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 4 client.27020 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.612+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe535f0df0 con 0x55fe5328f9f0 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.613+0000 7fade0db3640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe5328f9f0 0x55fe536961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.613+0000 7fade0db3640 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe5328f9f0 0x55fe536961a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=620b83703b9acd6 server_cookie=d09e9edd792b7d3 in_seq=0 out_seq=0 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.613+0000 7fadbbfff640 10 client.27020 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fadbbfff640 1 -- 172.21.15.83:0/172157548 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe535f0df0 con 0x55fe5328f9f0 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fadbbfff640 10 client.27020 handle_command_reply: tid=0 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 client.27020 shutdown 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 msgr2=0x55fe53684280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadc8012c40 tx=0x7fadc8004730 comp rx=0 tx=0).stop 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 shutdown_connections 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532d1070 0x55fe53690170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53684280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe5328f9f0 0x55fe536961a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 --2- 172.21.15.83:0/172157548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53683d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 >> 172.21.15.83:0/172157548 conn(0x55fe532c5460 msgr2=0x55fe5367b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fadbbfff640 0 client.27020 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:29:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 shutdown_connections 2024-09-21T03:29:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.614+0000 7fade958b740 1 -- 172.21.15.83:0/172157548 wait complete. 2024-09-21T03:29:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.615+0000 7fade958b740 20 client.27020 trim_cache size 0 max 16384 2024-09-21T03:29:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.631+0000 7fade958b740 1 Processor -- start 2024-09-21T03:29:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.631+0000 7fade958b740 1 -- start start 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.631+0000 7fade958b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade958b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53690070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade958b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532d1070 0x55fe53680f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade958b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ba180 con 0x55fe532a81b0 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade958b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1bbcf0 con 0x55fe532b0390 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade958b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ba850 con 0x55fe532d1070 2024-09-21T03:29:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37902/0 (socket says 172.21.15.83:37902) 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 -- 172.21.15.83:0/2032319233 learned_addr learned my addr 172.21.15.83:0/2032319233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7faddbfff640 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53690070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade15b4640 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532d1070 0x55fe53680f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 -- 172.21.15.83:0/2032319233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532d1070 msgr2=0x55fe53680f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532d1070 0x55fe53680f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 -- 172.21.15.83:0/2032319233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 msgr2=0x55fe53690070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53690070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 -- 172.21.15.83:0/2032319233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe53682240 con 0x55fe532a81b0 2024-09-21T03:29:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.632+0000 7fade0db3640 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13aaa9c4045e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.633+0000 7fadc67fc640 1 -- 172.21.15.83:0/2032319233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0049020 con 0x55fe532a81b0 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.633+0000 7fadc67fc640 1 -- 172.21.15.83:0/2032319233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0031950 con 0x55fe532a81b0 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.633+0000 7fadc67fc640 1 -- 172.21.15.83:0/2032319233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0035040 con 0x55fe532a81b0 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- 172.21.15.83:0/2032319233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 msgr2=0x55fe53684200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0009580 comp rx=0 tx=0).stop 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- 172.21.15.83:0/2032319233 shutdown_connections 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532d1070 0x55fe53680f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532b0390 0x55fe53690070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 --2- 172.21.15.83:0/2032319233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532a81b0 0x55fe53684200 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- 172.21.15.83:0/2032319233 >> 172.21.15.83:0/2032319233 conn(0x55fe532c5460 msgr2=0x55fe53683af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- 172.21.15.83:0/2032319233 shutdown_connections 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- 172.21.15.83:0/2032319233 wait complete. 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 -- start start 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.634+0000 7fade958b740 1 Processor -- start 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532d1070 0x55fe536ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1bb5f0 con 0x55fe532b0390 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1baf20 con 0x55fe532d1070 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1b8180 con 0x55fe532a81b0 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7faddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7faddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37914/0 (socket says 172.21.15.83:37914) 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7faddbfff640 1 -- 172.21.15.83:0/1213093220 learned_addr learned my addr 172.21.15.83:0/1213093220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54550/0 (socket says 172.21.15.83:54550) 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade958b740 10 client.0 _ll_register_callbacks cb 0x7fadc8011c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade15b4640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532d1070 0x55fe536ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 -- 172.21.15.83:0/1213093220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532d1070 msgr2=0x55fe536ffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532d1070 0x55fe536ffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 -- 172.21.15.83:0/1213093220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 msgr2=0x55fe53688df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7fade0db3640 1 -- 172.21.15.83:0/1213093220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe53700150 con 0x55fe532a81b0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.635+0000 7faddbfff640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fade0db3640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd0031440 tx=0x7fadd0004130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11afdffa1c76d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fadbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0035040 con 0x55fe532a81b0 2024-09-21T03:29:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe535ec8a0 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fadd00316a0 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fadd00035f0 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fade958b740 10 client.26977 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.636+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fe537013b0 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0053070 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fadd0058780 con 0x55fe532a81b0 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 10 client.26977 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 10 client.26977 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 20 client.26977 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe53264da0 0x55fe53703bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 4 client.26977 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe535ec8a0 con 0x55fe53264da0 2024-09-21T03:29:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.637+0000 7faddbfff640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe53264da0 0x55fe53703bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.639+0000 7faddbfff640 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe53264da0 0x55fe53703bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8152aa0e2eeca23e server_cookie=5cdb1d5df75efd04 in_seq=0 out_seq=0 2024-09-21T03:29:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.639+0000 7fadbbfff640 10 client.26977 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:29:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.639+0000 7fadbbfff640 1 -- 172.21.15.83:0/1213093220 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe535ec8a0 con 0x55fe53264da0 2024-09-21T03:29:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.640+0000 7fadbbfff640 10 client.26977 handle_command_reply: tid=0 2024-09-21T03:29:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.640+0000 7fade958b740 1 client.26977 shutdown 2024-09-21T03:29:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.640+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 msgr2=0x55fe53688890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.640+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd0031440 tx=0x7fadd0004130 comp rx=0 tx=0).stop 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 shutdown_connections 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe532d1070 0x55fe536ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe532b0390 0x55fe53688df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55fe53264da0 0x55fe53703bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 --2- 172.21.15.83:0/1213093220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe532a81b0 0x55fe53688890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.642+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 >> 172.21.15.83:0/1213093220 conn(0x55fe532c5460 msgr2=0x55fe536796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.645+0000 7fadbbfff640 0 client.26977 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:29:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.645+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 shutdown_connections 2024-09-21T03:29:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.646+0000 7fade958b740 1 -- 172.21.15.83:0/1213093220 wait complete. 2024-09-21T03:29:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.650+0000 7fade958b740 20 client.26977 trim_cache size 0 max 16384 2024-09-21T03:29:21.656 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:29:21.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.655+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 msgr2=0x7fadcc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.655+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 0x7fadcc0b8c10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faddc074b60 tx=0x7fadc8005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:21.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.655+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 msgr2=0x7faddc0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:21.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.655+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009fd0 tx=0x7fadd400ef10 comp rx=0 tx=0).stop 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.659+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 shutdown_connections 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.659+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fadcc0b6700 0x7fadcc0b8c10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.659+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faddc10ead0 0x7faddc0729c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.659+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc10e180 0x7faddc072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.659+0000 7fade3017640 1 --2- 172.21.15.83:0/119877739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faddc10aee0 0x7faddc071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.660+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 >> 172.21.15.83:0/119877739 conn(0x7faddc070ec0 msgr2=0x7faddc10c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.662+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 shutdown_connections 2024-09-21T03:29:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:21.663+0000 7fade3017640 1 -- 172.21.15.83:0/119877739 wait complete. 2024-09-21T03:29:22.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:29:22.820 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:23.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:23 smithi193 ceph-mon[35224]: pgmap v720: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-21T03:29:23.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:23 smithi083 ceph-mon[31219]: pgmap v720: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-21T03:29:23.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:23 smithi142 ceph-mon[34512]: pgmap v720: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-09-21T03:29:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.392+0000 7ff8c0c24640 1 -- 172.21.15.83:0/2590147230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 msgr2=0x7ff8bc102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.392+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/2590147230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc102a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009900 tx=0x7ff8ac02a480 comp rx=0 tx=0).stop 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 -- 172.21.15.83:0/2590147230 shutdown_connections 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/2590147230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 0x7ff8bc112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/2590147230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 0x7ff8bc10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/2590147230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc102a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 -- 172.21.15.83:0/2590147230 >> 172.21.15.83:0/2590147230 conn(0x7ff8bc079730 msgr2=0x7ff8bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 -- 172.21.15.83:0/2590147230 shutdown_connections 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.393+0000 7ff8c0c24640 1 -- 172.21.15.83:0/2590147230 wait complete. 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.394+0000 7ff8c0c24640 1 Processor -- start 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.394+0000 7ff8c0c24640 1 -- start start 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 0x7ff8bc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 0x7ff8bc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc115c70 con 0x7ff8bc110200 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc115af0 con 0x7ff8bc102650 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8c0c24640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc115df0 con 0x7ff8bc102fa0 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35698/0 (socket says 172.21.15.83:35698) 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1504676534 learned_addr learned my addr 172.21.15.83:0/1504676534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 0x7ff8bc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 0x7ff8bc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1504676534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 msgr2=0x7ff8bc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 0x7ff8bc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 msgr2=0x7ff8bc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 0x7ff8bc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1504676534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc1a2650 con 0x7ff8bc102650 2024-09-21T03:29:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005940 tx=0x7ff8ac004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.395+0000 7ff8b37fe640 1 -- 172.21.15.83:0/1504676534 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac03f030 con 0x7ff8bc102650 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.396+0000 7ff8b37fe640 1 -- 172.21.15.83:0/1504676534 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ac003aa0 con 0x7ff8bc102650 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.396+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc1a28e0 con 0x7ff8bc102650 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.396+0000 7ff8b37fe640 1 -- 172.21.15.83:0/1504676534 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac035070 con 0x7ff8bc102650 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.396+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc1a90a0 con 0x7ff8bc102650 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 Processor -- start 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 -- start start 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68a2070 0x5633c6c492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68791b0 0x5633c6c49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5633c6bbfea0 con 0x5633c6881390 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5633c6baf2d0 con 0x5633c68791b0 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.399+0000 7ff8c7198740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5633c6bb30e0 con 0x5633c68a2070 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.400+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.400+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37954/0 (socket says 172.21.15.83:37954) 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.400+0000 7ff8b3fff640 1 -- 172.21.15.83:0/2637127133 learned_addr learned my addr 172.21.15.83:0/2637127133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.400+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68a2070 0x5633c6c492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b37fe640 1 -- 172.21.15.83:0/1504676534 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff8ac042030 con 0x7ff8bc102650 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b3fff640 1 -- 172.21.15.83:0/2637127133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68a2070 msgr2=0x5633c6c492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68a2070 0x5633c6c492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b3fff640 1 -- 172.21.15.83:0/2637127133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68791b0 msgr2=0x5633c6c49800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68791b0 0x5633c6c49800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.401+0000 7ff8b3fff640 1 -- 172.21.15.83:0/2637127133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c690d650 con 0x5633c6881390 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.402+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400bcb0 tx=0x7ff8a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7d5cecacfd8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.402+0000 7ff8b0ff9640 1 -- 172.21.15.83:0/2637127133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4004030 con 0x5633c6881390 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.402+0000 7ff8b0ff9640 1 -- 172.21.15.83:0/2637127133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a40035f0 con 0x5633c6881390 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.402+0000 7ff8b0ff9640 1 -- 172.21.15.83:0/2637127133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40116a0 con 0x5633c6881390 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.402+0000 7ff8b37fe640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 0x7ff8a80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 0x7ff8a80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8b37fe640 1 -- 172.21.15.83:0/1504676534 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7ff8ac107ca0 con 0x7ff8bc102650 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 -- 172.21.15.83:0/2637127133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 msgr2=0x5633c6c49d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400bcb0 tx=0x7ff8a40107a0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 0x7ff8a80b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099a0 tx=0x7ff8b4009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 -- 172.21.15.83:0/2637127133 shutdown_connections 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c49d40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68791b0 0x5633c6c49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 --2- 172.21.15.83:0/2637127133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68a2070 0x5633c6c492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 -- 172.21.15.83:0/2637127133 >> 172.21.15.83:0/2637127133 conn(0x5633c6896460 msgr2=0x5633c6c471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 -- 172.21.15.83:0/2637127133 shutdown_connections 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.403+0000 7ff8c7198740 1 -- 172.21.15.83:0/2637127133 wait complete. 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 -- start start 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 Processor -- start 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c54970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 0x5633c6c60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5633c6a10700 con 0x5633c68a2070 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5633c6b4a160 con 0x5633c6881390 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5633c6c57aa0 con 0x5633c68791b0 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8c7198740 10 client.0 _ll_register_callbacks cb 0x5633c6b639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.404+0000 7ff8bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 0x5633c6c60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35710/0 (socket says 172.21.15.83:35710) 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8b3fff640 1 -- 172.21.15.83:0/1477620581 learned_addr learned my addr 172.21.15.83:0/1477620581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 -- 172.21.15.83:0/1477620581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 msgr2=0x5633c6c54970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c54970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 -- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 msgr2=0x5633c6c54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 -- 172.21.15.83:0/1477620581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c6b77c90 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 0x5633c6c60d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40109b0 tx=0x7ff8b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73f1792014ff37b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004d20 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7ff8b401d990 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.405+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a40035f0 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8b401dbd0 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 10 client.17484 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5633c6c63860 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a40035f0 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7ff8b4027420 con 0x5633c68a2070 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 10 client.17484 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 20 client.17484 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68609f0 0x5633c6c66dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 4 client.17484 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c6bc1df0 con 0x5633c68609f0 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.406+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68609f0 0x5633c6c66dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.407+0000 7ff8a1ffb640 10 client.17484 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.407+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68609f0 0x5633c6c66dd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dd51ca8ad462ab3 server_cookie=82ecbec6fe114c5d in_seq=0 out_seq=0 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1477620581 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633c6bc1df0 con 0x5633c68609f0 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8a1ffb640 10 client.17484 handle_command_reply: tid=0 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 client.17484 shutdown 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 msgr2=0x5633c6c60d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 0x5633c6c60d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40109b0 tx=0x7ff8b4004a40 comp rx=0 tx=0).stop 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 shutdown_connections 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68609f0 0x5633c6c66dd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68a2070 0x5633c6c60d90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8a1ffb640 0 client.17484 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c6881390 0x5633c6c54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1477620581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c54970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 >> 172.21.15.83:0/1477620581 conn(0x5633c6896460 msgr2=0x5633c6c4c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 shutdown_connections 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.408+0000 7ff8c7198740 1 -- 172.21.15.83:0/1477620581 wait complete. 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.409+0000 7ff8c7198740 20 client.17484 trim_cache size 0 max 16384 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 Processor -- start 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 -- start start 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c56850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c4a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b1830 con 0x5633c6881390 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b33a0 con 0x5633c68609f0 2024-09-21T03:29:23.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.424+0000 7ff8c7198740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b1f00 con 0x5633c68791b0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54598/0 (socket says 172.21.15.83:54598) 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 -- 172.21.15.83:0/1883039618 learned_addr learned my addr 172.21.15.83:0/1883039618 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c56850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 -- 172.21.15.83:0/1883039618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 msgr2=0x5633c6c56850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c56850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 -- 172.21.15.83:0/1883039618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 msgr2=0x5633c6c4a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c4a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.425+0000 7ff8b3fff640 1 -- 172.21.15.83:0/1883039618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c6c55c30 con 0x5633c68791b0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.426+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a400bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a831cb23195a5ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.427+0000 7ff8a3fff640 1 -- 172.21.15.83:0/1883039618 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a402b070 con 0x5633c68791b0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.427+0000 7ff8a3fff640 1 -- 172.21.15.83:0/1883039618 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a40035f0 con 0x5633c68791b0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.427+0000 7ff8a3fff640 1 -- 172.21.15.83:0/1883039618 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40266e0 con 0x5633c68791b0 2024-09-21T03:29:23.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.427+0000 7ff8c7198740 1 -- 172.21.15.83:0/1883039618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 msgr2=0x5633c6c60c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.427+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a400bf30 comp rx=0 tx=0).stop 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 -- 172.21.15.83:0/1883039618 shutdown_connections 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c6881390 0x5633c6c4a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c68791b0 0x5633c6c60c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1883039618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c56850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 -- 172.21.15.83:0/1883039618 >> 172.21.15.83:0/1883039618 conn(0x5633c6896460 msgr2=0x5633c6c4a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 -- 172.21.15.83:0/1883039618 shutdown_connections 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 -- 172.21.15.83:0/1883039618 wait complete. 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 -- start start 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 Processor -- start 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.428+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68791b0 0x5633c6c59650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8c7198740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c6881390 0x5633c6cd0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8c7198740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b2ca0 con 0x5633c68791b0 2024-09-21T03:29:23.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8c7198740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b25d0 con 0x5633c68609f0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8c7198740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1af830 con 0x5633c6881390 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8c7198740 10 client.0 _ll_register_callbacks cb 0x7ff8b4009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35716/0 (socket says 172.21.15.83:35716) 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1637205250 learned_addr learned my addr 172.21.15.83:0/1637205250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8bad9d640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c6881390 0x5633c6cd0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1637205250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c6881390 msgr2=0x5633c6cd0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c6881390 0x5633c6cd0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1637205250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68791b0 msgr2=0x5633c6c59650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68791b0 0x5633c6c59650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 -- 172.21.15.83:0/1637205250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c6cd0cd0 con 0x5633c68609f0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8ba59c640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0cd7e0 tx=0x7ff8ac0cd970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d75a838f9199469 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:23.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.429+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac049040 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a40035f0 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7ff8ac0c9d50 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac0cd5f0 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 10 client.27041 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5633c6cd1f50 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a40035f0 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7ff8ac0d2220 con 0x5633c68609f0 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 10 client.27041 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 10 client.27041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 20 client.27041 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68a2070 0x5633c6cd4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 4 client.27041 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633c6bbd8a0 con 0x5633c68a2070 2024-09-21T03:29:23.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.430+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68a2070 0x5633c6cd4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8a1ffb640 10 client.27041 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8b3fff640 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68a2070 0x5633c6cd4670 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb9a554f5447b8d1 server_cookie=808cb5ffa9d0a9c2 in_seq=0 out_seq=0 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8a1ffb640 1 -- 172.21.15.83:0/1637205250 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633c6bbd8a0 con 0x5633c68a2070 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8a1ffb640 10 client.27041 handle_command_reply: tid=0 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 client.27041 shutdown 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 msgr2=0x5633c6c590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0cd7e0 tx=0x7ff8ac0cd970 comp rx=0 tx=0).stop 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 shutdown_connections 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5633c68a2070 0x5633c6cd4670 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633c6881390 0x5633c6cd0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633c68791b0 0x5633c6c59650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 --2- 172.21.15.83:0/1637205250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633c68609f0 0x5633c6c590d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 >> 172.21.15.83:0/1637205250 conn(0x5633c6896460 msgr2=0x5633c6c4ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8a1ffb640 0 client.27041 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 shutdown_connections 2024-09-21T03:29:23.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 1 -- 172.21.15.83:0/1637205250 wait complete. 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.431+0000 7ff8c7198740 20 client.27041 trim_cache size 0 max 16384 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 msgr2=0x7ff8a80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 0x7ff8a80b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099a0 tx=0x7ff8b4009500 comp rx=0 tx=0).stop 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 msgr2=0x7ff8bc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005940 tx=0x7ff8ac004270 comp rx=0 tx=0).stop 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 shutdown_connections 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff8a80b6680 0x7ff8a80b8b90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8bc110200 0x7ff8bc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8bc102fa0 0x7ff8bc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 --2- 172.21.15.83:0/1504676534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8bc102650 0x7ff8bc19c6e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:23.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 >> 172.21.15.83:0/1504676534 conn(0x7ff8bc079730 msgr2=0x7ff8bc108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:23.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 shutdown_connections 2024-09-21T03:29:23.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:23.432+0000 7ff8c0c24640 1 -- 172.21.15.83:0/1504676534 wait complete. 2024-09-21T03:29:24.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:29:24.594 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 -- 172.21.15.83:0/2182184992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 msgr2=0x7f5044102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 --2- 172.21.15.83:0/2182184992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044102e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5038009900 tx=0x7f503802a440 comp rx=0 tx=0).stop 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 -- 172.21.15.83:0/2182184992 shutdown_connections 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 --2- 172.21.15.83:0/2182184992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5044108cf0 0x7f504410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 --2- 172.21.15.83:0/2182184992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50441083a0 0x7f50441087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 --2- 172.21.15.83:0/2182184992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044102e70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 -- 172.21.15.83:0/2182184992 >> 172.21.15.83:0/2182184992 conn(0x7f5044079730 msgr2=0x7f5044079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 -- 172.21.15.83:0/2182184992 shutdown_connections 2024-09-21T03:29:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.189+0000 7f504ae3d640 1 -- 172.21.15.83:0/2182184992 wait complete. 2024-09-21T03:29:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 Processor -- start 2024-09-21T03:29:25.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 -- start start 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 0x7f504419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50441118a0 con 0x7f50441083a0 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5044111720 con 0x7f5044108cf0 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37990/0 (socket says 172.21.15.83:37990) 2024-09-21T03:29:25.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f5043fff640 1 -- 172.21.15.83:0/679716444 learned_addr learned my addr 172.21.15.83:0/679716444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f50493da640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 0x7f504419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.190+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5044111a20 con 0x7f5044102a60 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 -- 172.21.15.83:0/679716444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 msgr2=0x7f504419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 0x7f504419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 -- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 msgr2=0x7f5044198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 -- 172.21.15.83:0/679716444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504419e200 con 0x7f5044102a60 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5043fff640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f50493da640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 0x7f504419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044198290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f503802a960 tx=0x7f5038003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5041ffb640 1 -- 172.21.15.83:0/679716444 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503803f070 con 0x7f5044102a60 2024-09-21T03:29:25.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5041ffb640 1 -- 172.21.15.83:0/679716444 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503803bb20 con 0x7f5044102a60 2024-09-21T03:29:25.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f5041ffb640 1 -- 172.21.15.83:0/679716444 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038035030 con 0x7f5044102a60 2024-09-21T03:29:25.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.192+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504419e490 con 0x7f5044102a60 2024-09-21T03:29:25.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.193+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441a4c50 con 0x7f5044102a60 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 Processor -- start 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f5041ffb640 1 -- 172.21.15.83:0/679716444 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5038042070 con 0x7f5044102a60 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 -- start start 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f286800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eebe390 0x55c83f286d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50493da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c83f1fcea0 con 0x55c83eeb61b0 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c83f1ec2d0 con 0x55c83eebe390 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50513b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c83f1f00e0 con 0x55c83eedf070 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50493da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54636/0 (socket says 172.21.15.83:54636) 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f50493da640 1 -- 172.21.15.83:0/373582888 learned_addr learned my addr 172.21.15.83:0/373582888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.197+0000 7f5048bd9640 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f286800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f5043fff640 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eebe390 0x55c83f286d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 -- 172.21.15.83:0/373582888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eebe390 msgr2=0x55c83f286d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eebe390 0x55c83f286d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 -- 172.21.15.83:0/373582888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 msgr2=0x55c83f286800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f286800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 -- 172.21.15.83:0/373582888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83ef4a650 con 0x55c83eedf070 2024-09-21T03:29:25.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f50493da640 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f503c011790 tx=0x7f503c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a87aead399bb3ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f5041ffb640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 0x7f50340b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.198+0000 7f502effd640 1 -- 172.21.15.83:0/373582888 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c003c30 con 0x55c83eedf070 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.199+0000 7f502effd640 1 -- 172.21.15.83:0/373582888 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c00fe50 con 0x55c83eedf070 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.199+0000 7f502effd640 1 -- 172.21.15.83:0/373582888 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0193e0 con 0x55c83eedf070 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.199+0000 7f5041ffb640 1 -- 172.21.15.83:0/679716444 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f5038107bc0 con 0x7f5044102a60 2024-09-21T03:29:25.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.199+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 0x7f50340b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- 172.21.15.83:0/373582888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 msgr2=0x55c83f2862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f503c011790 tx=0x7f503c011c50 comp rx=0 tx=0).stop 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- 172.21.15.83:0/373582888 shutdown_connections 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eebe390 0x55c83f286d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f286800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 --2- 172.21.15.83:0/373582888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2862c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- 172.21.15.83:0/373582888 >> 172.21.15.83:0/373582888 conn(0x55c83eed3460 msgr2=0x55c83f2841b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f5048bd9640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 0x7f50340b8c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55c83f284f40 tx=0x7f503810b5f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- 172.21.15.83:0/373582888 shutdown_connections 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- 172.21.15.83:0/373582888 wait complete. 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.200+0000 7f50513b1740 1 -- start start 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 Processor -- start 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eeb61b0 0x55c83f29a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c83f04d700 con 0x55c83eebe390 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c83f187160 con 0x55c83eeb61b0 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c83f29d1d0 con 0x55c83eedf070 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f50513b1740 10 client.0 _ll_register_callbacks cb 0x55c83f1a09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:25.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.201+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38008/0 (socket says 172.21.15.83:38008) 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 -- 172.21.15.83:0/1879727887 learned_addr learned my addr 172.21.15.83:0/1879727887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f50493da640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eeb61b0 0x55c83f29a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f50493da640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 -- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 msgr2=0x55c83f2a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 -- 172.21.15.83:0/1879727887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eeb61b0 msgr2=0x55c83f29a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eeb61b0 0x55c83f29a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 -- 172.21.15.83:0/1879727887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83f1b4c90 con 0x55c83eebe390 2024-09-21T03:29:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5043fff640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5030009b00 tx=0x7f5030009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b060334ab610b524 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5023fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.202+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503001d030 con 0x55c83eebe390 2024-09-21T03:29:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.203+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f50300139a0 con 0x55c83eebe390 2024-09-21T03:29:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.203+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c00fe50 con 0x55c83eebe390 2024-09-21T03:29:25.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.204+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5030027050 con 0x55c83eebe390 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.204+0000 7f50513b1740 10 client.17496 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.204+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c83f2a8fc0 con 0x55c83eebe390 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50300054d0 con 0x55c83eebe390 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f5030004be0 con 0x55c83eebe390 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 10 client.17496 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 20 client.17496 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee9d9f0 0x55c83f2ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 4 client.17496 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f503c00fe50 con 0x55c83ee9d9f0 2024-09-21T03:29:25.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.205+0000 7f50493da640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee9d9f0 0x55c83f2ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.206+0000 7f50493da640 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee9d9f0 0x55c83f2ac550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2aa2a02f8afb4b43 server_cookie=c3bdf09247059979 in_seq=0 out_seq=0 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.206+0000 7f5023fff640 10 client.17496 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f5023fff640 1 -- 172.21.15.83:0/1879727887 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f503c00fe50 con 0x55c83ee9d9f0 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f5023fff640 10 client.17496 handle_command_reply: tid=0 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 client.17496 shutdown 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 msgr2=0x55c83f29a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5030009b00 tx=0x7f5030009fc0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 shutdown_connections 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee9d9f0 0x55c83f2ac550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f5023fff640 0 client.17496 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:29:25.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eebe390 0x55c83f29a6d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 --2- 172.21.15.83:0/1879727887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83eeb61b0 0x55c83f29a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 >> 172.21.15.83:0/1879727887 conn(0x55c83eed3460 msgr2=0x55c83f282750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 shutdown_connections 2024-09-21T03:29:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.207+0000 7f50513b1740 1 -- 172.21.15.83:0/1879727887 wait complete. 2024-09-21T03:29:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.208+0000 7f50513b1740 20 client.17496 trim_cache size 0 max 16384 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 Processor -- start 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 -- start start 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50441ad3e0 con 0x55c83eeb61b0 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50441aef50 con 0x55c83ee9d9f0 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50513b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50441adab0 con 0x55c83eedf070 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.223+0000 7f50493da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f5043fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38024/0 (socket says 172.21.15.83:38024) 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f5043fff640 1 -- 172.21.15.83:0/251576644 learned_addr learned my addr 172.21.15.83:0/251576644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35762/0 (socket says 172.21.15.83:35762) 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 -- 172.21.15.83:0/251576644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 msgr2=0x55c83f2978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 -- 172.21.15.83:0/251576644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 msgr2=0x55c83f297380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 -- 172.21.15.83:0/251576644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83f29b020 con 0x55c83ee9d9f0 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f5043fff640 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f50493da640 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f503c004590 tx=0x7f503c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb0b0de7598d126 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f502e7fc640 1 -- 172.21.15.83:0/251576644 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c004220 con 0x55c83ee9d9f0 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f502e7fc640 1 -- 172.21.15.83:0/251576644 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c00f9a0 con 0x55c83ee9d9f0 2024-09-21T03:29:25.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.224+0000 7f502e7fc640 1 -- 172.21.15.83:0/251576644 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c01a440 con 0x55c83ee9d9f0 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.225+0000 7f50513b1740 1 -- 172.21.15.83:0/251576644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 msgr2=0x55c83f2a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.225+0000 7f50513b1740 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f503c004590 tx=0x7f503c009500 comp rx=0 tx=0).stop 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.225+0000 7f50513b1740 1 -- 172.21.15.83:0/251576644 shutdown_connections 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.225+0000 7f50513b1740 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f2978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f297380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 --2- 172.21.15.83:0/251576644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f2a6460 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 -- 172.21.15.83:0/251576644 >> 172.21.15.83:0/251576644 conn(0x55c83eed3460 msgr2=0x55c83f282980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 -- 172.21.15.83:0/251576644 shutdown_connections 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 -- 172.21.15.83:0/251576644 wait complete. 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 -- start start 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.226+0000 7f50513b1740 1 Processor -- start 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f29f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f315f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50441ae850 con 0x55c83eeb61b0 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50441ae180 con 0x55c83ee9d9f0 2024-09-21T03:29:25.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50441ab3e0 con 0x55c83eedf070 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50513b1740 10 client.0 _ll_register_callbacks cb 0x7f503000bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50493da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50493da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35774/0 (socket says 172.21.15.83:35774) 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50493da640 1 -- 172.21.15.83:0/3415168923 learned_addr learned my addr 172.21.15.83:0/3415168923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f5043fff640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f29f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.227+0000 7f50493da640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f315f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 -- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 msgr2=0x55c83f315f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f315f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 -- 172.21.15.83:0/3415168923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 msgr2=0x55c83f29f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f29f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 -- 172.21.15.83:0/3415168923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83f316500 con 0x55c83ee9d9f0 2024-09-21T03:29:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50493da640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f503c00fc30 tx=0x7f503c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecaf33c0125dad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f5023fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c003a30 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f503c003bd0 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c00f9a0 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f503c003dd0 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50513b1740 10 client.27059 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c83f317760 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.228+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c00f9a0 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f503c018070 con 0x55c83ee9d9f0 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 10 client.27059 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:25.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 10 client.27059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 20 client.27059 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee72da0 0x55c83f319d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 4 client.27059 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c83f1fedf0 con 0x55c83ee72da0 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f5043fff640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee72da0 0x55c83f319d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f5023fff640 10 client.27059 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.229+0000 7f5043fff640 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee72da0 0x55c83f319d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a78090a73544f85 server_cookie=fea00cce00d4f4dd in_seq=0 out_seq=0 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f5023fff640 1 -- 172.21.15.83:0/3415168923 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c83f1fedf0 con 0x55c83ee72da0 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f5023fff640 10 client.27059 handle_command_reply: tid=0 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 client.27059 shutdown 2024-09-21T03:29:25.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 msgr2=0x55c83f29ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f503c00fc30 tx=0x7f503c011c50 comp rx=0 tx=0).stop 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 shutdown_connections 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55c83ee72da0 0x55c83f319d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c83eedf070 0x55c83f315f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c83eeb61b0 0x55c83f29f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 --2- 172.21.15.83:0/3415168923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c83ee9d9f0 0x55c83f29ecb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 >> 172.21.15.83:0/3415168923 conn(0x55c83eed3460 msgr2=0x55c83f2872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f5023fff640 0 client.27059 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 shutdown_connections 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 1 -- 172.21.15.83:0/3415168923 wait complete. 2024-09-21T03:29:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.230+0000 7f50513b1740 20 client.27059 trim_cache size 0 max 16384 2024-09-21T03:29:25.240 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 msgr2=0x7f50340b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 0x7f50340b8c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55c83f284f40 tx=0x7f503810b5f0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 msgr2=0x7f5044198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044198290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f503802a960 tx=0x7f5038003c10 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 shutdown_connections 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f50340b6780 0x7f50340b8c90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5044108cf0 0x7f504419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50441083a0 0x7f5044198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 --2- 172.21.15.83:0/679716444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5044102a60 0x7f5044198290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:25.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 >> 172.21.15.83:0/679716444 conn(0x7f5044079730 msgr2=0x7f5044101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 shutdown_connections 2024-09-21T03:29:25.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:25.231+0000 7f504ae3d640 1 -- 172.21.15.83:0/679716444 wait complete. 2024-09-21T03:29:25.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:25 smithi083 ceph-mon[31219]: pgmap v721: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T03:29:25.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:25 smithi142 ceph-mon[34512]: pgmap v721: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T03:29:25.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:25 smithi193 ceph-mon[35224]: pgmap v721: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T03:29:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 725.0 s, 1461.2 tps, lat 67.910 ms stddev 70.189 2024-09-21T03:29:26.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:29:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 -- 172.21.15.83:0/2555749811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410b8e0 msgr2=0x7efea410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 --2- 172.21.15.83:0/2555749811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410b8e0 0x7efea410bcf0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efe94009860 tx=0x7efe9402a340 comp rx=0 tx=0).stop 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 -- 172.21.15.83:0/2555749811 shutdown_connections 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 --2- 172.21.15.83:0/2555749811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 0x7efea407b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 --2- 172.21.15.83:0/2555749811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea410f8c0 0x7efea410fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 --2- 172.21.15.83:0/2555749811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410b8e0 0x7efea410bcf0 secure :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efe94009860 tx=0x7efe9402a340 comp rx=0 tx=0).stop 2024-09-21T03:29:27.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.007+0000 7efea359e640 1 -- 172.21.15.83:0/2555749811 >> 172.21.15.83:0/2555749811 conn(0x7efea406e9f0 msgr2=0x7efea406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.010+0000 7efea359e640 1 -- 172.21.15.83:0/2555749811 shutdown_connections 2024-09-21T03:29:27.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.010+0000 7efea359e640 1 -- 172.21.15.83:0/2555749811 wait complete. 2024-09-21T03:29:27.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.011+0000 7efea359e640 1 Processor -- start 2024-09-21T03:29:27.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.011+0000 7efea359e640 1 -- start start 2024-09-21T03:29:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 0x7efea40853c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 0x7efea407f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 0x7efea41ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efea407d990 con 0x7efea410f8c0 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efea407d810 con 0x7efea4110210 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea359e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efea407db10 con 0x7efea407fc60 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.012+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 0x7efea41ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.013+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 0x7efea41ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:54686/0 (socket says 172.21.15.83:54686) 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.013+0000 7efea2d9d640 1 -- 172.21.15.83:0/121269366 learned_addr learned my addr 172.21.15.83:0/121269366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.013+0000 7efea259c640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 0x7efea40853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea1d9b640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 0x7efea407f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 -- 172.21.15.83:0/121269366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 msgr2=0x7efea41ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 0x7efea41ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 -- 172.21.15.83:0/121269366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 msgr2=0x7efea407f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 0x7efea407f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 -- 172.21.15.83:0/121269366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea41cf0e0 con 0x7efea410f8c0 2024-09-21T03:29:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.014+0000 7efea259c640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 0x7efea40853c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efe94008000 tx=0x7efe94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.015+0000 7efe937fe640 1 -- 172.21.15.83:0/121269366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94040020 con 0x7efea410f8c0 2024-09-21T03:29:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.015+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea41cf2b0 con 0x7efea410f8c0 2024-09-21T03:29:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.015+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea41cf650 con 0x7efea410f8c0 2024-09-21T03:29:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.015+0000 7efe937fe640 1 -- 172.21.15.83:0/121269366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe940054d0 con 0x7efea410f8c0 2024-09-21T03:29:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.017+0000 7efe937fe640 1 -- 172.21.15.83:0/121269366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94037040 con 0x7efea410f8c0 2024-09-21T03:29:27.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.019+0000 7efe937fe640 1 -- 172.21.15.83:0/121269366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efe94004e30 con 0x7efea410f8c0 2024-09-21T03:29:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.020+0000 7efe937fe640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 0x7efe8c0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.021+0000 7efe937fe640 1 -- 172.21.15.83:0/121269366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efe941069a0 con 0x7efea410f8c0 2024-09-21T03:29:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.025+0000 7efeaf51c740 1 Processor -- start 2024-09-21T03:29:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efea1d9b640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 0x7efe8c0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 -- start start 2024-09-21T03:29:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f50f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f50fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56293f485ea0 con 0x56293f168070 2024-09-21T03:29:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56293f4752d0 con 0x56293f147390 2024-09-21T03:29:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.027+0000 7efeaf51c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56293f4790e0 con 0x56293f13f1b0 2024-09-21T03:29:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f50f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35804/0 (socket says 172.21.15.83:35804) 2024-09-21T03:29:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f50f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38068/0 (socket says 172.21.15.83:38068) 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea2d9d640 1 -- 172.21.15.83:0/6720481 learned_addr learned my addr 172.21.15.83:0/6720481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.029+0000 7efea1d9b640 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f50fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea1d9b640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 0x7efe8c0b8c00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efea410c8e0 tx=0x7efe9c0095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea2d9d640 1 -- 172.21.15.83:0/6720481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 msgr2=0x56293f50fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea2d9d640 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f50fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea2d9d640 1 -- 172.21.15.83:0/6720481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 msgr2=0x56293f50f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea2d9d640 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f50f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.030+0000 7efea2d9d640 1 -- 172.21.15.83:0/6720481 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293f1d3650 con 0x56293f147390 2024-09-21T03:29:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.033+0000 7efea2d9d640 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe9800bca0 tx=0x7efe9800a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f9816593590d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.033+0000 7efe90ff9640 1 -- 172.21.15.83:0/6720481 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980155f0 con 0x56293f147390 2024-09-21T03:29:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.034+0000 7efe90ff9640 1 -- 172.21.15.83:0/6720481 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9801a070 con 0x56293f147390 2024-09-21T03:29:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.035+0000 7efeaf51c740 1 -- 172.21.15.83:0/6720481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 msgr2=0x56293f5102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.035+0000 7efeaf51c740 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe9800bca0 tx=0x7efe9800a680 comp rx=0 tx=0).stop 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 -- 172.21.15.83:0/6720481 shutdown_connections 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f147390 0x56293f5102e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f50fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 --2- 172.21.15.83:0/6720481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f50f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 -- 172.21.15.83:0/6720481 >> 172.21.15.83:0/6720481 conn(0x56293f15c460 msgr2=0x56293f50d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.036+0000 7efeaf51c740 1 -- 172.21.15.83:0/6720481 shutdown_connections 2024-09-21T03:29:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.037+0000 7efeaf51c740 1 -- 172.21.15.83:0/6720481 wait complete. 2024-09-21T03:29:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.037+0000 7efeaf51c740 1 -- start start 2024-09-21T03:29:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.038+0000 7efeaf51c740 1 Processor -- start 2024-09-21T03:29:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.038+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.038+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f51b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efea259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efea259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35812/0 (socket says 172.21.15.83:35812) 2024-09-21T03:29:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efea259c640 1 -- 172.21.15.83:0/3602787516 learned_addr learned my addr 172.21.15.83:0/3602787516 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efea1d9b640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56293f2d6700 con 0x56293f168070 2024-09-21T03:29:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56293f410160 con 0x56293f13f1b0 2024-09-21T03:29:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56293f51e090 con 0x56293f147390 2024-09-21T03:29:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.039+0000 7efeaf51c740 10 client.0 _ll_register_callbacks cb 0x56293f4299e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea2d9d640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f51b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea259c640 1 -- 172.21.15.83:0/3602787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 msgr2=0x56293f51b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea259c640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea259c640 1 -- 172.21.15.83:0/3602787516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 msgr2=0x56293f51b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea259c640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f51b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.042+0000 7efea259c640 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293f43dc90 con 0x56293f13f1b0 2024-09-21T03:29:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.043+0000 7efea259c640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe940303f0 tx=0x7efe94005460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f95b7dcbe9544b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.044+0000 7efe89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:29:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.046+0000 7efea2d9d640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f51b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.046+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe940d1070 con 0x56293f13f1b0 2024-09-21T03:29:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.046+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efe940cdd90 con 0x56293f13f1b0 2024-09-21T03:29:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.046+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56293f487df0 con 0x56293f13f1b0 2024-09-21T03:29:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.047+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efe940cc1b0 con 0x56293f13f1b0 2024-09-21T03:29:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.047+0000 7efeaf51c740 10 client.27068 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.047+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56293f529e80 con 0x56293f13f1b0 2024-09-21T03:29:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.049+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56293f487df0 con 0x56293f13f1b0 2024-09-21T03:29:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.049+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7efe940c81b0 con 0x56293f13f1b0 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 10 client.27068 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 10 client.27068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 20 client.27068 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f1269f0 0x56293f52d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 4 client.27068 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56293f4838a0 con 0x56293f1269f0 2024-09-21T03:29:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.050+0000 7efea1d9b640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f1269f0 0x56293f52d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.056+0000 7efea1d9b640 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f1269f0 0x56293f52d310 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f711639d5106c42 server_cookie=acb61ff41f01d6e1 in_seq=0 out_seq=0 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.057+0000 7efe89ffb640 10 client.27068 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.057+0000 7efe89ffb640 1 -- 172.21.15.83:0/3602787516 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56293f4838a0 con 0x56293f1269f0 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.057+0000 7efe89ffb640 10 client.27068 handle_command_reply: tid=0 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.058+0000 7efeaf51c740 1 client.27068 shutdown 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.058+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 msgr2=0x56293f51aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.058+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe940303f0 tx=0x7efe94005460 comp rx=0 tx=0).stop 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 shutdown_connections 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f1269f0 0x56293f52d310 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f168070 0x56293f51b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 --2- 172.21.15.83:0/3602787516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f13f1b0 0x56293f51aed0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 >> 172.21.15.83:0/3602787516 conn(0x56293f15c460 msgr2=0x56293f512a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efe89ffb640 0 client.27068 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 shutdown_connections 2024-09-21T03:29:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.059+0000 7efeaf51c740 1 -- 172.21.15.83:0/3602787516 wait complete. 2024-09-21T03:29:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.060+0000 7efeaf51c740 20 client.27068 trim_cache size 0 max 16384 2024-09-21T03:29:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 Processor -- start 2024-09-21T03:29:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 -- start start 2024-09-21T03:29:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f13f1b0 0x56293f51ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400e770 con 0x56293f13f1b0 2024-09-21T03:29:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe840102e0 con 0x56293f168070 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.078+0000 7efeaf51c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400ee40 con 0x56293f147390 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.079+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.079+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:35818/0 (socket says 172.21.15.83:35818) 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.079+0000 7efea2d9d640 1 -- 172.21.15.83:0/1584280888 learned_addr learned my addr 172.21.15.83:0/1584280888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.079+0000 7efea259c640 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f13f1b0 0x56293f51ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.079+0000 7efea1d9b640 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 -- 172.21.15.83:0/1584280888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 msgr2=0x56293f51b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 -- 172.21.15.83:0/1584280888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f13f1b0 msgr2=0x56293f51ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f13f1b0 0x56293f51ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 -- 172.21.15.83:0/1584280888 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293f5126c0 con 0x56293f168070 2024-09-21T03:29:27.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efea2d9d640 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b540 tx=0x7efe98005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9814db1ea22d6db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efe8bfff640 1 -- 172.21.15.83:0/1584280888 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98022020 con 0x56293f168070 2024-09-21T03:29:27.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efe8bfff640 1 -- 172.21.15.83:0/1584280888 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9801a070 con 0x56293f168070 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.080+0000 7efe8bfff640 1 -- 172.21.15.83:0/1584280888 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980048e0 con 0x56293f168070 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 -- 172.21.15.83:0/1584280888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 msgr2=0x56293f5180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b540 tx=0x7efe98005fd0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 -- 172.21.15.83:0/1584280888 shutdown_connections 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f5180c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f147390 0x56293f51b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1584280888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f13f1b0 0x56293f51ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 -- 172.21.15.83:0/1584280888 >> 172.21.15.83:0/1584280888 conn(0x56293f15c460 msgr2=0x56293f519750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.081+0000 7efeaf51c740 1 -- 172.21.15.83:0/1584280888 shutdown_connections 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.082+0000 7efeaf51c740 1 -- 172.21.15.83:0/1584280888 wait complete. 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.082+0000 7efeaf51c740 1 -- start start 2024-09-21T03:29:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.082+0000 7efeaf51c740 1 Processor -- start 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f523bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f51f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400fbe0 con 0x56293f147390 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efea1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efea1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38106/0 (socket says 172.21.15.83:38106) 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efea1d9b640 1 -- 172.21.15.83:0/1101690921 learned_addr learned my addr 172.21.15.83:0/1101690921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efea259c640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f523bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400f510 con 0x56293f168070 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe8400c770 con 0x56293f13f1b0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efeaf51c740 10 client.0 _ll_register_callbacks cb 0x7efe94005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.083+0000 7efea2d9d640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f51f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 -- 172.21.15.83:0/1101690921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 msgr2=0x56293f523bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f523bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 -- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 msgr2=0x56293f51f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f51f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293f51fd30 con 0x56293f147390 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea259c640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f523bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efea1d9b640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efea406c910 tx=0x7efe9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b2601deac18ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efe89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c01e040 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efe9c004b20 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9801a070 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efe9c004d60 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efeaf51c740 10 client.17517 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.084+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56293f597c90 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9c0054d0 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7efe9c00b040 con 0x56293f147390 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 10 client.17517 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 20 client.17517 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f0fbda0 0x56293f59a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 4 client.17517 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efe9801a070 con 0x56293f0fbda0 2024-09-21T03:29:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efea259c640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f0fbda0 0x56293f59a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.085+0000 7efe89ffb640 10 client.17517 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efea259c640 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f0fbda0 0x56293f59a4e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865aa8f4baac4988 server_cookie=e0a426c76f4ec315 in_seq=0 out_seq=0 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efe89ffb640 1 -- 172.21.15.83:0/1101690921 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efe9801a070 con 0x56293f0fbda0 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efe89ffb640 10 client.17517 handle_command_reply: tid=0 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 client.17517 shutdown 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 msgr2=0x56293f51f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efea406c910 tx=0x7efe9c009f90 comp rx=0 tx=0).stop 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 shutdown_connections 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56293f0fbda0 0x56293f59a4e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efe89ffb640 0 client.17517 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56293f168070 0x56293f51f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56293f147390 0x56293f51f060 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 --2- 172.21.15.83:0/1101690921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56293f13f1b0 0x56293f523bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 >> 172.21.15.83:0/1101690921 conn(0x56293f15c460 msgr2=0x56293f51ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 shutdown_connections 2024-09-21T03:29:27.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.086+0000 7efeaf51c740 1 -- 172.21.15.83:0/1101690921 wait complete. 2024-09-21T03:29:27.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efeaf51c740 20 client.17517 trim_cache size 0 max 16384 2024-09-21T03:29:27.094 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 msgr2=0x7efe8c0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 0x7efe8c0b8c00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efea410c8e0 tx=0x7efe9c0095c0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 msgr2=0x7efea40853c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 0x7efea40853c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efe94008000 tx=0x7efe94004270 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 shutdown_connections 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe8c0b66f0 0x7efe8c0b8c00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea407fc60 0x7efea41ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4110210 0x7efea407f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 --2- 172.21.15.83:0/121269366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efea410f8c0 0x7efea40853c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.087+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 >> 172.21.15.83:0/121269366 conn(0x7efea406e9f0 msgr2=0x7efea410dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.088+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 shutdown_connections 2024-09-21T03:29:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:27.088+0000 7efea359e640 1 -- 172.21.15.83:0/121269366 wait complete. 2024-09-21T03:29:27.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:27 smithi083 ceph-mon[31219]: pgmap v722: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T03:29:27.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:27 smithi142 ceph-mon[34512]: pgmap v722: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T03:29:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:27 smithi193 ceph-mon[35224]: pgmap v722: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T03:29:28.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:29:28.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:29:29.313 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:29.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:29 smithi083 ceph-mon[31219]: pgmap v723: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-21T03:29:29.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:29 smithi142 ceph-mon[34512]: pgmap v723: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-21T03:29:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:29 smithi193 ceph-mon[35224]: pgmap v723: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-09-21T03:29:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 -- 172.21.15.83:0/1953001390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880105270 msgr2=0x7f488010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 --2- 172.21.15.83:0/1953001390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880105270 0x7f488010d8b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4870009900 tx=0x7f487002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 -- 172.21.15.83:0/1953001390 shutdown_connections 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 --2- 172.21.15.83:0/1953001390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f4880110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 --2- 172.21.15.83:0/1953001390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880105270 0x7f488010d8b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 --2- 172.21.15.83:0/1953001390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880104840 0x7f4880104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 -- 172.21.15.83:0/1953001390 >> 172.21.15.83:0/1953001390 conn(0x7f4880079730 msgr2=0x7f4880079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 -- 172.21.15.83:0/1953001390 shutdown_connections 2024-09-21T03:29:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.855+0000 7f488841f640 1 -- 172.21.15.83:0/1953001390 wait complete. 2024-09-21T03:29:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.856+0000 7f488841f640 1 Processor -- start 2024-09-21T03:29:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.856+0000 7f488841f640 1 -- start start 2024-09-21T03:29:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880104840 0x7f48801a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 0x7f48801a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f48801ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4880113a70 con 0x7f4880104840 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48801138f0 con 0x7f488010ddf0 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f488841f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4880113bf0 con 0x7f4880105270 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.857+0000 7f48859ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 0x7f48801a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48859ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 0x7f48801a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37412/0 (socket says 172.21.15.83:37412) 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f48801ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48859ba640 1 -- 172.21.15.83:0/315448208 learned_addr learned my addr 172.21.15.83:0/315448208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48861bb640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880104840 0x7f48801a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 -- 172.21.15.83:0/315448208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 msgr2=0x7f48801a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 0x7f48801a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 -- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880104840 msgr2=0x7f48801a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880104840 0x7f48801a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.858+0000 7f48869bc640 1 -- 172.21.15.83:0/315448208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801accd0 con 0x7f488010ddf0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f48869bc640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f48801ac790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4878009fd0 tx=0x7f487800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f48777fe640 1 -- 172.21.15.83:0/315448208 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800ce20 con 0x7f488010ddf0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801acea0 con 0x7f488010ddf0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f48777fe640 1 -- 172.21.15.83:0/315448208 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4878005020 con 0x7f488010ddf0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f48777fe640 1 -- 172.21.15.83:0/315448208 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878010690 con 0x7f488010ddf0 2024-09-21T03:29:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.859+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801ad3a0 con 0x7f488010ddf0 2024-09-21T03:29:29.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.862+0000 7f488e993740 1 Processor -- start 2024-09-21T03:29:29.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.862+0000 7f48777fe640 1 -- 172.21.15.83:0/315448208 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4878004b20 con 0x7f488010ddf0 2024-09-21T03:29:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.862+0000 7f488e993740 1 -- start start 2024-09-21T03:29:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e53970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e53eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557984dc9e60 con 0x557984a83490 2024-09-21T03:29:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557984db9290 con 0x557984a8b670 2024-09-21T03:29:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f488e993740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557984dbd0a0 con 0x557984aac350 2024-09-21T03:29:29.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f48861bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e53970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f48859ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f48859ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37418/0 (socket says 172.21.15.83:37418) 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f48859ba640 1 -- 172.21.15.83:0/3130107036 learned_addr learned my addr 172.21.15.83:0/3130107036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.863+0000 7f48859ba640 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e53eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48777fe640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 0x7f486c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 -- 172.21.15.83:0/3130107036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 msgr2=0x557984e53eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e53eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 -- 172.21.15.83:0/3130107036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 msgr2=0x557984e53970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e53970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 -- 172.21.15.83:0/3130107036 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557984b17930 con 0x557984aac350 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48861bb640 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e53970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48777fe640 1 -- 172.21.15.83:0/315448208 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f48780e7cd0 con 0x7f488010ddf0 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48869bc640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 0x7f486c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f48859ba640 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f48700057c0 tx=0x7f48700058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e36bdd91f0e84d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.864+0000 7f4874d9a640 1 -- 172.21.15.83:0/3130107036 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870035040 con 0x557984aac350 2024-09-21T03:29:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.865+0000 7f4874d9a640 1 -- 172.21.15.83:0/3130107036 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4870042070 con 0x557984aac350 2024-09-21T03:29:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.865+0000 7f4874d9a640 1 -- 172.21.15.83:0/3130107036 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48700039f0 con 0x557984aac350 2024-09-21T03:29:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f48869bc640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 0x7f486c0b8c10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f487800c760 tx=0x7f4878005f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 -- 172.21.15.83:0/3130107036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 msgr2=0x557984e53430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f48700057c0 tx=0x7f48700058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 -- 172.21.15.83:0/3130107036 shutdown_connections 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e53eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e53970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 --2- 172.21.15.83:0/3130107036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984aac350 0x557984e53430 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.866+0000 7f488e993740 1 -- 172.21.15.83:0/3130107036 >> 172.21.15.83:0/3130107036 conn(0x557984aa0740 msgr2=0x557984e51320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.867+0000 7f488e993740 1 -- 172.21.15.83:0/3130107036 shutdown_connections 2024-09-21T03:29:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.867+0000 7f488e993740 1 -- 172.21.15.83:0/3130107036 wait complete. 2024-09-21T03:29:29.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.867+0000 7f488e993740 1 -- start start 2024-09-21T03:29:29.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.867+0000 7f488e993740 1 Processor -- start 2024-09-21T03:29:29.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a83490 0x557984e5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557984c1a890 con 0x557984aac350 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557984d53d40 con 0x557984a8b670 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557984e61c50 con 0x557984a83490 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f488e993740 10 client.0 _ll_register_callbacks cb 0x557984db7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48861bb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49362/0 (socket says 172.21.15.83:49362) 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 -- 172.21.15.83:0/2426789506 learned_addr learned my addr 172.21.15.83:0/2426789506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a83490 0x557984e5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 -- 172.21.15.83:0/2426789506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a83490 msgr2=0x557984e5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a83490 0x557984e5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 -- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 msgr2=0x557984e5f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e5f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.868+0000 7f48859ba640 1 -- 172.21.15.83:0/2426789506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557984e6b4b0 con 0x557984aac350 2024-09-21T03:29:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f48859ba640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4870032e90 tx=0x7f4870005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb71084e46c5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f4869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487003e940 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f487003eae0 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4870042070 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f487003ece0 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f488e993740 10 client.17529 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.869+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557984e6ddc0 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4870042070 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f487003a040 con 0x557984aac350 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 10 client.17529 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 20 client.17529 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984a6acd0 0x557984e71120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 4 client.17529 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.870+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557984dcbdb0 con 0x557984a6acd0 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.871+0000 7f48861bb640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984a6acd0 0x557984e71120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.871+0000 7f48861bb640 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984a6acd0 0x557984e71120 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303972d7d6606f34 server_cookie=2f961e7712fc6782 in_seq=0 out_seq=0 2024-09-21T03:29:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.871+0000 7f4869ffb640 10 client.17529 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f4869ffb640 1 -- 172.21.15.83:0/2426789506 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557984dcbdb0 con 0x557984a6acd0 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f4869ffb640 10 client.17529 handle_command_reply: tid=0 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f488e993740 1 client.17529 shutdown 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 msgr2=0x557984e6af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4870032e90 tx=0x7f4870005c90 comp rx=0 tx=0).stop 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 shutdown_connections 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.872+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984a6acd0 0x557984e71120 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f4869ffb640 0 client.17529 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984aac350 0x557984e6af70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984a8b670 0x557984e5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 --2- 172.21.15.83:0/2426789506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a83490 0x557984e5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 >> 172.21.15.83:0/2426789506 conn(0x557984aa0740 msgr2=0x557984e565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 shutdown_connections 2024-09-21T03:29:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.873+0000 7f488e993740 1 -- 172.21.15.83:0/2426789506 wait complete. 2024-09-21T03:29:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.874+0000 7f488e993740 20 client.17529 trim_cache size 0 max 16384 2024-09-21T03:29:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.904+0000 7f488e993740 1 Processor -- start 2024-09-21T03:29:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.904+0000 7f488e993740 1 -- start start 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a8b670 0x557984e584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b5be0 con 0x557984a83490 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b7750 con 0x557984aac350 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f488e993740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b62b0 con 0x557984a8b670 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49364/0 (socket says 172.21.15.83:49364) 2024-09-21T03:29:29.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 -- 172.21.15.83:0/673965326 learned_addr learned my addr 172.21.15.83:0/673965326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48861bb640 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a8b670 0x557984e584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 -- 172.21.15.83:0/673965326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a8b670 msgr2=0x557984e584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a8b670 0x557984e584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 -- 172.21.15.83:0/673965326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 msgr2=0x557984e58a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e58a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.905+0000 7f48859ba640 1 -- 172.21.15.83:0/673965326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557984e5dd20 con 0x557984a83490 2024-09-21T03:29:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.915+0000 7f48859ba640 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f48700057c0 tx=0x7f4870031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4c26c4a5dd7668 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.921+0000 7f486bfff640 1 -- 172.21.15.83:0/673965326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870035020 con 0x557984a83490 2024-09-21T03:29:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.921+0000 7f486bfff640 1 -- 172.21.15.83:0/673965326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4870042070 con 0x557984a83490 2024-09-21T03:29:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.921+0000 7f486bfff640 1 -- 172.21.15.83:0/673965326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870005570 con 0x557984a83490 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 -- 172.21.15.83:0/673965326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 msgr2=0x557984e56c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f48700057c0 tx=0x7f4870031400 comp rx=0 tx=0).stop 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 -- 172.21.15.83:0/673965326 shutdown_connections 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e58a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a8b670 0x557984e584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 --2- 172.21.15.83:0/673965326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a83490 0x557984e56c40 secure :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f48700057c0 tx=0x7f4870031400 comp rx=0 tx=0).stop 2024-09-21T03:29:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.923+0000 7f488e993740 1 -- 172.21.15.83:0/673965326 >> 172.21.15.83:0/673965326 conn(0x557984aa0740 msgr2=0x557984e56530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.931+0000 7f488e993740 1 -- 172.21.15.83:0/673965326 shutdown_connections 2024-09-21T03:29:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.934+0000 7f488e993740 1 -- 172.21.15.83:0/673965326 wait complete. 2024-09-21T03:29:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.935+0000 7f488e993740 1 -- start start 2024-09-21T03:29:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.935+0000 7f488e993740 1 Processor -- start 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 0x557984e73450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 0x557984edb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b7050 con 0x557984a8b670 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b6980 con 0x557984aac350 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b3be0 con 0x557984a40080 2024-09-21T03:29:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.936+0000 7f488e993740 10 client.0 _ll_register_callbacks cb 0x7f487c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.944+0000 7f48859ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 0x557984edb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.944+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 0x557984e73450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.944+0000 7f48859ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 0x557984e73450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49372/0 (socket says 172.21.15.83:49372) 2024-09-21T03:29:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.944+0000 7f48859ba640 1 -- 172.21.15.83:0/1832025749 learned_addr learned my addr 172.21.15.83:0/1832025749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.945+0000 7f48861bb640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.949+0000 7f48859ba640 1 -- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 msgr2=0x557984e73990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.949+0000 7f48859ba640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e73990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.949+0000 7f48859ba640 1 -- 172.21.15.83:0/1832025749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 msgr2=0x557984e73450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.949+0000 7f48859ba640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 0x557984e73450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.949+0000 7f48859ba640 1 -- 172.21.15.83:0/1832025749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557984e74210 con 0x557984a40080 2024-09-21T03:29:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.953+0000 7f48859ba640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 0x557984edb8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x557984e6c480 tx=0x7f4870000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a670f0d267e40c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.953+0000 7f4869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:29:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.957+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870035040 con 0x557984a40080 2024-09-21T03:29:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.957+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4870042070 con 0x557984a40080 2024-09-21T03:29:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.957+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f487003e220 con 0x557984a40080 2024-09-21T03:29:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.960+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f48700051f0 con 0x557984a40080 2024-09-21T03:29:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.960+0000 7f488e993740 10 client.27049 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.960+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557984edce70 con 0x557984a40080 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4870042070 con 0x557984a40080 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f4870066040 con 0x557984a40080 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 10 client.27049 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 10 client.27049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 20 client.27049 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984edd380 0x557984edf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 4 client.27049 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:29:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.965+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557984dc7860 con 0x557984edd380 2024-09-21T03:29:29.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.966+0000 7f48861bb640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984edd380 0x557984edf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.969+0000 7f4869ffb640 10 client.27049 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.969+0000 7f48861bb640 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984edd380 0x557984edf7a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1053a9c5880f75fd server_cookie=5e755371579e9acc in_seq=0 out_seq=0 2024-09-21T03:29:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.972+0000 7f4869ffb640 1 -- 172.21.15.83:0/1832025749 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557984dc7860 con 0x557984edd380 2024-09-21T03:29:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.972+0000 7f4869ffb640 10 client.27049 handle_command_reply: tid=0 2024-09-21T03:29:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.972+0000 7f488e993740 1 client.27049 shutdown 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.973+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 msgr2=0x557984edb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.973+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 0x557984edb8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x557984e6c480 tx=0x7f4870000bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.973+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 shutdown_connections 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.973+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x557984edd380 0x557984edf7a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.974+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557984a40080 0x557984edb8b0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x557984e6c480 tx=0x7f4870000bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.974+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557984aac350 0x557984e73990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.974+0000 7f488e993740 1 --2- 172.21.15.83:0/1832025749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557984a8b670 0x557984e73450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.974+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 >> 172.21.15.83:0/1832025749 conn(0x557984aa0740 msgr2=0x557984e54460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.974+0000 7f4869ffb640 0 client.27049 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.975+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 shutdown_connections 2024-09-21T03:29:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.975+0000 7f488e993740 1 -- 172.21.15.83:0/1832025749 wait complete. 2024-09-21T03:29:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.975+0000 7f488e993740 20 client.27049 trim_cache size 0 max 16384 2024-09-21T03:29:29.977 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:29:29.977 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:29:29.978 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d107bf75-3d29-4357-a930-5487411ac4d2", 2024-09-21T03:29:29.978 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:29:29.978 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 msgr2=0x7f486c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 0x7f486c0b8c10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f487800c760 tx=0x7f4878005f70 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 msgr2=0x7f48801ac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f48801ac790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4878009fd0 tx=0x7f487800ef10 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 shutdown_connections 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f486c0b6700 0x7f486c0b8c10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488010ddf0 0x7f48801ac790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880105270 0x7f48801a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 --2- 172.21.15.83:0/315448208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4880104840 0x7f48801a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.977+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 >> 172.21.15.83:0/315448208 conn(0x7f4880079730 msgr2=0x7f48801068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.978+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 shutdown_connections 2024-09-21T03:29:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:29.978+0000 7f488841f640 1 -- 172.21.15.83:0/315448208 wait complete. 2024-09-21T03:29:30.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:29:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 730.0 s, 1517.1 tps, lat 66.000 ms stddev 79.068 2024-09-21T03:29:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:31.367 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: pgmap v724: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-09-21T03:29:31.367 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:29:31.367 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.367 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:31 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: pgmap v724: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:29:31.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.429 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:31 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: pgmap v724: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:31 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 -- 172.21.15.83:0/3997563067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0111f40 msgr2=0x7efeb010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 --2- 172.21.15.83:0/3997563067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0111f40 0x7efeb010a3c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe98009900 tx=0x7efe9802a440 comp rx=0 tx=0).stop 2024-09-21T03:29:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 -- 172.21.15.83:0/3997563067 shutdown_connections 2024-09-21T03:29:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 --2- 172.21.15.83:0/3997563067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb011cc30 0x7efeb011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 --2- 172.21.15.83:0/3997563067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb010a9e0 0x7efeb010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 --2- 172.21.15.83:0/3997563067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0111f40 0x7efeb010a3c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.822+0000 7efeb5496640 1 -- 172.21.15.83:0/3997563067 >> 172.21.15.83:0/3997563067 conn(0x7efeb0070ec0 msgr2=0x7efeb00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.826+0000 7efeb5496640 1 -- 172.21.15.83:0/3997563067 shutdown_connections 2024-09-21T03:29:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.826+0000 7efeb5496640 1 -- 172.21.15.83:0/3997563067 wait complete. 2024-09-21T03:29:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.827+0000 7efeb5496640 1 Processor -- start 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.830+0000 7efeb5496640 1 -- start start 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb010a9e0 0x7efeb011c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb011cc30 0x7efeb0116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112c80 con 0x7efeb010a9e0 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112b00 con 0x7efeb011cc30 2024-09-21T03:29:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.831+0000 7efeb5496640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112e00 con 0x7efeb0111f40 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.833+0000 7efeae7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.833+0000 7efeae7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37458/0 (socket says 172.21.15.83:37458) 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.833+0000 7efeae7fc640 1 -- 172.21.15.83:0/2508768432 learned_addr learned my addr 172.21.15.83:0/2508768432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.834+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb011cc30 0x7efeb0116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 -- 172.21.15.83:0/2508768432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb011cc30 msgr2=0x7efeb0116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb011cc30 0x7efeb0116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 -- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb010a9e0 msgr2=0x7efeb011c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb010a9e0 0x7efeb011c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 -- 172.21.15.83:0/2508768432 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01175a0 con 0x7efeb0111f40 2024-09-21T03:29:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.835+0000 7efeae7fc640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efea000b9b0 tx=0x7efea000be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.836+0000 7efea7fff640 1 -- 172.21.15.83:0/2508768432 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0019070 con 0x7efeb0111f40 2024-09-21T03:29:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.836+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb01bded0 con 0x7efeb0111f40 2024-09-21T03:29:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.836+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb01be380 con 0x7efeb0111f40 2024-09-21T03:29:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.837+0000 7efea7fff640 1 -- 172.21.15.83:0/2508768432 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea000cba0 con 0x7efeb0111f40 2024-09-21T03:29:31.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.837+0000 7efea7fff640 1 -- 172.21.15.83:0/2508768432 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00125a0 con 0x7efeb0111f40 2024-09-21T03:29:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.841+0000 7efea7fff640 1 -- 172.21.15.83:0/2508768432 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efea001c070 con 0x7efeb0111f40 2024-09-21T03:29:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.842+0000 7efea7fff640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 0x7efe9c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.842+0000 7efea7fff640 1 -- 172.21.15.83:0/2508768432 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efea00e6e90 con 0x7efeb0111f40 2024-09-21T03:29:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.844+0000 7efeaeffd640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 0x7efe9c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.847+0000 7efeaeffd640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 0x7efe9c0b8cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe98009230 tx=0x7efe98000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.849+0000 7efebba0a740 1 Processor -- start 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 -- start start 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282ab6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ab67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ab6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561282a2ce50 con 0x5612826ee390 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561282a1c280 con 0x5612826e61b0 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.854+0000 7efebba0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561282a20090 con 0x56128270f070 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.855+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282ab6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.855+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282ab6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:37474/0 (socket says 172.21.15.83:37474) 2024-09-21T03:29:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.855+0000 7efeaeffd640 1 -- 172.21.15.83:0/737943093 learned_addr learned my addr 172.21.15.83:0/737943093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.855+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ab6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 -- 172.21.15.83:0/737943093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 msgr2=0x561282ab6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282ab6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 -- 172.21.15.83:0/737943093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 msgr2=0x561282ab67b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ab67b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 -- 172.21.15.83:0/737943093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128277a650 con 0x5612826ee390 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.856+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ab6cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efea800b9f0 tx=0x7efea800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff4b1f9d3d0bf79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.858+0000 7efea57fa640 1 -- 172.21.15.83:0/737943093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea800de50 con 0x5612826ee390 2024-09-21T03:29:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.858+0000 7efea57fa640 1 -- 172.21.15.83:0/737943093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea80178d0 con 0x5612826ee390 2024-09-21T03:29:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.858+0000 7efea57fa640 1 -- 172.21.15.83:0/737943093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea800f930 con 0x5612826ee390 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 -- 172.21.15.83:0/737943093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 msgr2=0x561282ab6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ab6cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efea800b9f0 tx=0x7efea800beb0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 -- 172.21.15.83:0/737943093 shutdown_connections 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ab6cf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ab67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 --2- 172.21.15.83:0/737943093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282ab6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 -- 172.21.15.83:0/737943093 >> 172.21.15.83:0/737943093 conn(0x561282703460 msgr2=0x561282ab4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 -- 172.21.15.83:0/737943093 shutdown_connections 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.859+0000 7efebba0a740 1 -- 172.21.15.83:0/737943093 wait complete. 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 -- start start 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 Processor -- start 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ac1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 0x561282acddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56128287d700 con 0x56128270f070 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5612829b7110 con 0x5612826e61b0 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561282ac4a50 con 0x5612826ee390 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.860+0000 7efebba0a740 10 client.0 _ll_register_callbacks cb 0x5612829d0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:37620/0 (socket says 172.21.15.83:37620) 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaeffd640 1 -- 172.21.15.83:0/4229232518 learned_addr learned my addr 172.21.15.83:0/4229232518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 0x561282acddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeae7fc640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ac1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 -- 172.21.15.83:0/4229232518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 msgr2=0x561282ac1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ac1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 -- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 msgr2=0x561282ac1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaf7fe640 1 -- 172.21.15.83:0/4229232518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612829e4c40 con 0x56128270f070 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeae7fc640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ac1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.861+0000 7efeaeffd640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 0x561282acddd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efea8019040 tx=0x7efea8017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1880199ea2507cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efe967fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8026020 con 0x56128270f070 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea80178d0 con 0x56128270f070 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efea8017c90 con 0x56128270f070 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.862+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efea8031050 con 0x56128270f070 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 10 client.17541 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561282ad0870 con 0x56128270f070 2024-09-21T03:29:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea80178d0 con 0x56128270f070 2024-09-21T03:29:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7efea8004ab0 con 0x56128270f070 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 10 client.17541 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 20 client.17541 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826cd9f0 0x561282ad3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 4 client.17541 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.863+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561282a2eda0 con 0x5612826cd9f0 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.864+0000 7efeaeffd640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826cd9f0 0x561282ad3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.864+0000 7efeaeffd640 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826cd9f0 0x561282ad3c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7094922211bb5f8d server_cookie=247560c89a753c27 in_seq=0 out_seq=0 2024-09-21T03:29:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.865+0000 7efe967fc640 10 client.17541 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efe967fc640 1 -- 172.21.15.83:0/4229232518 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561282a2eda0 con 0x5612826cd9f0 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efe967fc640 10 client.17541 handle_command_reply: tid=0 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 client.17541 shutdown 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 msgr2=0x561282acddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 0x561282acddd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efea8019040 tx=0x7efea8017650 comp rx=0 tx=0).stop 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 shutdown_connections 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826cd9f0 0x561282ad3c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56128270f070 0x561282acddd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ac1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 --2- 172.21.15.83:0/4229232518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 >> 172.21.15.83:0/4229232518 conn(0x561282703460 msgr2=0x561282ab9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.867+0000 7efe967fc640 0 client.17541 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.868+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 shutdown_connections 2024-09-21T03:29:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.868+0000 7efebba0a740 1 -- 172.21.15.83:0/4229232518 wait complete. 2024-09-21T03:29:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.868+0000 7efebba0a740 20 client.17541 trim_cache size 0 max 16384 2024-09-21T03:29:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.890+0000 7efebba0a740 1 Processor -- start 2024-09-21T03:29:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.890+0000 7efebba0a740 1 -- start start 2024-09-21T03:29:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.890+0000 7efebba0a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.890+0000 7efebba0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efebba0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282abec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efebba0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe9000e6b0 con 0x5612826ee390 2024-09-21T03:29:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efebba0a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe90010220 con 0x5612826e61b0 2024-09-21T03:29:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efebba0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe9000ed80 con 0x56128270f070 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49418/0 (socket says 172.21.15.83:49418) 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeae7fc640 1 -- 172.21.15.83:0/288376346 learned_addr learned my addr 172.21.15.83:0/288376346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282abec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 -- 172.21.15.83:0/288376346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 msgr2=0x561282ac17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 -- 172.21.15.83:0/288376346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 msgr2=0x561282ac1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeaf7fe640 1 -- 172.21.15.83:0/288376346 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561282ab4ad0 con 0x56128270f070 2024-09-21T03:29:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.891+0000 7efeae7fc640 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.892+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282abec00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efea8019040 tx=0x7efea8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c35119a0f59e738 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.892+0000 7efea4ff9640 1 -- 172.21.15.83:0/288376346 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8026040 con 0x56128270f070 2024-09-21T03:29:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.892+0000 7efea4ff9640 1 -- 172.21.15.83:0/288376346 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea803a070 con 0x56128270f070 2024-09-21T03:29:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.892+0000 7efea4ff9640 1 -- 172.21.15.83:0/288376346 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80049c0 con 0x56128270f070 2024-09-21T03:29:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 -- 172.21.15.83:0/288376346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 msgr2=0x561282abec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282abec00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efea8019040 tx=0x7efea8005b50 comp rx=0 tx=0).stop 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 -- 172.21.15.83:0/288376346 shutdown_connections 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56128270f070 0x561282abec00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826ee390 0x561282ac1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 --2- 172.21.15.83:0/288376346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612826e61b0 0x561282ac17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 -- 172.21.15.83:0/288376346 >> 172.21.15.83:0/288376346 conn(0x561282703460 msgr2=0x561282ac0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 -- 172.21.15.83:0/288376346 shutdown_connections 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.893+0000 7efebba0a740 1 -- 172.21.15.83:0/288376346 wait complete. 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 -- start start 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 Processor -- start 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ad48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128270f070 0x561282ad4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efeaeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49434/0 (socket says 172.21.15.83:49434) 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efeaeffd640 1 -- 172.21.15.83:0/594215524 learned_addr learned my addr 172.21.15.83:0/594215524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe9000fb20 con 0x5612826e61b0 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe9000f450 con 0x56128270f070 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe9000c6b0 con 0x5612826ee390 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efeae7fc640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ad48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.894+0000 7efebba0a740 10 client.0 _ll_register_callbacks cb 0x7efe98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeaf7fe640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128270f070 0x561282ad4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 -- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128270f070 msgr2=0x561282ad4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128270f070 0x561282ad4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 -- 172.21.15.83:0/594215524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 msgr2=0x561282ad7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561282ad5500 con 0x5612826ee390 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeaeffd640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efeae7fc640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ad48d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efea00a9250 tx=0x7efea00b65a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b3e8e92c33447c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.895+0000 7efe967fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.896+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00d4020 con 0x5612826ee390 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.896+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea803a070 con 0x5612826ee390 2024-09-21T03:29:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.896+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7efea00b4d40 con 0x5612826ee390 2024-09-21T03:29:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.898+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efea00c6020 con 0x5612826ee390 2024-09-21T03:29:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.898+0000 7efebba0a740 10 client.27064 fetch_fsmap learned FSMap version 49 2024-09-21T03:29:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.898+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561282b3e360 con 0x5612826ee390 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea803a070 con 0x5612826ee390 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7efea00cece0 con 0x5612826ee390 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 10 client.27064 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 20 client.27064 populate_metadata read hostname 'smithi083' 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826a2da0 0x561282b40b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 4 client.27064 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561282a2a850 con 0x5612826a2da0 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.899+0000 7efeaeffd640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826a2da0 0x561282b40b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efeaeffd640 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826a2da0 0x561282b40b80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c06c8de52c0acb0 server_cookie=e01d4669c3820f04 in_seq=0 out_seq=0 2024-09-21T03:29:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efe967fc640 10 client.27064 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efe967fc640 1 -- 172.21.15.83:0/594215524 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x561282a2a850 con 0x5612826a2da0 2024-09-21T03:29:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efe967fc640 10 client.27064 handle_command_reply: tid=0 2024-09-21T03:29:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 client.27064 shutdown 2024-09-21T03:29:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 msgr2=0x561282ad48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ad48d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efea00a9250 tx=0x7efea00b65a0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 shutdown_connections 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5612826a2da0 0x561282b40b80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128270f070 0x561282ad4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efe967fc640 0 client.27064 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612826ee390 0x561282ad48d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 --2- 172.21.15.83:0/594215524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612826e61b0 0x561282ad7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.900+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 >> 172.21.15.83:0/594215524 conn(0x561282703460 msgr2=0x561282ac2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 shutdown_connections 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efebba0a740 1 -- 172.21.15.83:0/594215524 wait complete. 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efebba0a740 20 client.27064 trim_cache size 0 max 16384 2024-09-21T03:29:31.904 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d107bf75-3d29-4357-a930-5487411ac4d2", 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.906 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.907 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.908 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:29:31.910 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.911 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.912 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:29:31.913 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:29:31.914 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 msgr2=0x7efe9c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 0x7efe9c0b8cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe98009230 tx=0x7efe98000bc0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 msgr2=0x7efeb01167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efea000b9b0 tx=0x7efea000be70 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 shutdown_connections 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7efe9c0b67b0 0x7efe9c0b8cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb011cc30 0x7efeb0116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeb0111f40 0x7efeb01167b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 --2- 172.21.15.83:0/2508768432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb010a9e0 0x7efeb011c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.901+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 >> 172.21.15.83:0/2508768432 conn(0x7efeb0070ec0 msgr2=0x7efeb010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.902+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 shutdown_connections 2024-09-21T03:29:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:29:31.902+0000 7efeb5496640 1 -- 172.21.15.83:0/2508768432 wait complete. 2024-09-21T03:29:32.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:32 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:32.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:32 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:32.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:32 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:32.721 INFO:tasks.cephfs.filesystem:scrub status for tag:d107bf75-3d29-4357-a930-5487411ac4d2 - {'path': '/', 'tag': 'd107bf75-3d29-4357-a930-5487411ac4d2', 'options': 'recursive,force'} 2024-09-21T03:29:33.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:33 smithi083 ceph-mon[31219]: pgmap v725: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-21T03:29:33.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:33 smithi142 ceph-mon[34512]: pgmap v725: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-21T03:29:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:33 smithi193 ceph-mon[35224]: pgmap v725: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-21T03:29:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:35 smithi083 ceph-mon[31219]: pgmap v726: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.9 MiB/s wr, 990 op/s 2024-09-21T03:29:35.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:35 smithi142 ceph-mon[34512]: pgmap v726: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.9 MiB/s wr, 990 op/s 2024-09-21T03:29:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:35 smithi193 ceph-mon[35224]: pgmap v726: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.9 MiB/s wr, 990 op/s 2024-09-21T03:29:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 735.0 s, 1558.2 tps, lat 64.714 ms stddev 74.150 2024-09-21T03:29:37.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:37 smithi083 ceph-mon[31219]: pgmap v727: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-09-21T03:29:37.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:37 smithi142 ceph-mon[34512]: pgmap v727: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-09-21T03:29:37.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:37 smithi193 ceph-mon[35224]: pgmap v727: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-09-21T03:29:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:39 smithi083 ceph-mon[31219]: pgmap v728: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-09-21T03:29:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:29:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:29:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:29:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:39 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:29:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:39 smithi142 ceph-mon[34512]: pgmap v728: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-09-21T03:29:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:29:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:29:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:29:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:39 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:29:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:39 smithi193 ceph-mon[35224]: pgmap v728: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-09-21T03:29:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:29:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:29:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:29:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:39 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:29:40.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:40 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:29:40.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:40 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:40.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:40 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:29:40.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:40 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:40 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:29:40.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:40 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:29:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 740.0 s, 1505.4 tps, lat 65.911 ms stddev 76.660 2024-09-21T03:29:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:41 smithi083 ceph-mon[31219]: pgmap v729: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-09-21T03:29:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:41 smithi142 ceph-mon[34512]: pgmap v729: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-09-21T03:29:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:41 smithi193 ceph-mon[35224]: pgmap v729: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-09-21T03:29:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:43 smithi142 ceph-mon[34512]: pgmap v730: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 696 op/s 2024-09-21T03:29:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:43 smithi193 ceph-mon[35224]: pgmap v730: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 696 op/s 2024-09-21T03:29:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:43 smithi083 ceph-mon[31219]: pgmap v730: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.9 MiB/s wr, 696 op/s 2024-09-21T03:29:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:45 smithi142 ceph-mon[34512]: pgmap v731: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-09-21T03:29:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:45 smithi193 ceph-mon[35224]: pgmap v731: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-09-21T03:29:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:45 smithi083 ceph-mon[31219]: pgmap v731: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 694 op/s 2024-09-21T03:29:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 745.0 s, 1901.8 tps, lat 52.479 ms stddev 60.977 2024-09-21T03:29:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:47 smithi142 ceph-mon[34512]: pgmap v732: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.5 MiB/s wr, 714 op/s 2024-09-21T03:29:47.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:47 smithi193 ceph-mon[35224]: pgmap v732: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.5 MiB/s wr, 714 op/s 2024-09-21T03:29:47.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:47 smithi083 ceph-mon[31219]: pgmap v732: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.5 MiB/s wr, 714 op/s 2024-09-21T03:29:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:49 smithi142 ceph-mon[34512]: pgmap v733: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.5 MiB/s wr, 659 op/s 2024-09-21T03:29:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:49 smithi193 ceph-mon[35224]: pgmap v733: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.5 MiB/s wr, 659 op/s 2024-09-21T03:29:49.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:49 smithi083 ceph-mon[31219]: pgmap v733: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.5 MiB/s wr, 659 op/s 2024-09-21T03:29:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 750.0 s, 1586.6 tps, lat 63.435 ms stddev 68.510 2024-09-21T03:29:51.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:51 smithi142 ceph-mon[34512]: pgmap v734: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-09-21T03:29:51.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:51 smithi193 ceph-mon[35224]: pgmap v734: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-09-21T03:29:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:51 smithi083 ceph-mon[31219]: pgmap v734: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-09-21T03:29:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:53 smithi142 ceph-mon[34512]: pgmap v735: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 718 op/s 2024-09-21T03:29:53.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:53 smithi193 ceph-mon[35224]: pgmap v735: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 718 op/s 2024-09-21T03:29:53.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:53 smithi083 ceph-mon[31219]: pgmap v735: 97 pgs: 97 active+clean; 4.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 718 op/s 2024-09-21T03:29:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:55 smithi142 ceph-mon[34512]: pgmap v736: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T03:29:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:55 smithi193 ceph-mon[35224]: pgmap v736: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T03:29:55.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:55 smithi083 ceph-mon[31219]: pgmap v736: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T03:29:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 755.0 s, 1535.8 tps, lat 65.291 ms stddev 76.515 2024-09-21T03:29:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:57 smithi142 ceph-mon[34512]: pgmap v737: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:29:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:57 smithi193 ceph-mon[35224]: pgmap v737: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:29:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:57 smithi083 ceph-mon[31219]: pgmap v737: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-09-21T03:29:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:29:59 smithi142 ceph-mon[34512]: pgmap v738: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T03:29:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:29:59 smithi193 ceph-mon[35224]: pgmap v738: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T03:29:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:29:59 smithi083 ceph-mon[31219]: pgmap v738: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-09-21T03:30:00.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:00 smithi142 ceph-mon[34512]: overall HEALTH_OK 2024-09-21T03:30:00.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:00 smithi193 ceph-mon[35224]: overall HEALTH_OK 2024-09-21T03:30:00.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:00 smithi083 ceph-mon[31219]: overall HEALTH_OK 2024-09-21T03:30:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 760.0 s, 1364.2 tps, lat 71.525 ms stddev 86.103 2024-09-21T03:30:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:01 smithi142 ceph-mon[34512]: pgmap v739: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 626 op/s 2024-09-21T03:30:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:01 smithi193 ceph-mon[35224]: pgmap v739: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 626 op/s 2024-09-21T03:30:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:01 smithi083 ceph-mon[31219]: pgmap v739: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 626 op/s 2024-09-21T03:30:02.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:03 smithi142 ceph-mon[34512]: pgmap v740: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 623 op/s 2024-09-21T03:30:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:03 smithi193 ceph-mon[35224]: pgmap v740: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 623 op/s 2024-09-21T03:30:03.596 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:03 smithi083 ceph-mon[31219]: pgmap v740: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 623 op/s 2024-09-21T03:30:04.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.061+0000 7fe8cd95e640 1 -- 172.21.15.83:0/2446854605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8111f40 msgr2=0x7fe8c810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.061+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/2446854605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8111f40 0x7fe8c810a3c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe8b801acb0 tx=0x7fe8b803b8e0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.063+0000 7fe8cd95e640 1 -- 172.21.15.83:0/2446854605 shutdown_connections 2024-09-21T03:30:04.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.063+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/2446854605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c811cc30 0x7fe8c811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.063+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/2446854605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8c810a9e0 0x7fe8c810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.063+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/2446854605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8111f40 0x7fe8c810a3c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.063+0000 7fe8cd95e640 1 -- 172.21.15.83:0/2446854605 >> 172.21.15.83:0/2446854605 conn(0x7fe8c8070ec0 msgr2=0x7fe8c80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.068+0000 7fe8cd95e640 1 -- 172.21.15.83:0/2446854605 shutdown_connections 2024-09-21T03:30:04.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.068+0000 7fe8cd95e640 1 -- 172.21.15.83:0/2446854605 wait complete. 2024-09-21T03:30:04.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 Processor -- start 2024-09-21T03:30:04.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 -- start start 2024-09-21T03:30:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8c810a9e0 0x7fe8c811c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 0x7fe8c8118540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8114c10 con 0x7fe8c811cc30 2024-09-21T03:30:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8114a90 con 0x7fe8c8111f40 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.069+0000 7fe8cd95e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8114d90 con 0x7fe8c810a9e0 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8be7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 0x7fe8c8118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49148/0 (socket says 172.21.15.83:49148) 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 -- 172.21.15.83:0/1330092412 learned_addr learned my addr 172.21.15.83:0/1330092412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 -- 172.21.15.83:0/1330092412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8c810a9e0 msgr2=0x7fe8c811c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8c810a9e0 0x7fe8c811c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 -- 172.21.15.83:0/1330092412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 msgr2=0x7fe8c8118540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 0x7fe8c8118540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8c77fe640 1 -- 172.21.15.83:0/1330092412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8119120 con 0x7fe8c811cc30 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.070+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 0x7fe8c8118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.071+0000 7fe8c77fe640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c970 tx=0x7fe8c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.076+0000 7fe8cc95c640 1 -- 172.21.15.83:0/1330092412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0007bb0 con 0x7fe8c811cc30 2024-09-21T03:30:04.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.076+0000 7fe8cc95c640 1 -- 172.21.15.83:0/1330092412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c0007d50 con 0x7fe8c811cc30 2024-09-21T03:30:04.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.076+0000 7fe8cc95c640 1 -- 172.21.15.83:0/1330092412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004e60 con 0x7fe8c811cc30 2024-09-21T03:30:04.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.078+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81acdd0 con 0x7fe8c811cc30 2024-09-21T03:30:04.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.078+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81ad170 con 0x7fe8c811cc30 2024-09-21T03:30:04.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.081+0000 7fe8d3ed2740 1 Processor -- start 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 -- start start 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x56509141b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 0x56509141c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565091391e50 con 0x56509104b1b0 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565091381280 con 0x565091053390 2024-09-21T03:30:04.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.082+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565091385090 con 0x565091074070 2024-09-21T03:30:04.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.087+0000 7fe8c6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.087+0000 7fe8c6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49164/0 (socket says 172.21.15.83:49164) 2024-09-21T03:30:04.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.087+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/1229689423 learned_addr learned my addr 172.21.15.83:0/1229689423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:04.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.088+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 0x56509141c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.088+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x56509141b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.089+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1229689423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 msgr2=0x56509141b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.089+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x56509141b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.089+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1229689423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 msgr2=0x56509141bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.089+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.089+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1229689423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650910df650 con 0x565091053390 2024-09-21T03:30:04.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.090+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 0x56509141c0d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000afd0 tx=0x7fe8b000a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a965bc6106d0d1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.092+0000 7fe8cc95c640 1 -- 172.21.15.83:0/1330092412 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe8c0005000 con 0x7fe8c811cc30 2024-09-21T03:30:04.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.096+0000 7fe8cc95c640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 0x7fe8b40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.096+0000 7fe8cc95c640 1 -- 172.21.15.83:0/1330092412 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fe8c00e76a0 con 0x7fe8c811cc30 2024-09-21T03:30:04.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.096+0000 7fe8bdffb640 1 -- 172.21.15.83:0/1229689423 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0011e50 con 0x565091053390 2024-09-21T03:30:04.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.096+0000 7fe8bdffb640 1 -- 172.21.15.83:0/1229689423 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b0014110 con 0x565091053390 2024-09-21T03:30:04.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.096+0000 7fe8bdffb640 1 -- 172.21.15.83:0/1229689423 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0003bc0 con 0x565091053390 2024-09-21T03:30:04.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.098+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1229689423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 msgr2=0x56509141c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.098+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 0x56509141c0d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000afd0 tx=0x7fe8b000a4d0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.098+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 0x7fe8b40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.103+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 0x7fe8b40b8d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56509141a2d0 tx=0x7fe8b801a550 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.104+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1229689423 shutdown_connections 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.104+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565091053390 0x56509141c0d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.104+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.104+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1229689423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x56509141b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.104+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1229689423 >> 172.21.15.83:0/1229689423 conn(0x565091068460 msgr2=0x565091419540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.111+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1229689423 shutdown_connections 2024-09-21T03:30:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.112+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1229689423 wait complete. 2024-09-21T03:30:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.113+0000 7fe8d3ed2740 1 -- start start 2024-09-21T03:30:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.113+0000 7fe8d3ed2740 1 Processor -- start 2024-09-21T03:30:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.114+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x565091426d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.115+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565091053390 0x5650914272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.115+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x565091433150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.115+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5650911e2700 con 0x565091053390 2024-09-21T03:30:04.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.115+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56509131c110 con 0x56509104b1b0 2024-09-21T03:30:04.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.115+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565091429e30 con 0x565091074070 2024-09-21T03:30:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.116+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x565091433150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.116+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x565091433150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47122/0 (socket says 172.21.15.83:47122) 2024-09-21T03:30:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.116+0000 7fe8c77fe640 1 -- 172.21.15.83:0/760383078 learned_addr learned my addr 172.21.15.83:0/760383078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.116+0000 7fe8d3ed2740 10 client.0 _ll_register_callbacks cb 0x565091335990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.117+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x565091426d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.117+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565091053390 0x5650914272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.121+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/760383078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 msgr2=0x565091433150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.121+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x565091433150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.121+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/760383078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565091053390 msgr2=0x5650914272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.121+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565091053390 0x5650914272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.121+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/760383078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565091349c40 con 0x56509104b1b0 2024-09-21T03:30:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.122+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x565091426d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8042730 tx=0x7fe8b8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1519c9d94bbefcab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.122+0000 7fe8a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:30:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.125+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b805d070 con 0x56509104b1b0 2024-09-21T03:30:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.125+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fe8b80415e0 con 0x56509104b1b0 2024-09-21T03:30:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.126+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b0014110 con 0x56509104b1b0 2024-09-21T03:30:04.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.130+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe8b8067050 con 0x56509104b1b0 2024-09-21T03:30:04.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.131+0000 7fe8d3ed2740 10 client.27110 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:04.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.131+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x565091435ba0 con 0x56509104b1b0 2024-09-21T03:30:04.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.131+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8047040 con 0x56509104b1b0 2024-09-21T03:30:04.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fe8b8059780 con 0x56509104b1b0 2024-09-21T03:30:04.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 10 client.27110 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:04.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:04.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 20 client.27110 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:04.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5650910329f0 0x5650914390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 4 client.27110 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:04.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.132+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe8b0014110 con 0x5650910329f0 2024-09-21T03:30:04.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.134+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5650910329f0 0x5650914390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.136+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5650910329f0 0x5650914390c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde8f100b8ac4f58 server_cookie=6d3d5695f70b401 in_seq=0 out_seq=0 2024-09-21T03:30:04.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.136+0000 7fe8a6ffd640 10 client.27110 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:04.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.139+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/760383078 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe8b0014110 con 0x5650910329f0 2024-09-21T03:30:04.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.139+0000 7fe8a6ffd640 10 client.27110 handle_command_reply: tid=0 2024-09-21T03:30:04.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.139+0000 7fe8d3ed2740 1 client.27110 shutdown 2024-09-21T03:30:04.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.145+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 msgr2=0x565091426d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.145+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x565091426d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8042730 tx=0x7fe8b8000f40 comp rx=0 tx=0).stop 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.146+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 shutdown_connections 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.146+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5650910329f0 0x5650914390c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091074070 0x565091433150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565091053390 0x5650914272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/760383078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x565091426d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8a6ffd640 0 client.27110 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 >> 172.21.15.83:0/760383078 conn(0x565091068460 msgr2=0x56509141e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 shutdown_connections 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/760383078 wait complete. 2024-09-21T03:30:04.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.147+0000 7fe8d3ed2740 20 client.27110 trim_cache size 0 max 16384 2024-09-21T03:30:04.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.184+0000 7fe8d3ed2740 1 Processor -- start 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.184+0000 7fe8d3ed2740 1 -- start start 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.185+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.185+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.185+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x565091418b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.185+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b5ac0 con 0x56509104b1b0 2024-09-21T03:30:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.186+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b7600 con 0x5650910329f0 2024-09-21T03:30:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.186+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b6190 con 0x565091053390 2024-09-21T03:30:04.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.187+0000 7fe8c6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.187+0000 7fe8c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x565091418b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.187+0000 7fe8c6ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59176/0 (socket says 172.21.15.83:59176) 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.187+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/2336006650 learned_addr learned my addr 172.21.15.83:0/2336006650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.187+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.188+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/2336006650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 msgr2=0x565091418b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.188+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x565091418b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.188+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/2336006650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 msgr2=0x56509141cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.188+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.188+0000 7fe8c6ffd640 1 -- 172.21.15.83:0/2336006650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565091421980 con 0x5650910329f0 2024-09-21T03:30:04.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.191+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8059b50 tx=0x7fe8b8042d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20405ac0414721a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.195+0000 7fe8bd7fa640 1 -- 172.21.15.83:0/2336006650 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b805d070 con 0x5650910329f0 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.195+0000 7fe8bd7fa640 1 -- 172.21.15.83:0/2336006650 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8047030 con 0x5650910329f0 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.195+0000 7fe8bd7fa640 1 -- 172.21.15.83:0/2336006650 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80415e0 con 0x5650910329f0 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.196+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/2336006650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 msgr2=0x565091425c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.196+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8059b50 tx=0x7fe8b8042d60 comp rx=0 tx=0).stop 2024-09-21T03:30:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.197+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/2336006650 shutdown_connections 2024-09-21T03:30:04.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.197+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x565091418b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.197+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509104b1b0 0x56509141cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.197+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/2336006650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650910329f0 0x565091425c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.197+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/2336006650 >> 172.21.15.83:0/2336006650 conn(0x565091068460 msgr2=0x56509141c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.201+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/2336006650 shutdown_connections 2024-09-21T03:30:04.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.201+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/2336006650 wait complete. 2024-09-21T03:30:04.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.202+0000 7fe8d3ed2740 1 -- start start 2024-09-21T03:30:04.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.202+0000 7fe8d3ed2740 1 Processor -- start 2024-09-21T03:30:04.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650910329f0 0x56509142b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x56509142bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b6f30 con 0x5650910329f0 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8be7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8be7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:59186/0 (socket says 172.21.15.83:59186) 2024-09-21T03:30:04.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1861733032 learned_addr learned my addr 172.21.15.83:0/1861733032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b6860 con 0x56509104b1b0 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8c77fe640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x56509142bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.203+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81b3ac0 con 0x565091053390 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8d3ed2740 10 client.0 _ll_register_callbacks cb 0x7fe8b8050070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1861733032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 msgr2=0x56509142bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x56509142bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1861733032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650910329f0 msgr2=0x56509142b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650910329f0 0x56509142b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650914a2a80 con 0x56509104b1b0 2024-09-21T03:30:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.204+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00148f0 tx=0x7fe8b000aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a871f6de0247b025 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.207+0000 7fe8a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:30:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.207+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0014110 con 0x56509104b1b0 2024-09-21T03:30:04.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.207+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7fe8b0014340 con 0x56509104b1b0 2024-09-21T03:30:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.207+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b8047030 con 0x56509104b1b0 2024-09-21T03:30:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.208+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe8b0038050 con 0x56509104b1b0 2024-09-21T03:30:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.209+0000 7fe8d3ed2740 10 client.27116 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.209+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5650914a3cf0 con 0x56509104b1b0 2024-09-21T03:30:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8047030 con 0x56509104b1b0 2024-09-21T03:30:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fe8b003b020 con 0x56509104b1b0 2024-09-21T03:30:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 10 client.27116 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 10 client.27116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 20 client.27116 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x565091074070 0x5650914a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:04.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 4 client.27116 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:04.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.214+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565091393da0 con 0x565091074070 2024-09-21T03:30:04.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.215+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x565091074070 0x5650914a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:04.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.219+0000 7fe8c6ffd640 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x565091074070 0x5650914a6190 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44066a29722f38df server_cookie=c085f489b2de6237 in_seq=0 out_seq=0 2024-09-21T03:30:04.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.219+0000 7fe8a6ffd640 10 client.27116 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:04.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.221+0000 7fe8a6ffd640 1 -- 172.21.15.83:0/1861733032 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x565091393da0 con 0x565091074070 2024-09-21T03:30:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.221+0000 7fe8a6ffd640 10 client.27116 handle_command_reply: tid=0 2024-09-21T03:30:04.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.222+0000 7fe8d3ed2740 1 client.27116 shutdown 2024-09-21T03:30:04.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 msgr2=0x56509142b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00148f0 tx=0x7fe8b000aa90 comp rx=0 tx=0).stop 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 shutdown_connections 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x565091074070 0x5650914a6190 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565091053390 0x56509142bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56509104b1b0 0x56509142b5c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 --2- 172.21.15.83:0/1861733032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5650910329f0 0x56509142b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 >> 172.21.15.83:0/1861733032 conn(0x565091068460 msgr2=0x56509141c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8a6ffd640 0 client.27116 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 shutdown_connections 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 1 -- 172.21.15.83:0/1861733032 wait complete. 2024-09-21T03:30:04.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.229+0000 7fe8d3ed2740 20 client.27116 trim_cache size 0 max 16384 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:30:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:30:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:30:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:30:04.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 msgr2=0x7fe8b40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 0x7fe8b40b8d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56509141a2d0 tx=0x7fe8b801a550 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 msgr2=0x7fe8c8118a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c970 tx=0x7fe8c000ce30 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 shutdown_connections 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.233+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe8b40b6880 0x7fe8b40b8d90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c811cc30 0x7fe8c8118a80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8c8111f40 0x7fe8c8118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 --2- 172.21.15.83:0/1330092412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8c810a9e0 0x7fe8c811c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 >> 172.21.15.83:0/1330092412 conn(0x7fe8c8070ec0 msgr2=0x7fe8c810ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 shutdown_connections 2024-09-21T03:30:04.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:04.234+0000 7fe8cd95e640 1 -- 172.21.15.83:0/1330092412 wait complete. 2024-09-21T03:30:05.126 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:30:05.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:05.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:05 smithi083 ceph-mon[31219]: pgmap v741: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T03:30:05.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:05 smithi193 ceph-mon[35224]: pgmap v741: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T03:30:05.598 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:05 smithi142 ceph-mon[34512]: pgmap v741: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 616 op/s 2024-09-21T03:30:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 765.0 s, 224.2 tps, lat 394.921 ms stddev 417.408 2024-09-21T03:30:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 -- 172.21.15.83:0/126167719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108cf0 msgr2=0x7f5b6810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/126167719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108cf0 0x7f5b6810f520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009ca0 tx=0x7f5b6002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 -- 172.21.15.83:0/126167719 shutdown_connections 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/126167719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108cf0 0x7f5b6810f520 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/126167719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b681083a0 0x7f5b681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/126167719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68102a60 0x7f5b68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 -- 172.21.15.83:0/126167719 >> 172.21.15.83:0/126167719 conn(0x7f5b68079730 msgr2=0x7f5b68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 -- 172.21.15.83:0/126167719 shutdown_connections 2024-09-21T03:30:06.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.246+0000 7f5b6eeee640 1 -- 172.21.15.83:0/126167719 wait complete. 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 Processor -- start 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 -- start start 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b68102a60 0x7f5b6819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68108cf0 0x7f5b6819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112a30 con 0x7f5b681083a0 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681128b0 con 0x7f5b68102a60 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.247+0000 7f5b6eeee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112bb0 con 0x7f5b68108cf0 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49212/0 (socket says 172.21.15.83:49212) 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 -- 172.21.15.83:0/2502161009 learned_addr learned my addr 172.21.15.83:0/2502161009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:06.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b6d48b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68108cf0 0x7f5b6819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b6cc8a640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b68102a60 0x7f5b6819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 -- 172.21.15.83:0/2502161009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68108cf0 msgr2=0x7f5b6819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68108cf0 0x7f5b6819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 -- 172.21.15.83:0/2502161009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b68102a60 msgr2=0x7f5b6819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b68102a60 0x7f5b6819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 -- 172.21.15.83:0/2502161009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b681a6570 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b67fff640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9c0 tx=0x7f5b5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400cd80 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b540054d0 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.248+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681a6860 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.249+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54010430 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.249+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a6e70 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.250+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b68102e70 con 0x7f5b681083a0 2024-09-21T03:30:06.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.251+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5b54004f90 con 0x7f5b681083a0 2024-09-21T03:30:06.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.252+0000 7f5b65ffb640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 0x7f5b580b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:06.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.252+0000 7f5b6cc8a640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 0x7f5b580b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:06.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.252+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f5b540e7a70 con 0x7f5b681083a0 2024-09-21T03:30:06.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.252+0000 7f5b6cc8a640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 0x7f5b580b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806b9c0 tx=0x7f5b5c0090e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.256+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b540ae010 con 0x7f5b681083a0 2024-09-21T03:30:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:06.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:30:06.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.502+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b6802e640 con 0x7f5b681083a0 2024-09-21T03:30:06.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.503+0000 7f5b65ffb640 1 -- 172.21.15.83:0/2502161009 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11989 (secure 0 0 0) 0x7f5b540b23f0 con 0x7f5b681083a0 2024-09-21T03:30:06.506 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:30:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":50,"btime":"2024-09-21T03:29:39:102993+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:29:38.263816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:30:06.506 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 50 2024-09-21T03:30:06.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.507+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 msgr2=0x7f5b580b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.507+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 0x7f5b580b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806b9c0 tx=0x7f5b5c0090e0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.507+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 msgr2=0x7f5b6819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:06.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.507+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9c0 tx=0x7f5b5400ee80 comp rx=0 tx=0).stop 2024-09-21T03:30:06.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 shutdown_connections 2024-09-21T03:30:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b580b6730 0x7f5b580b8c40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b68108cf0 0x7f5b6819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b681083a0 0x7f5b6819b820 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 --2- 172.21.15.83:0/2502161009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b68102a60 0x7f5b6819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.516+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 >> 172.21.15.83:0/2502161009 conn(0x7f5b68079730 msgr2=0x7f5b68104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:06.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.520+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 shutdown_connections 2024-09-21T03:30:06.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:06.522+0000 7f5b6eeee640 1 -- 172.21.15.83:0/2502161009 wait complete. 2024-09-21T03:30:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:06.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:30:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:06.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:06.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:06.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:30:07.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:30:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:07 smithi193 ceph-mon[35224]: pgmap v742: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-21T03:30:07.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:07 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2502161009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:07.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:07 smithi083 ceph-mon[31219]: pgmap v742: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-21T03:30:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:07 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2502161009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:07 smithi142 ceph-mon[34512]: pgmap v742: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-21T03:30:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:07 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2502161009' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 -- 172.21.15.83:0/1415990579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 msgr2=0x7f3f7410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 --2- 172.21.15.83:0/1415990579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7410f540 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009ca0 tx=0x7f3f6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 -- 172.21.15.83:0/1415990579 shutdown_connections 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 --2- 172.21.15.83:0/1415990579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7410f540 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 --2- 172.21.15.83:0/1415990579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f741087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 --2- 172.21.15.83:0/1415990579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f74102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 -- 172.21.15.83:0/1415990579 >> 172.21.15.83:0/1415990579 conn(0x7f3f74079740 msgr2=0x7f3f74079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 -- 172.21.15.83:0/1415990579 shutdown_connections 2024-09-21T03:30:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.223+0000 7f3f790b4640 1 -- 172.21.15.83:0/1415990579 wait complete. 2024-09-21T03:30:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.224+0000 7f3f790b4640 1 Processor -- start 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.224+0000 7f3f790b4640 1 -- start start 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.224+0000 7f3f790b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f7419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f790b4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f7419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f790b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f790b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74112980 con 0x7f3f74108d10 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f790b4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74112800 con 0x7f3f74102a80 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f790b4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74112b00 con 0x7f3f741083c0 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49228/0 (socket says 172.21.15.83:49228) 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 -- 172.21.15.83:0/3014231001 learned_addr learned my addr 172.21.15.83:0/3014231001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f7419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f7419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 -- 172.21.15.83:0/3014231001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 msgr2=0x7f3f7419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f7419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 -- 172.21.15.83:0/3014231001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 msgr2=0x7f3f7419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f7419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 -- 172.21.15.83:0/3014231001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f741a0390 con 0x7f3f74108d10 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f7419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f7419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f7359e640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0098d0 tx=0x7f3f6c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.225+0000 7f3f70d99640 1 -- 172.21.15.83:0/3014231001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c003b70 con 0x7f3f74108d10 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.226+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f741a0620 con 0x7f3f74108d10 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.226+0000 7f3f70d99640 1 -- 172.21.15.83:0/3014231001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f6c003d10 con 0x7f3f74108d10 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.226+0000 7f3f70d99640 1 -- 172.21.15.83:0/3014231001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c02fa20 con 0x7f3f74108d10 2024-09-21T03:30:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.226+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f741a6de0 con 0x7f3f74108d10 2024-09-21T03:30:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.228+0000 7f3f7f628740 1 Processor -- start 2024-09-21T03:30:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.228+0000 7f3f70d99640 1 -- 172.21.15.83:0/3014231001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3f6c02fc40 con 0x7f3f74108d10 2024-09-21T03:30:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.228+0000 7f3f7f628740 1 -- start start 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea188c390 0x559ea1c54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ea1bcaea0 con 0x559ea18ad070 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ea1bba2d0 con 0x559ea188c390 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7f628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ea1bbe0e0 con 0x559ea18841b0 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49242/0 (socket says 172.21.15.83:49242) 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1747464735 learned_addr learned my addr 172.21.15.83:0/1747464735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea188c390 0x559ea1c54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f70d99640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 0x7f3f600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1747464735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 msgr2=0x559ea1c54800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c54800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1747464735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea188c390 msgr2=0x559ea1c54d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea188c390 0x559ea1c54d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1747464735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea1918650 con 0x559ea18ad070 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.229+0000 7f3f7359e640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 0x7f3f600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.230+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0135f0 tx=0x7f3f5c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ca40d8a1a6311e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.230+0000 7f3f70d99640 1 -- 172.21.15.83:0/3014231001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f3f6c108a20 con 0x7f3f74108d10 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.230+0000 7f3f6959a640 1 -- 172.21.15.83:0/1747464735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c011470 con 0x559ea18ad070 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.230+0000 7f3f6959a640 1 -- 172.21.15.83:0/1747464735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c00f950 con 0x559ea18ad070 2024-09-21T03:30:08.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.230+0000 7f3f6959a640 1 -- 172.21.15.83:0/1747464735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c010630 con 0x559ea18ad070 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- 172.21.15.83:0/1747464735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 msgr2=0x559ea1c542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0135f0 tx=0x7f3f5c0097a0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7359e640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 0x7f3f600b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c1067c0 tx=0x7f3f6c03e070 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- 172.21.15.83:0/1747464735 shutdown_connections 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea188c390 0x559ea1c54d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c54800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1747464735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c542c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- 172.21.15.83:0/1747464735 >> 172.21.15.83:0/1747464735 conn(0x559ea18a1460 msgr2=0x559ea1c521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- 172.21.15.83:0/1747464735 shutdown_connections 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- 172.21.15.83:0/1747464735 wait complete. 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 -- start start 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.231+0000 7f3f7f628740 1 Processor -- start 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea18841b0 0x559ea1c5f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c5fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ea1a1b700 con 0x559ea18ad070 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ea1b55160 con 0x559ea18841b0 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ea1c62aa0 con 0x559ea188c390 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f7f628740 10 client.0 _ll_register_callbacks cb 0x559ea1b6e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea18841b0 0x559ea1c5f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49244/0 (socket says 172.21.15.83:49244) 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1294766676 learned_addr learned my addr 172.21.15.83:0/1294766676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c5fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 msgr2=0x559ea1c5fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c5fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1294766676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea18841b0 msgr2=0x559ea1c5f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea18841b0 0x559ea1c5f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 -- 172.21.15.83:0/1294766676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea1b82c90 con 0x559ea18ad070 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00f3b0 tx=0x7f3f5c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b49308996ff6601 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.232+0000 7f3f5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c026030 con 0x559ea18ad070 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f3f5c0052c0 con 0x559ea18ad070 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f5c00f950 con 0x559ea18ad070 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f5c029040 con 0x559ea18ad070 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 10 client.17580 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:08.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559ea1c6e7e0 con 0x559ea18ad070 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c00f950 con 0x559ea18ad070 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f3f5c02d030 con 0x559ea18ad070 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 10 client.17580 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 10 client.17580 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 20 client.17580 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea186b9f0 0x559ea1c71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 4 client.17580 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea1bccdf0 con 0x559ea186b9f0 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.233+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea186b9f0 0x559ea1c71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.234+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea186b9f0 0x559ea1c71bf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15cced5daab40f30 server_cookie=601c5faacd29919d in_seq=0 out_seq=0 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.234+0000 7f3f5a7fc640 10 client.17580 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.234+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/1294766676 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ea1bccdf0 con 0x559ea186b9f0 2024-09-21T03:30:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.234+0000 7f3f5a7fc640 10 client.17580 handle_command_reply: tid=0 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 client.17580 shutdown 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 msgr2=0x559ea1c6bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00f3b0 tx=0x7f3f5c005970 comp rx=0 tx=0).stop 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 shutdown_connections 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea186b9f0 0x559ea1c71bf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18ad070 0x559ea1c6bdc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f5a7fc640 0 client.17580 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c5fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 --2- 172.21.15.83:0/1294766676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea18841b0 0x559ea1c5f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 >> 172.21.15.83:0/1294766676 conn(0x559ea18a1460 msgr2=0x559ea1c574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 shutdown_connections 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 1 -- 172.21.15.83:0/1294766676 wait complete. 2024-09-21T03:30:08.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.235+0000 7f3f7f628740 20 client.17580 trim_cache size 0 max 16384 2024-09-21T03:30:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.248+0000 7f3f7f628740 1 Processor -- start 2024-09-21T03:30:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.248+0000 7f3f7f628740 1 -- start start 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c5fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741ad360 con 0x559ea188c390 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741aeea0 con 0x559ea186b9f0 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f7f628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741ada30 con 0x559ea18841b0 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49258/0 (socket says 172.21.15.83:49258) 2024-09-21T03:30:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 -- 172.21.15.83:0/160446350 learned_addr learned my addr 172.21.15.83:0/160446350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c5fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 -- 172.21.15.83:0/160446350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 msgr2=0x559ea1c6bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c6bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 -- 172.21.15.83:0/160446350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 msgr2=0x559ea1c5fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c5fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 -- 172.21.15.83:0/160446350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea1c60c60 con 0x559ea188c390 2024-09-21T03:30:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.249+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0135f0 tx=0x7f3f5c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24808cdc5576e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.250+0000 7f3f68d99640 1 -- 172.21.15.83:0/160446350 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c026070 con 0x559ea188c390 2024-09-21T03:30:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.250+0000 7f3f68d99640 1 -- 172.21.15.83:0/160446350 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c005250 con 0x559ea188c390 2024-09-21T03:30:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.250+0000 7f3f68d99640 1 -- 172.21.15.83:0/160446350 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00f950 con 0x559ea188c390 2024-09-21T03:30:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.250+0000 7f3f7f628740 1 -- 172.21.15.83:0/160446350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 msgr2=0x559ea1c5cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0135f0 tx=0x7f3f5c022040 comp rx=0 tx=0).stop 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 -- 172.21.15.83:0/160446350 shutdown_connections 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea188c390 0x559ea1c5cb90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea18841b0 0x559ea1c6bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- 172.21.15.83:0/160446350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c5fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 -- 172.21.15.83:0/160446350 >> 172.21.15.83:0/160446350 conn(0x559ea18a1460 msgr2=0x559ea1c5f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 -- 172.21.15.83:0/160446350 shutdown_connections 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 -- 172.21.15.83:0/160446350 wait complete. 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 -- start start 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 Processor -- start 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.251+0000 7f3f7f628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c63d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f7f628740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741ae7d0 con 0x559ea18841b0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f7f628740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741ae100 con 0x559ea186b9f0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f7f628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741ab360 con 0x559ea188c390 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f7f628740 10 client.0 _ll_register_callbacks cb 0x7f3f5c013420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49268/0 (socket says 172.21.15.83:49268) 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 -- 172.21.15.83:0/973100309 learned_addr learned my addr 172.21.15.83:0/973100309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c63d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 -- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 msgr2=0x559ea1c63d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c63d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 -- 172.21.15.83:0/973100309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 msgr2=0x559ea1c68320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c68320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 -- 172.21.15.83:0/973100309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea1c642d0 con 0x559ea18841b0 2024-09-21T03:30:08.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c68320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f72d9d640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3f64013d30 tx=0x7f3f64013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6398582d3f2a3269 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64029030 con 0x559ea18841b0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8441+0+0 (secure 0 0 0) 0x7f3f64004020 con 0x559ea18841b0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.252+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f5c005250 con 0x559ea18841b0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f64033050 con 0x559ea18841b0 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 10 client.17586 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:08.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559ea1cdc1d0 con 0x559ea18841b0 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c005250 con 0x559ea18841b0 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f3f64003710 con 0x559ea18841b0 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 10 client.17586 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 10 client.17586 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 20 client.17586 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea18ad070 0x559ea1cdea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 4 client.17586 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ea1bc88a0 con 0x559ea18ad070 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.253+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea18ad070 0x559ea1cdea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f5a7fc640 10 client.17586 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f6a59c640 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea18ad070 0x559ea1cdea40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a4872e62bbf93ee server_cookie=8e1ea850c5003617 in_seq=0 out_seq=0 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f5a7fc640 1 -- 172.21.15.83:0/973100309 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ea1bc88a0 con 0x559ea18ad070 2024-09-21T03:30:08.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f5a7fc640 10 client.17586 handle_command_reply: tid=0 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 client.17586 shutdown 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 msgr2=0x559ea1c637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3f64013d30 tx=0x7f3f64013e20 comp rx=0 tx=0).stop 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 shutdown_connections 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x559ea18ad070 0x559ea1cdea40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea188c390 0x559ea1c63d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ea18841b0 0x559ea1c637d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f5a7fc640 0 client.17586 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 --2- 172.21.15.83:0/973100309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ea186b9f0 0x559ea1c68320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 >> 172.21.15.83:0/973100309 conn(0x559ea18a1460 msgr2=0x559ea1c604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 shutdown_connections 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.254+0000 7f3f7f628740 1 -- 172.21.15.83:0/973100309 wait complete. 2024-09-21T03:30:08.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f7f628740 20 client.17586 trim_cache size 0 max 16384 2024-09-21T03:30:08.261 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:08.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 msgr2=0x7f3f600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 0x7f3f600b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c1067c0 tx=0x7f3f6c03e070 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 msgr2=0x7f3f7419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0098d0 tx=0x7f3f6c0317b0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 shutdown_connections 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3f600b6680 0x7f3f600b8b90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f74108d10 0x7f3f7419fc60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f741083c0 0x7f3f7419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 --2- 172.21.15.83:0/3014231001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f74102a80 0x7f3f7419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 >> 172.21.15.83:0/3014231001 conn(0x7f3f74079740 msgr2=0x7f3f7406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 shutdown_connections 2024-09-21T03:30:08.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:08.255+0000 7f3f790b4640 1 -- 172.21.15.83:0/3014231001 wait complete. 2024-09-21T03:30:09.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:09 smithi193 ceph-mon[35224]: pgmap v743: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2024-09-21T03:30:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:09 smithi083 ceph-mon[31219]: pgmap v743: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2024-09-21T03:30:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:09 smithi142 ceph-mon[34512]: pgmap v743: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.26k op/s 2024-09-21T03:30:09.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:30:09.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:10.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:10 smithi083 ceph-mon[31219]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:30:10.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 -- 172.21.15.83:0/3930601019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc105300 msgr2=0x7fedfc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 --2- 172.21.15.83:0/3930601019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc105300 0x7fedfc10d850 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fede4009900 tx=0x7fede402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 -- 172.21.15.83:0/3930601019 shutdown_connections 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 --2- 172.21.15.83:0/3930601019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc10de10 0x7fedfc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 --2- 172.21.15.83:0/3930601019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc105300 0x7fedfc10d850 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 --2- 172.21.15.83:0/3930601019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc1049b0 0x7fedfc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 -- 172.21.15.83:0/3930601019 >> 172.21.15.83:0/3930601019 conn(0x7fedfc079730 msgr2=0x7fedfc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 -- 172.21.15.83:0/3930601019 shutdown_connections 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.461+0000 7fee01219640 1 -- 172.21.15.83:0/3930601019 wait complete. 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 Processor -- start 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 -- start start 2024-09-21T03:30:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 0x7fedfc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 0x7fedfc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc113ad0 con 0x7fedfc1049b0 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc113950 con 0x7fedfc105300 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fee01219640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc113c50 con 0x7fedfc10de10 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48008/0 (socket says 172.21.15.83:48008) 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 -- 172.21.15.83:0/2297726718 learned_addr learned my addr 172.21.15.83:0/2297726718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfb59e640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 0x7fedfc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedf3fff640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 0x7fedfc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 -- 172.21.15.83:0/2297726718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 msgr2=0x7fedfc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 0x7fedfc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 -- 172.21.15.83:0/2297726718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 msgr2=0x7fedfc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.462+0000 7fedfad9d640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 0x7fedfc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedfad9d640 1 -- 172.21.15.83:0/2297726718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc1a0470 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedf3fff640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 0x7fedfc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedfb59e640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 0x7fedfc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedfad9d640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b450 tx=0x7fedec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedf8d99640 1 -- 172.21.15.83:0/2297726718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec015020 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedf8d99640 1 -- 172.21.15.83:0/2297726718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedec011d40 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc1a0760 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fedf8d99640 1 -- 172.21.15.83:0/2297726718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec00f340 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.463+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1a6ec0 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.465+0000 7fee0778d740 1 Processor -- start 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.465+0000 7fee0778d740 1 -- start start 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.465+0000 7fedf8d99640 1 -- 172.21.15.83:0/2297726718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fedec018020 con 0x7fedfc1049b0 2024-09-21T03:30:10.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 0x55bca173d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca173dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bca16b3ea0 con 0x55bca1396070 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bca16a32d0 con 0x55bca1375390 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fee0778d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bca16a70e0 con 0x55bca136d1b0 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48010/0 (socket says 172.21.15.83:48010) 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 -- 172.21.15.83:0/1697123009 learned_addr learned my addr 172.21.15.83:0/1697123009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfad9d640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 0x55bca173d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedf3fff640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca173dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 -- 172.21.15.83:0/1697123009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 msgr2=0x55bca173d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 0x55bca173d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 -- 172.21.15.83:0/1697123009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 msgr2=0x55bca173dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca173dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfb59e640 1 -- 172.21.15.83:0/1697123009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca1401650 con 0x55bca1396070 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedf3fff640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca173dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.466+0000 7fedfad9d640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 0x55bca173d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedfb59e640 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009fc0 tx=0x7fedf400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10392d7e72cf345c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedf8d99640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 0x7fede80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedf17fa640 1 -- 172.21.15.83:0/1697123009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4003b70 con 0x55bca1396070 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedf17fa640 1 -- 172.21.15.83:0/1697123009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4003d10 con 0x55bca1396070 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedfad9d640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 0x7fede80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedf17fa640 1 -- 172.21.15.83:0/1697123009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf40119a0 con 0x55bca1396070 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.467+0000 7fedf8d99640 1 -- 172.21.15.83:0/2297726718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fedec0e71f0 con 0x7fedfc1049b0 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- 172.21.15.83:0/1697123009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 msgr2=0x55bca173d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009fc0 tx=0x7fedf400f7f0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fedfad9d640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 0x7fede80b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedec0e4a10 tx=0x7fedec0ec040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- 172.21.15.83:0/1697123009 shutdown_connections 2024-09-21T03:30:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca173dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca136d1b0 0x55bca173d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 --2- 172.21.15.83:0/1697123009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca1396070 0x55bca173d2c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- 172.21.15.83:0/1697123009 >> 172.21.15.83:0/1697123009 conn(0x55bca138a460 msgr2=0x55bca173b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- 172.21.15.83:0/1697123009 shutdown_connections 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- 172.21.15.83:0/1697123009 wait complete. 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 -- start start 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.468+0000 7fee0778d740 1 Processor -- start 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca174d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1396070 0x55bca174d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bca1504700 con 0x55bca136d1b0 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bca163e160 con 0x55bca1375390 2024-09-21T03:30:10.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bca174fdd0 con 0x55bca1396070 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fee0778d740 10 client.0 _ll_register_callbacks cb 0x55bca16579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48024/0 (socket says 172.21.15.83:48024) 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 -- 172.21.15.83:0/3765039871 learned_addr learned my addr 172.21.15.83:0/3765039871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedf3fff640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca174d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1396070 0x55bca174d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 -- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1396070 msgr2=0x55bca174d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1396070 0x55bca174d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 -- 172.21.15.83:0/3765039871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 msgr2=0x55bca174d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca174d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 -- 172.21.15.83:0/3765039871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca166bc90 con 0x55bca136d1b0 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedf3fff640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca174d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:10.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fedf4011540 tx=0x7fedf400ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f30d22b9fab826 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.469+0000 7fede27fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf40042d0 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fedf4004470 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4003d10 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fedf40116a0 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 10 client.17598 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55bca175bae0 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4003d10 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fedf4029020 con 0x55bca136d1b0 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 10 client.17598 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 10 client.17598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:30:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 20 client.17598 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca13549f0 0x55bca175eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 4 client.17598 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.470+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bca16b5df0 con 0x55bca13549f0 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.471+0000 7fedf3fff640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca13549f0 0x55bca175eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.471+0000 7fede27fc640 10 client.17598 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.471+0000 7fedf3fff640 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca13549f0 0x55bca175eef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1226b4a50dce5ec server_cookie=f341fb09148bf742 in_seq=0 out_seq=0 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fede27fc640 1 -- 172.21.15.83:0/3765039871 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bca16b5df0 con 0x55bca13549f0 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fede27fc640 10 client.17598 handle_command_reply: tid=0 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 client.17598 shutdown 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 msgr2=0x55bca174cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fedf4011540 tx=0x7fedf400ff80 comp rx=0 tx=0).stop 2024-09-21T03:30:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 shutdown_connections 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca13549f0 0x55bca175eef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1396070 0x55bca174d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca174d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fede27fc640 0 client.17598 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 --2- 172.21.15.83:0/3765039871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca174cd00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 >> 172.21.15.83:0/3765039871 conn(0x55bca138a460 msgr2=0x55bca1740450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 shutdown_connections 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 1 -- 172.21.15.83:0/3765039871 wait complete. 2024-09-21T03:30:10.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.472+0000 7fee0778d740 20 client.17598 trim_cache size 0 max 16384 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.485+0000 7fee0778d740 1 Processor -- start 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.485+0000 7fee0778d740 1 -- start start 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca136d1b0 0x55bca174cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1375390 0x55bca1749ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ad600 con 0x55bca13549f0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1af140 con 0x55bca136d1b0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fee0778d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1adcd0 con 0x55bca1375390 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48030/0 (socket says 172.21.15.83:48030) 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 -- 172.21.15.83:0/3204996501 learned_addr learned my addr 172.21.15.83:0/3204996501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 -- 172.21.15.83:0/3204996501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1375390 msgr2=0x55bca1749ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1375390 0x55bca1749ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 -- 172.21.15.83:0/3204996501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca136d1b0 msgr2=0x55bca174cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca136d1b0 0x55bca174cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 -- 172.21.15.83:0/3204996501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca174b200 con 0x55bca13549f0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.486+0000 7fedfb59e640 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fedf4010010 tx=0x7fedf4005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8964a39a387749 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.487+0000 7fedf0ff9640 1 -- 172.21.15.83:0/3204996501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4039020 con 0x55bca13549f0 2024-09-21T03:30:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.487+0000 7fedf0ff9640 1 -- 172.21.15.83:0/3204996501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4020020 con 0x55bca13549f0 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.487+0000 7fedf0ff9640 1 -- 172.21.15.83:0/3204996501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf401b410 con 0x55bca13549f0 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.487+0000 7fee0778d740 1 -- 172.21.15.83:0/3204996501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 msgr2=0x55bca174cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.487+0000 7fee0778d740 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fedf4010010 tx=0x7fedf4005f30 comp rx=0 tx=0).stop 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- 172.21.15.83:0/3204996501 shutdown_connections 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca1375390 0x55bca1749ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca136d1b0 0x55bca174cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- 172.21.15.83:0/3204996501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca13549f0 0x55bca174cab0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- 172.21.15.83:0/3204996501 >> 172.21.15.83:0/3204996501 conn(0x55bca138a460 msgr2=0x55bca174b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- 172.21.15.83:0/3204996501 shutdown_connections 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- 172.21.15.83:0/3204996501 wait complete. 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- start start 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 Processor -- start 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca13549f0 0x55bca17626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca1760160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1aea70 con 0x55bca136d1b0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ae3a0 con 0x55bca1375390 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ab600 con 0x55bca13549f0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.488+0000 7fee0778d740 10 client.0 _ll_register_callbacks cb 0x7fedf4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48036/0 (socket says 172.21.15.83:48036) 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 -- 172.21.15.83:0/512018523 learned_addr learned my addr 172.21.15.83:0/512018523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedfb59e640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca13549f0 0x55bca17626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedfb59e640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca1760160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 -- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca13549f0 msgr2=0x55bca17626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca13549f0 0x55bca17626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 -- 172.21.15.83:0/512018523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 msgr2=0x55bca1760160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca1760160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 -- 172.21.15.83:0/512018523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca17608a0 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedfb59e640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca1760160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fedf3fff640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fedfc06ad30 tx=0x7fede4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e4b22a3d48fde2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fede27fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede404a060 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fede4004390 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4020020 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fede4035040 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fee0778d740 10 client.17604 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.489+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55bca17c95f0 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4020020 con 0x55bca136d1b0 2024-09-21T03:30:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fede4004af0 con 0x55bca136d1b0 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 10 client.17604 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 10 client.17604 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 20 client.17604 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca1396070 0x55bca17cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 4 client.17604 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bca16b18a0 con 0x55bca1396070 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.490+0000 7fedfb59e640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca1396070 0x55bca17cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fede27fc640 10 client.17604 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fedfb59e640 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca1396070 0x55bca17cbe50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a54025149786ec1f server_cookie=a6c79a63d9b64386 in_seq=0 out_seq=0 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fede27fc640 1 -- 172.21.15.83:0/512018523 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bca16b18a0 con 0x55bca1396070 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fede27fc640 10 client.17604 handle_command_reply: tid=0 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 client.17604 shutdown 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 msgr2=0x55bca175fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fedfc06ad30 tx=0x7fede4005f00 comp rx=0 tx=0).stop 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 shutdown_connections 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bca1375390 0x55bca1760160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bca136d1b0 0x55bca175fc20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55bca1396070 0x55bca17cbe50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 --2- 172.21.15.83:0/512018523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca13549f0 0x55bca17626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 >> 172.21.15.83:0/512018523 conn(0x55bca138a460 msgr2=0x55bca174b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fede27fc640 0 client.17604 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 shutdown_connections 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 1 -- 172.21.15.83:0/512018523 wait complete. 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.491+0000 7fee0778d740 20 client.17604 trim_cache size 0 max 16384 2024-09-21T03:30:10.500 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 msgr2=0x7fede80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 0x7fede80b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedec0e4a10 tx=0x7fedec0ec040 comp rx=0 tx=0).stop 2024-09-21T03:30:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 msgr2=0x7fedfc19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b450 tx=0x7fedec00b910 comp rx=0 tx=0).stop 2024-09-21T03:30:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 shutdown_connections 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fede80b6680 0x7fede80b8b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedfc10de10 0x7fedfc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedfc105300 0x7fedfc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 --2- 172.21.15.83:0/2297726718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedfc1049b0 0x7fedfc19a500 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 >> 172.21.15.83:0/2297726718 conn(0x7fedfc079730 msgr2=0x7fedfc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 shutdown_connections 2024-09-21T03:30:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:10.492+0000 7fee01219640 1 -- 172.21.15.83:0/2297726718 wait complete. 2024-09-21T03:30:10.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:10 smithi193 ceph-mon[35224]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:30:10.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:10 smithi142 ceph-mon[34512]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:30:10.941 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 770.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T03:30:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:11 smithi193 ceph-mon[35224]: pgmap v745: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-09-21T03:30:11.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:11 smithi083 ceph-mon[31219]: pgmap v745: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-09-21T03:30:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:11 smithi142 ceph-mon[34512]: pgmap v745: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-09-21T03:30:11.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:30:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:12.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 -- 172.21.15.83:0/4024776671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 msgr2=0x7f33f4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/4024776671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f4102e90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009900 tx=0x7f33f002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 -- 172.21.15.83:0/4024776671 shutdown_connections 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/4024776671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f4108d10 0x7f33f410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/4024776671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f41083c0 0x7f33f41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/4024776671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f4102e90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 -- 172.21.15.83:0/4024776671 >> 172.21.15.83:0/4024776671 conn(0x7f33f4079740 msgr2=0x7f33f4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 -- 172.21.15.83:0/4024776671 shutdown_connections 2024-09-21T03:30:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 -- 172.21.15.83:0/4024776671 wait complete. 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.536+0000 7f33fb3c7640 1 Processor -- start 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 -- start start 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 0x7f33f419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f4108d10 0x7f33f419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4112980 con 0x7f33f4102a80 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4112800 con 0x7f33f4108d10 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33fb3c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4112b00 con 0x7f33f41083c0 2024-09-21T03:30:12.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48058/0 (socket says 172.21.15.83:48058) 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 -- 172.21.15.83:0/2730904927 learned_addr learned my addr 172.21.15.83:0/2730904927 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f8962640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 0x7f33f419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 -- 172.21.15.83:0/2730904927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 msgr2=0x7f33f419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 0x7f33f419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 -- 172.21.15.83:0/2730904927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f4108d10 msgr2=0x7f33f419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f4108d10 0x7f33f419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 -- 172.21.15.83:0/2730904927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f41a03b0 con 0x7f33f4102a80 2024-09-21T03:30:12.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f8962640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 0x7f33f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.537+0000 7f33f9163640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005940 tx=0x7f33f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.538+0000 7f33ea7fc640 1 -- 172.21.15.83:0/2730904927 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f003f030 con 0x7f33f4102a80 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.538+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f41a0640 con 0x7f33f4102a80 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.538+0000 7f33ea7fc640 1 -- 172.21.15.83:0/2730904927 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f0003cb0 con 0x7f33f4102a80 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.538+0000 7f33ea7fc640 1 -- 172.21.15.83:0/2730904927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035070 con 0x7f33f4102a80 2024-09-21T03:30:12.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.538+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f41a6e00 con 0x7f33f4102a80 2024-09-21T03:30:12.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.556+0000 7f340193b740 1 Processor -- start 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 -- start start 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a698763100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698763680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a698775a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6986dbea0 con 0x55a6983be070 2024-09-21T03:30:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a6986cb2d0 con 0x55a6983951b0 2024-09-21T03:30:12.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.558+0000 7f340193b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a6986cf0e0 con 0x55a69839d390 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.559+0000 7f33f9163640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698763680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.559+0000 7f33ea7fc640 1 -- 172.21.15.83:0/2730904927 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f33f0042030 con 0x7f33f4102a80 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.560+0000 7f33ea7fc640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 0x7f33e00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.560+0000 7f33ea7fc640 1 -- 172.21.15.83:0/2730904927 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f33f01080c0 con 0x7f33f4102a80 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.559+0000 7f33f9163640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698763680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45628/0 (socket says 172.21.15.83:45628) 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.560+0000 7f33f9163640 1 -- 172.21.15.83:0/2442037996 learned_addr learned my addr 172.21.15.83:0/2442037996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.561+0000 7f33f9964640 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a698763100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 0x7f33e00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 -- 172.21.15.83:0/2442037996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 msgr2=0x55a698775a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:30:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a698775a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 -- 172.21.15.83:0/2442037996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 msgr2=0x55a698763680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698763680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.562+0000 7f33f9964640 1 -- 172.21.15.83:0/2442037996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a698429650 con 0x55a6983be070 2024-09-21T03:30:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.563+0000 7f33f9964640 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a698763100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015840 tx=0x7f33ec015c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477f5139d15f370e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.563+0000 7f33f9964640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 0x7f33e00b8c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015fd0 tx=0x7f33ec019000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.564+0000 7f33df7fe640 1 -- 172.21.15.83:0/2442037996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec021070 con 0x55a6983be070 2024-09-21T03:30:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.564+0000 7f33df7fe640 1 -- 172.21.15.83:0/2442037996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec004d20 con 0x55a6983be070 2024-09-21T03:30:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.564+0000 7f33df7fe640 1 -- 172.21.15.83:0/2442037996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec01d8d0 con 0x55a6983be070 2024-09-21T03:30:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.568+0000 7f340193b740 1 -- 172.21.15.83:0/2442037996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 msgr2=0x55a698763100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.568+0000 7f340193b740 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a698763100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015840 tx=0x7f33ec015c40 comp rx=0 tx=0).stop 2024-09-21T03:30:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.572+0000 7f340193b740 1 -- 172.21.15.83:0/2442037996 shutdown_connections 2024-09-21T03:30:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.572+0000 7f340193b740 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a698775a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.572+0000 7f340193b740 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698763680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.572+0000 7f340193b740 1 --2- 172.21.15.83:0/2442037996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a698763100 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.572+0000 7f340193b740 1 -- 172.21.15.83:0/2442037996 >> 172.21.15.83:0/2442037996 conn(0x55a6983b2460 msgr2=0x55a698765620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.575+0000 7f340193b740 1 -- 172.21.15.83:0/2442037996 shutdown_connections 2024-09-21T03:30:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.576+0000 7f340193b740 1 -- 172.21.15.83:0/2442037996 wait complete. 2024-09-21T03:30:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.576+0000 7f340193b740 1 -- start start 2024-09-21T03:30:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.576+0000 7f340193b740 1 Processor -- start 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698779e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a69877a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a69877f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a69852c700 con 0x55a6983be070 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a698666160 con 0x55a6983951b0 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a6987764f0 con 0x55a69839d390 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.577+0000 7f340193b740 10 client.0 _ll_register_callbacks cb 0x55a69867f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f9964640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a69877f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f9964640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a69877f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48070/0 (socket says 172.21.15.83:48070) 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f9964640 1 -- 172.21.15.83:0/2676755326 learned_addr learned my addr 172.21.15.83:0/2676755326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:12.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a69877a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f9163640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698779e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 -- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 msgr2=0x55a698779e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698779e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 -- 172.21.15.83:0/2676755326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 msgr2=0x55a69877f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a69877f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.578+0000 7f33f8962640 1 -- 172.21.15.83:0/2676755326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a698693c90 con 0x55a69839d390 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f33f8962640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a69877a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f406b6c0 tx=0x7f33e400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97e19806583903a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f33dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4016070 con 0x55a69839d390 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec004d20 con 0x55a69839d390 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f33e4011880 con 0x55a69839d390 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f33e4022050 con 0x55a69839d390 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.579+0000 7f340193b740 10 client.27109 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.580+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a6987993c0 con 0x55a69839d390 2024-09-21T03:30:12.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.580+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec004d20 con 0x55a69839d390 2024-09-21T03:30:12.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.580+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f33e400f040 con 0x55a69839d390 2024-09-21T03:30:12.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 10 client.27109 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:12.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 10 client.27109 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:30:12.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 20 client.27109 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a69837c9f0 0x55a69879c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 4 client.27109 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6986dddf0 con 0x55a69837c9f0 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.581+0000 7f33f9163640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a69837c9f0 0x55a69879c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.582+0000 7f33f9163640 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a69837c9f0 0x55a69879c900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1b1285d84e046fc server_cookie=5ed74bce1607fd13 in_seq=0 out_seq=0 2024-09-21T03:30:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.582+0000 7f33dcff9640 10 client.27109 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:30:12.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f33dcff9640 1 -- 172.21.15.83:0/2676755326 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6986dddf0 con 0x55a69837c9f0 2024-09-21T03:30:12.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f33dcff9640 10 client.27109 handle_command_reply: tid=0 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 client.27109 shutdown 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 msgr2=0x55a69877a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a69877a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33f406b6c0 tx=0x7f33e400e6c0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 shutdown_connections 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a69837c9f0 0x55a69879c900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6983be070 0x55a69877f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a69839d390 0x55a69877a3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 --2- 172.21.15.83:0/2676755326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698779e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.583+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 >> 172.21.15.83:0/2676755326 conn(0x55a6983b2460 msgr2=0x55a6987645d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.584+0000 7f33dcff9640 0 client.27109 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:30:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.584+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 shutdown_connections 2024-09-21T03:30:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.584+0000 7f340193b740 1 -- 172.21.15.83:0/2676755326 wait complete. 2024-09-21T03:30:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.584+0000 7f340193b740 20 client.27109 trim_cache size 0 max 16384 2024-09-21T03:30:12.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.599+0000 7f340193b740 1 Processor -- start 2024-09-21T03:30:12.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.599+0000 7f340193b740 1 -- start start 2024-09-21T03:30:12.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698777710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a69877f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4073220 con 0x55a69839d390 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4074d60 con 0x55a6983951b0 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f340193b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33f40738f0 con 0x55a6983be070 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48078/0 (socket says 172.21.15.83:48078) 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 -- 172.21.15.83:0/2733070733 learned_addr learned my addr 172.21.15.83:0/2733070733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f9964640 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a69877f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 -- 172.21.15.83:0/2733070733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 msgr2=0x55a69877f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a69877f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 -- 172.21.15.83:0/2733070733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 msgr2=0x55a698777710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698777710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.600+0000 7f33f8962640 1 -- 172.21.15.83:0/2733070733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a698778100 con 0x55a69839d390 2024-09-21T03:30:12.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.601+0000 7f33f8962640 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f33e4010ae0 tx=0x7f33e4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeffda8402c976da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.601+0000 7f33deffd640 1 -- 172.21.15.83:0/2733070733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e402b070 con 0x55a69839d390 2024-09-21T03:30:12.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.601+0000 7f33deffd640 1 -- 172.21.15.83:0/2733070733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e4010880 con 0x55a69839d390 2024-09-21T03:30:12.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.601+0000 7f33deffd640 1 -- 172.21.15.83:0/2733070733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4016070 con 0x55a69839d390 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- 172.21.15.83:0/2733070733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 msgr2=0x55a69877f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f33e4010ae0 tx=0x7f33e4010dd0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- 172.21.15.83:0/2733070733 shutdown_connections 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a69877f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a69877f370 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 --2- 172.21.15.83:0/2733070733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a698777710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- 172.21.15.83:0/2733070733 >> 172.21.15.83:0/2733070733 conn(0x55a6983b2460 msgr2=0x55a698779e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- 172.21.15.83:0/2733070733 shutdown_connections 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- 172.21.15.83:0/2733070733 wait complete. 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 -- start start 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.602+0000 7f340193b740 1 Processor -- start 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a6987a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f9163640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f9163640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45658/0 (socket says 172.21.15.83:45658) 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f8962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f8962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48088/0 (socket says 172.21.15.83:48088) 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f9163640 1 -- 172.21.15.83:0/3271615465 learned_addr learned my addr 172.21.15.83:0/3271615465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:12.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4074690 con 0x55a69839d390 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4073fc0 con 0x55a6983951b0 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4071220 con 0x55a6983be070 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f33f9964640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a6987a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.603+0000 7f340193b740 10 client.0 _ll_register_callbacks cb 0x7f33e4005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 -- 172.21.15.83:0/3271615465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 msgr2=0x55a6987a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a6987a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 -- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 msgr2=0x55a6987a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6987a24e0 con 0x55a69839d390 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f9163640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33f8962640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f33e4010cd0 tx=0x7f33e4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fea5282c2080e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e402b070 con 0x55a69839d390 2024-09-21T03:30:12.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e4010880 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f33e4005240 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f33e4016040 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f340193b740 10 client.17622 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a698807980 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.604+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e4005440 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f33e4036020 con 0x55a69839d390 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 10 client.17622 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 10 client.17622 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 20 client.17622 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a698351da0 0x55a698809ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:12.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 4 client.17622 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f33e4010880 con 0x55a698351da0 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f33f9163640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a698351da0 0x55a698809ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f33dcff9640 10 client.17622 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.605+0000 7f33f9163640 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a698351da0 0x55a698809ff0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1fbd8015257f8b1 server_cookie=35246a787f032a24 in_seq=0 out_seq=0 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f33dcff9640 1 -- 172.21.15.83:0/3271615465 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f33e4010880 con 0x55a698351da0 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f33dcff9640 10 client.17622 handle_command_reply: tid=0 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 client.17622 shutdown 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 msgr2=0x55a6987a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f33e4010cd0 tx=0x7f33e4009580 comp rx=0 tx=0).stop 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 shutdown_connections 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55a698351da0 0x55a698809ff0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6983be070 0x55a6987a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a69839d390 0x55a6987a1830 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 --2- 172.21.15.83:0/3271615465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6983951b0 0x55a6987a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 >> 172.21.15.83:0/3271615465 conn(0x55a6983b2460 msgr2=0x55a69877f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f33dcff9640 0 client.17622 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:30:12.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 shutdown_connections 2024-09-21T03:30:12.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 1 -- 172.21.15.83:0/3271615465 wait complete. 2024-09-21T03:30:12.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.606+0000 7f340193b740 20 client.17622 trim_cache size 0 max 16384 2024-09-21T03:30:12.617 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 msgr2=0x7f33e00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 0x7f33e00b8c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f33ec015fd0 tx=0x7f33ec019000 comp rx=0 tx=0).stop 2024-09-21T03:30:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 msgr2=0x7f33f419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005940 tx=0x7f33f00042f0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 shutdown_connections 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f33e00b6700 0x7f33e00b8c10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f4108d10 0x7f33f419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33f41083c0 0x7f33f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 --2- 172.21.15.83:0/2730904927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33f4102a80 0x7f33f419a440 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 >> 172.21.15.83:0/2730904927 conn(0x7f33f4079740 msgr2=0x7f33f4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 shutdown_connections 2024-09-21T03:30:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:12.607+0000 7f33fb3c7640 1 -- 172.21.15.83:0/2730904927 wait complete. 2024-09-21T03:30:13.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:13 smithi083 ceph-mon[31219]: pgmap v746: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-09-21T03:30:13.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:30:13.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:13 smithi193 ceph-mon[35224]: pgmap v746: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-09-21T03:30:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:13 smithi142 ceph-mon[34512]: pgmap v746: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-09-21T03:30:13.699 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- 172.21.15.83:0/2846412129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8590109df0 msgr2=0x7f8590110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 --2- 172.21.15.83:0/2846412129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8590109df0 0x7f8590110620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8588009900 tx=0x7f858802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- 172.21.15.83:0/2846412129 shutdown_connections 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 --2- 172.21.15.83:0/2846412129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8590109df0 0x7f8590110620 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 --2- 172.21.15.83:0/2846412129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85901094a0 0x7f85901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 --2- 172.21.15.83:0/2846412129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f8590103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- 172.21.15.83:0/2846412129 >> 172.21.15.83:0/2846412129 conn(0x7f859007b850 msgr2=0x7f859007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- 172.21.15.83:0/2846412129 shutdown_connections 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- 172.21.15.83:0/2846412129 wait complete. 2024-09-21T03:30:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 Processor -- start 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.157+0000 7f8596abd640 1 -- start start 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f859019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8590109df0 0x7f859019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8590113af0 con 0x7f85901094a0 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8590113970 con 0x7f8590109df0 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8596abd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8590113c70 con 0x7f8590103b60 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8594859640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f859019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48108/0 (socket says 172.21.15.83:48108) 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 -- 172.21.15.83:0/3218682111 learned_addr learned my addr 172.21.15.83:0/3218682111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f859505a640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8590109df0 0x7f859019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 -- 172.21.15.83:0/3218682111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 msgr2=0x7f859019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f859019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 -- 172.21.15.83:0/3218682111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8590109df0 msgr2=0x7f859019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8590109df0 0x7f859019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 -- 172.21.15.83:0/3218682111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85901a0430 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8594859640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f859019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8587fff640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8580006fd0 tx=0x7f858000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.158+0000 7f8585ffb640 1 -- 172.21.15.83:0/3218682111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580011830 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.159+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85901a0720 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.159+0000 7f8585ffb640 1 -- 172.21.15.83:0/3218682111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8580016070 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.159+0000 7f8585ffb640 1 -- 172.21.15.83:0/3218682111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580010e00 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.159+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a6f00 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.161+0000 7f859d031740 1 Processor -- start 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.161+0000 7f859d031740 1 -- start start 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.161+0000 7f8585ffb640 1 -- 172.21.15.83:0/3218682111 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f85800119d0 con 0x7f85901094a0 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91d4070 0x55aba957b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91b3390 0x55aba957c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aba94f1ea0 con 0x55aba91ab1b0 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aba94e12d0 con 0x55aba91b3390 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859d031740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aba94e50e0 con 0x55aba91d4070 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48124/0 (socket says 172.21.15.83:48124) 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 -- 172.21.15.83:0/3804142069 learned_addr learned my addr 172.21.15.83:0/3804142069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859505a640 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91d4070 0x55aba957b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8587fff640 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91b3390 0x55aba957c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 -- 172.21.15.83:0/3804142069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91d4070 msgr2=0x55aba957b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91d4070 0x55aba957b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 -- 172.21.15.83:0/3804142069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91b3390 msgr2=0x55aba957c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91b3390 0x55aba957c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 -- 172.21.15.83:0/3804142069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba923f650 con 0x55aba91ab1b0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8585ffb640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 0x7f85780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f8594859640 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f858c0107e0 tx=0x7f858c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a7ba8e4cf32f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.162+0000 7f859505a640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 0x7f85780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f8576ffd640 1 -- 172.21.15.83:0/3804142069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c016070 con 0x55aba91ab1b0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f8585ffb640 1 -- 172.21.15.83:0/3218682111 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f85800e6ba0 con 0x7f85901094a0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f8576ffd640 1 -- 172.21.15.83:0/3804142069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c01b460 con 0x55aba91ab1b0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f8576ffd640 1 -- 172.21.15.83:0/3804142069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c0126b0 con 0x55aba91ab1b0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 -- 172.21.15.83:0/3804142069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 msgr2=0x55aba957bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859505a640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 0x7f85780b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8588006010 tx=0x7f85880058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f858c0107e0 tx=0x7f858c010ca0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 -- 172.21.15.83:0/3804142069 shutdown_connections 2024-09-21T03:30:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91b3390 0x55aba957c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91ab1b0 0x55aba957bbe0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 --2- 172.21.15.83:0/3804142069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91d4070 0x55aba957b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.163+0000 7f859d031740 1 -- 172.21.15.83:0/3804142069 >> 172.21.15.83:0/3804142069 conn(0x55aba91c8460 msgr2=0x55aba9579590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.164+0000 7f859d031740 1 -- 172.21.15.83:0/3804142069 shutdown_connections 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.164+0000 7f859d031740 1 -- 172.21.15.83:0/3804142069 wait complete. 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.164+0000 7f859d031740 1 -- start start 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.164+0000 7f859d031740 1 Processor -- start 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba9586d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91d4070 0x55aba9587800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aba9342700 con 0x55aba91b3390 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aba947c160 con 0x55aba91d4070 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aba9589e80 con 0x55aba91ab1b0 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859d031740 10 client.0 _ll_register_callbacks cb 0x55aba94959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48128/0 (socket says 172.21.15.83:48128) 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 -- 172.21.15.83:0/978413655 learned_addr learned my addr 172.21.15.83:0/978413655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f859505a640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91d4070 0x55aba9587800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8594859640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba9586d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 -- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 msgr2=0x55aba9586d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba9586d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 -- 172.21.15.83:0/978413655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91d4070 msgr2=0x55aba9587800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91d4070 0x55aba9587800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 -- 172.21.15.83:0/978413655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba94a9c90 con 0x55aba91b3390 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8594859640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba9586d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f8587fff640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f85800a9c80 tx=0x7f8580000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c369d6b5a5f313 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f856ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85800d00b0 con 0x55aba91b3390 2024-09-21T03:30:14.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.165+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f85800a8360 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f858c01b460 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f85800ac520 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 10 client.17634 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aba9595be0 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c01b460 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f85800b1080 con 0x55aba91b3390 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 10 client.17634 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 10 client.17634 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 20 client.17634 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91929f0 0x55aba9599120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 4 client.17634 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aba94f3df0 con 0x55aba91929f0 2024-09-21T03:30:14.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.166+0000 7f8594859640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91929f0 0x55aba9599120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f856ffff640 10 client.17634 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f8594859640 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91929f0 0x55aba9599120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f59a6cfb0487300 server_cookie=19dee39008e0d3d0 in_seq=0 out_seq=0 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f856ffff640 1 -- 172.21.15.83:0/978413655 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aba94f3df0 con 0x55aba91929f0 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f856ffff640 10 client.17634 handle_command_reply: tid=0 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f859d031740 1 client.17634 shutdown 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 msgr2=0x55aba9587280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.168+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f85800a9c80 tx=0x7f8580000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 shutdown_connections 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91929f0 0x55aba9599120 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f856ffff640 0 client.17634 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91d4070 0x55aba9587800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9587280 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 --2- 172.21.15.83:0/978413655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba9586d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 >> 172.21.15.83:0/978413655 conn(0x55aba91c8460 msgr2=0x55aba957e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 shutdown_connections 2024-09-21T03:30:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 1 -- 172.21.15.83:0/978413655 wait complete. 2024-09-21T03:30:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.169+0000 7f859d031740 20 client.17634 trim_cache size 0 max 16384 2024-09-21T03:30:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 Processor -- start 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 -- start start 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91929f0 0x55aba957d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91ab1b0 0x55aba9588970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85901af770 con 0x55aba91b3390 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85901b12b0 con 0x55aba91ab1b0 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859d031740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85901afe40 con 0x55aba91929f0 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859505a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859505a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48134/0 (socket says 172.21.15.83:48134) 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f859505a640 1 -- 172.21.15.83:0/3109171012 learned_addr learned my addr 172.21.15.83:0/3109171012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f8587fff640 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91ab1b0 0x55aba9588970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.182+0000 7f8594859640 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91929f0 0x55aba957d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 -- 172.21.15.83:0/3109171012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91929f0 msgr2=0x55aba957d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91929f0 0x55aba957d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 -- 172.21.15.83:0/3109171012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91ab1b0 msgr2=0x55aba9588970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91ab1b0 0x55aba9588970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 -- 172.21.15.83:0/3109171012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba957a350 con 0x55aba91b3390 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f859505a640 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8588033000 tx=0x7f858803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8c2dd299f8fc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f85767fc640 1 -- 172.21.15.83:0/3109171012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858804e070 con 0x55aba91b3390 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f85767fc640 1 -- 172.21.15.83:0/3109171012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8588004440 con 0x55aba91b3390 2024-09-21T03:30:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.183+0000 7f85767fc640 1 -- 172.21.15.83:0/3109171012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588035040 con 0x55aba91b3390 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- 172.21.15.83:0/3109171012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 msgr2=0x55aba9578bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8588033000 tx=0x7f858803f000 comp rx=0 tx=0).stop 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- 172.21.15.83:0/3109171012 shutdown_connections 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba9578bc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91ab1b0 0x55aba9588970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 --2- 172.21.15.83:0/3109171012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91929f0 0x55aba957d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- 172.21.15.83:0/3109171012 >> 172.21.15.83:0/3109171012 conn(0x55aba91c8460 msgr2=0x55aba9588260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- 172.21.15.83:0/3109171012 shutdown_connections 2024-09-21T03:30:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- 172.21.15.83:0/3109171012 wait complete. 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 -- start start 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.184+0000 7f859d031740 1 Processor -- start 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 0x55aba958fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba958af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85901b0be0 con 0x55aba91b3390 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85901b0510 con 0x55aba91929f0 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85901ad770 con 0x55aba91ab1b0 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859d031740 10 client.0 _ll_register_callbacks cb 0x7f8580010720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859505a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 0x55aba958fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859505a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48142/0 (socket says 172.21.15.83:48142) 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859505a640 1 -- 172.21.15.83:0/3705835872 learned_addr learned my addr 172.21.15.83:0/3705835872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8587fff640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba958af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 -- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 msgr2=0x55aba958af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba958af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 -- 172.21.15.83:0/3705835872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 msgr2=0x55aba958b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8594859640 1 -- 172.21.15.83:0/3705835872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba9602ae0 con 0x55aba91929f0 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f8587fff640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba958af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.185+0000 7f859505a640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f8594859640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 0x55aba958fab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f858c0107b0 tx=0x7f858c004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3168fde482460788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f856ffff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:30:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c034060 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8588004440 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7f858c00eb90 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f858c016050 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f859d031740 10 client.27173 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55aba9603bb0 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.186+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8588004440 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f858c00f040 con 0x55aba91929f0 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 10 client.27173 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 10 client.27173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 20 client.27173 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91d4070 0x55aba9606060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 4 client.27173 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aba95828f0 con 0x55aba91d4070 2024-09-21T03:30:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f8587fff640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91d4070 0x55aba9606060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f856ffff640 10 client.27173 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:30:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.187+0000 7f8587fff640 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91d4070 0x55aba9606060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b68e9b44beaff965 server_cookie=9eabc5bcc3d94def in_seq=0 out_seq=0 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f856ffff640 1 -- 172.21.15.83:0/3705835872 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aba95828f0 con 0x55aba91d4070 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f856ffff640 10 client.27173 handle_command_reply: tid=0 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 client.27173 shutdown 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 msgr2=0x55aba958fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 0x55aba958fab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f858c0107b0 tx=0x7f858c004a50 comp rx=0 tx=0).stop 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 shutdown_connections 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55aba91d4070 0x55aba9606060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aba91b3390 0x55aba958b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f856ffff640 0 client.27173 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aba91ab1b0 0x55aba958af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 --2- 172.21.15.83:0/3705835872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aba91929f0 0x55aba958fab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 >> 172.21.15.83:0/3705835872 conn(0x55aba91c8460 msgr2=0x55aba9592a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 shutdown_connections 2024-09-21T03:30:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 1 -- 172.21.15.83:0/3705835872 wait complete. 2024-09-21T03:30:14.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.188+0000 7f859d031740 20 client.27173 trim_cache size 0 max 16384 2024-09-21T03:30:14.195 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 msgr2=0x7f85780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 0x7f85780b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8588006010 tx=0x7f85880058d0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 msgr2=0x7f859019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8580006fd0 tx=0x7f858000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 shutdown_connections 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f85780b6680 0x7f85780b8b90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8590109df0 0x7f859019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85901094a0 0x7f859019aa60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 --2- 172.21.15.83:0/3218682111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8590103b60 0x7f859019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 >> 172.21.15.83:0/3218682111 conn(0x7f859007b850 msgr2=0x7f85901074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 shutdown_connections 2024-09-21T03:30:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:14.189+0000 7f8596abd640 1 -- 172.21.15.83:0/3218682111 wait complete. 2024-09-21T03:30:15.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:15 smithi083 ceph-mon[31219]: pgmap v747: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-21T03:30:15.479 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:15 smithi142 ceph-mon[34512]: pgmap v747: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-21T03:30:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:15 smithi193 ceph-mon[35224]: pgmap v747: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-21T03:30:15.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:30:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 775.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T03:30:16.074 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.623+0000 7fafc46a2640 1 -- 172.21.15.83:0/3293941463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 msgr2=0x7fafbc1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.623+0000 7fafc46a2640 1 --2- 172.21.15.83:0/3293941463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc1087c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 -- 172.21.15.83:0/3293941463 shutdown_connections 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 --2- 172.21.15.83:0/3293941463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc108d00 0x7fafbc10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 --2- 172.21.15.83:0/3293941463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc1087c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 --2- 172.21.15.83:0/3293941463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc102ac0 0x7fafbc102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 -- 172.21.15.83:0/3293941463 >> 172.21.15.83:0/3293941463 conn(0x7fafbc079790 msgr2=0x7fafbc079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 -- 172.21.15.83:0/3293941463 shutdown_connections 2024-09-21T03:30:16.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 -- 172.21.15.83:0/3293941463 wait complete. 2024-09-21T03:30:16.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 Processor -- start 2024-09-21T03:30:16.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.624+0000 7fafc46a2640 1 -- start start 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc108d00 0x7fafbc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1129c0 con 0x7fafbc1083b0 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc112840 con 0x7fafbc102ac0 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc46a2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc112b40 con 0x7fafbc108d00 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48162/0 (socket says 172.21.15.83:48162) 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45740/0 (socket says 172.21.15.83:45740) 2024-09-21T03:30:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc1c3d640 1 -- 172.21.15.83:0/375547987 learned_addr learned my addr 172.21.15.83:0/375547987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc108d00 0x7fafbc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 -- 172.21.15.83:0/375547987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc108d00 msgr2=0x7fafbc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc108d00 0x7fafbc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 -- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 msgr2=0x7fafbc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.625+0000 7fafc243e640 1 -- 172.21.15.83:0/375547987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1a0450 con 0x7fafbc102ac0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.626+0000 7fafc243e640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e9f0 tx=0x7fafb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.626+0000 7fafb37fe640 1 -- 172.21.15.83:0/375547987 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb800ccc0 con 0x7fafbc102ac0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.626+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1a0740 con 0x7fafbc102ac0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.626+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc0ff530 con 0x7fafbc102ac0 2024-09-21T03:30:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.626+0000 7fafb37fe640 1 -- 172.21.15.83:0/375547987 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb80108e0 con 0x7fafbc102ac0 2024-09-21T03:30:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.628+0000 7fafb37fe640 1 -- 172.21.15.83:0/375547987 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8018880 con 0x7fafbc102ac0 2024-09-21T03:30:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.630+0000 7fafcac16740 1 Processor -- start 2024-09-21T03:30:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 -- start start 2024-09-21T03:30:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23f596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f59be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a23ecfea0 con 0x558a23b891b0 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558a23ebf2d0 con 0x558a23bb2070 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafcac16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a23ec30e0 con 0x558a23b91390 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55758/0 (socket says 172.21.15.83:55758) 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafc1c3d640 1 -- 172.21.15.83:0/730741846 learned_addr learned my addr 172.21.15.83:0/730741846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.631+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23f596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 -- 172.21.15.83:0/730741846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 msgr2=0x558a23f596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23f596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 -- 172.21.15.83:0/730741846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 msgr2=0x558a23f59be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:30:16.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f59be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 -- 172.21.15.83:0/730741846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a23c1d650 con 0x558a23b91390 2024-09-21T03:30:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.632+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fafac009230 tx=0x7fafac031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b5c9e756e09c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.633+0000 7fafb0ff9640 1 -- 172.21.15.83:0/730741846 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac040070 con 0x558a23b91390 2024-09-21T03:30:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.633+0000 7fafb37fe640 1 -- 172.21.15.83:0/375547987 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fafb8010430 con 0x7fafbc102ac0 2024-09-21T03:30:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.633+0000 7fafb0ff9640 1 -- 172.21.15.83:0/730741846 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafac004030 con 0x558a23b91390 2024-09-21T03:30:16.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafb37fe640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 0x7fafa80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafb0ff9640 1 -- 172.21.15.83:0/730741846 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac035040 con 0x558a23b91390 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 0x7fafa80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafb37fe640 1 -- 172.21.15.83:0/375547987 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fafb8018620 con 0x7fafbc102ac0 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafcac16740 1 -- 172.21.15.83:0/730741846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 msgr2=0x558a23f5a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.634+0000 7fafcac16740 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fafac009230 tx=0x7fafac031b10 comp rx=0 tx=0).stop 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 -- 172.21.15.83:0/730741846 shutdown_connections 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b91390 0x558a23f5a120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 --2- 172.21.15.83:0/730741846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23f596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 -- 172.21.15.83:0/730741846 >> 172.21.15.83:0/730741846 conn(0x558a23ba6460 msgr2=0x558a23f57590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 -- 172.21.15.83:0/730741846 shutdown_connections 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 0x7fafa80b8d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005fd0 tx=0x7fafb4009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 -- 172.21.15.83:0/730741846 wait complete. 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 -- start start 2024-09-21T03:30:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.635+0000 7fafcac16740 1 Processor -- start 2024-09-21T03:30:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 0x558a23f6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a23d20700 con 0x558a23b891b0 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558a23e5a160 con 0x558a23b91390 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a23f5fd90 con 0x558a23bb2070 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafcac16740 10 client.0 _ll_register_callbacks cb 0x558a23e739e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48166/0 (socket says 172.21.15.83:48166) 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 -- 172.21.15.83:0/3123711292 learned_addr learned my addr 172.21.15.83:0/3123711292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 0x558a23f6ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 -- 172.21.15.83:0/3123711292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 msgr2=0x558a23f6ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 0x558a23f6ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 -- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 msgr2=0x558a23f65bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f65bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 -- 172.21.15.83:0/3123711292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a23e87c90 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 0x558a23f6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafc243e640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafb80e4e80 tx=0x7fafb80e5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea676a252d396045 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.636+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80abb40 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafac004030 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fafb8010a90 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fafb800ce60 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 10 client.17649 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558a23f84bc0 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafac004030 con 0x558a23b891b0 2024-09-21T03:30:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fafb80b3140 con 0x558a23b891b0 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 10 client.17649 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 20 client.17649 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b709f0 0x558a23f88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 4 client.17649 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.637+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a23ed1df0 con 0x558a23b709f0 2024-09-21T03:30:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.638+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b709f0 0x558a23f88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.638+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b709f0 0x558a23f88050 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352968b08e8b4e59 server_cookie=53245b43a338903 in_seq=0 out_seq=0 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.638+0000 7fafa5ffb640 10 client.17649 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.639+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3123711292 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a23ed1df0 con 0x558a23b709f0 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.639+0000 7fafa5ffb640 10 client.17649 handle_command_reply: tid=0 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.639+0000 7fafcac16740 1 client.17649 shutdown 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.639+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 msgr2=0x558a23f65610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.639+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafb80e4e80 tx=0x7fafb80e5340 comp rx=0 tx=0).stop 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 shutdown_connections 2024-09-21T03:30:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b709f0 0x558a23f88050 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23bb2070 0x558a23f6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 --2- 172.21.15.83:0/3123711292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b891b0 0x558a23f65610 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 >> 172.21.15.83:0/3123711292 conn(0x558a23ba6460 msgr2=0x558a23f5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafa5ffb640 0 client.17649 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 shutdown_connections 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 1 -- 172.21.15.83:0/3123711292 wait complete. 2024-09-21T03:30:16.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.640+0000 7fafcac16740 20 client.17649 trim_cache size 0 max 16384 2024-09-21T03:30:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 Processor -- start 2024-09-21T03:30:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 -- start start 2024-09-21T03:30:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23bb2070 0x558a23f57100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafa000e670 con 0x558a23bb2070 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00101b0 con 0x558a23b91390 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafcac16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafa000ed40 con 0x558a23b891b0 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc243e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc243e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:55784/0 (socket says 172.21.15.83:55784) 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f56bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45778/0 (socket says 172.21.15.83:45778) 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc243e640 1 -- 172.21.15.83:0/3655643357 learned_addr learned my addr 172.21.15.83:0/3655643357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.657+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23bb2070 0x558a23f57100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 -- 172.21.15.83:0/3655643357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 msgr2=0x558a23f56bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f56bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 -- 172.21.15.83:0/3655643357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23bb2070 msgr2=0x558a23f57100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23bb2070 0x558a23f57100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 -- 172.21.15.83:0/3655643357 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a23f664b0 con 0x558a23b891b0 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafc243e640 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fafb80a9b00 tx=0x7fafb8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47854eea63ec1f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafa7fff640 1 -- 172.21.15.83:0/3655643357 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80ab950 con 0x558a23b891b0 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafa7fff640 1 -- 172.21.15.83:0/3655643357 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb80d0e50 con 0x558a23b891b0 2024-09-21T03:30:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.658+0000 7fafa7fff640 1 -- 172.21.15.83:0/3655643357 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80b6ef0 con 0x558a23b891b0 2024-09-21T03:30:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 -- 172.21.15.83:0/3655643357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 msgr2=0x558a23f5ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fafb80a9b00 tx=0x7fafb8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 -- 172.21.15.83:0/3655643357 shutdown_connections 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23bb2070 0x558a23f57100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23b91390 0x558a23f56bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 --2- 172.21.15.83:0/3655643357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f5ade0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 -- 172.21.15.83:0/3655643357 >> 172.21.15.83:0/3655643357 conn(0x558a23ba6460 msgr2=0x558a23f5a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 -- 172.21.15.83:0/3655643357 shutdown_connections 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.659+0000 7fafcac16740 1 -- 172.21.15.83:0/3655643357 wait complete. 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 -- start start 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 Processor -- start 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f95ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23ff19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafa000fae0 con 0x558a23b91390 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafa000f410 con 0x558a23bb2070 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafc243e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f95ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafc1c3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48198/0 (socket says 172.21.15.83:48198) 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafc1c3d640 1 -- 172.21.15.83:0/3755438840 learned_addr learned my addr 172.21.15.83:0/3755438840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.660+0000 7fafcac16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafa000c670 con 0x558a23b891b0 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafcac16740 10 client.0 _ll_register_callbacks cb 0x7fafb80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23ff19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 -- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 msgr2=0x558a23ff19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23ff19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 -- 172.21.15.83:0/3755438840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 msgr2=0x558a23f95ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 -- 172.21.15.83:0/3755438840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a23f968c0 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc2c3f640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23ff19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafc243e640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f95ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fafb80a9b00 tx=0x7fafb80c8d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8f0b7f143f147f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80ab950 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8485+0+0 (secure 0 0 0) 0x7fafb8010ca0 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb80d0e50 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.661+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fafb8010720 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 10 client.27148 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558a23ff2f80 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb80d0e50 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fafb80b6430 con 0x558a23b891b0 2024-09-21T03:30:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 10 client.27148 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 10 client.27148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 20 client.27148 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b45da0 0x558a23ff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 4 client.27148 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a23ecd8a0 con 0x558a23b45da0 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.662+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b45da0 0x558a23ff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafa5ffb640 10 client.27148 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafc1c3d640 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b45da0 0x558a23ff54d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a84f84ab5a8504 server_cookie=76670528716cf8e7 in_seq=0 out_seq=0 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafa5ffb640 1 -- 172.21.15.83:0/3755438840 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a23ecd8a0 con 0x558a23b45da0 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafa5ffb640 10 client.27148 handle_command_reply: tid=0 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 client.27148 shutdown 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 msgr2=0x558a23f95ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f95ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fafb80a9b00 tx=0x7fafb80c8d50 comp rx=0 tx=0).stop 2024-09-21T03:30:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 shutdown_connections 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558a23b45da0 0x558a23ff54d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a23bb2070 0x558a23ff19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafa5ffb640 0 client.27148 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a23b91390 0x558a23f95ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 --2- 172.21.15.83:0/3755438840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a23b891b0 0x558a23f95ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.663+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 >> 172.21.15.83:0/3755438840 conn(0x558a23ba6460 msgr2=0x558a23f699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 shutdown_connections 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafcac16740 1 -- 172.21.15.83:0/3755438840 wait complete. 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafcac16740 20 client.27148 trim_cache size 0 max 16384 2024-09-21T03:30:16.671 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 msgr2=0x7fafa80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 0x7fafa80b8d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005fd0 tx=0x7fafb4009500 comp rx=0 tx=0).stop 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 msgr2=0x7fafbc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.664+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e9f0 tx=0x7fafb800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 shutdown_connections 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fafa80b6880 0x7fafa80b8d90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc108d00 0x7fafbc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafbc1083b0 0x7fafbc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 --2- 172.21.15.83:0/375547987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafbc102ac0 0x7fafbc19a4e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 >> 172.21.15.83:0/375547987 conn(0x7fafbc079790 msgr2=0x7fafbc101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 shutdown_connections 2024-09-21T03:30:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:16.665+0000 7fafc46a2640 1 -- 172.21.15.83:0/375547987 wait complete. 2024-09-21T03:30:17.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:17 smithi193 ceph-mon[35224]: pgmap v748: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-21T03:30:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:17 smithi083 ceph-mon[31219]: pgmap v748: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-21T03:30:17.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:17 smithi142 ceph-mon[34512]: pgmap v748: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-21T03:30:18.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:30:18.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:30:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:19 smithi193 ceph-mon[35224]: pgmap v749: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-09-21T03:30:19.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:19 smithi193 ceph-mon[35224]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:30:19.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:19 smithi083 ceph-mon[31219]: pgmap v749: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-09-21T03:30:19.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:19 smithi083 ceph-mon[31219]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:30:19.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 -- 172.21.15.83:0/2123518939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350108cf0 msgr2=0x7ff35010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 --2- 172.21.15.83:0/2123518939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350108cf0 0x7ff35010f520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff348009ca0 tx=0x7ff34802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 -- 172.21.15.83:0/2123518939 shutdown_connections 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 --2- 172.21.15.83:0/2123518939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350108cf0 0x7ff35010f520 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 --2- 172.21.15.83:0/2123518939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3501083a0 0x7ff3501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 --2- 172.21.15.83:0/2123518939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff350102a60 0x7ff350102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.591+0000 7ff3574e0640 1 -- 172.21.15.83:0/2123518939 >> 172.21.15.83:0/2123518939 conn(0x7ff350079730 msgr2=0x7ff350079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.592+0000 7ff3574e0640 1 -- 172.21.15.83:0/2123518939 shutdown_connections 2024-09-21T03:30:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.592+0000 7ff3574e0640 1 -- 172.21.15.83:0/2123518939 wait complete. 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.592+0000 7ff3574e0640 1 Processor -- start 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.592+0000 7ff3574e0640 1 -- start start 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 0x7ff35019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff350112960 con 0x7ff350102a60 2024-09-21T03:30:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501127e0 con 0x7ff350108cf0 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff3574e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff350112ae0 con 0x7ff3501083a0 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff355a7d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51144/0 (socket says 172.21.15.83:51144) 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff355a7d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50426/0 (socket says 172.21.15.83:50426) 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff355a7d640 1 -- 172.21.15.83:0/268755390 learned_addr learned my addr 172.21.15.83:0/268755390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.593+0000 7ff35527c640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 0x7ff35019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 -- 172.21.15.83:0/268755390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 msgr2=0x7ff35019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 -- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 msgr2=0x7ff35019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 0x7ff35019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 -- 172.21.15.83:0/268755390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3501a6530 con 0x7ff3501083a0 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff35527c640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 0x7ff35019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.594+0000 7ff354a7b640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff34000e9c0 tx=0x7ff34000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.595+0000 7ff355a7d640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.595+0000 7ff3467fc640 1 -- 172.21.15.83:0/268755390 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34000ccc0 con 0x7ff3501083a0 2024-09-21T03:30:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.595+0000 7ff3467fc640 1 -- 172.21.15.83:0/268755390 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400108e0 con 0x7ff3501083a0 2024-09-21T03:30:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.596+0000 7ff3467fc640 1 -- 172.21.15.83:0/268755390 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400187f0 con 0x7ff3501083a0 2024-09-21T03:30:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.596+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3501a6820 con 0x7ff3501083a0 2024-09-21T03:30:19.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.596+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501a6d10 con 0x7ff3501083a0 2024-09-21T03:30:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.599+0000 7ff3467fc640 1 -- 172.21.15.83:0/268755390 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff340010430 con 0x7ff3501083a0 2024-09-21T03:30:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff3467fc640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 0x7ff33c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 Processor -- start 2024-09-21T03:30:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 -- start start 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff3467fc640 1 -- 172.21.15.83:0/268755390 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7ff3400e7a80 con 0x7ff3501083a0 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35527c640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 0x7ff33c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x563290536430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329016e670 0x563290536eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632904ace60 con 0x56329018f350 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56329049c290 con 0x56329016e670 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35da54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632904a00a0 con 0x563290166490 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51158/0 (socket says 172.21.15.83:51158) 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff354a7b640 1 -- 172.21.15.83:0/2581798577 learned_addr learned my addr 172.21.15.83:0/2581798577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.600+0000 7ff35527c640 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x563290536430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff355a7d640 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329016e670 0x563290536eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 -- 172.21.15.83:0/2581798577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329016e670 msgr2=0x563290536eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329016e670 0x563290536eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 -- 172.21.15.83:0/2581798577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 msgr2=0x563290536430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x563290536430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 -- 172.21.15.83:0/2581798577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632901fa930 con 0x563290166490 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a9ef0 tx=0x7ff3400ad150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef64059ee1cdc4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff35527c640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 0x7ff33c0b8ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff35006b9e0 tx=0x7ff34c013820 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff33b7fe640 1 -- 172.21.15.83:0/2581798577 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400b2020 con 0x563290166490 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff33b7fe640 1 -- 172.21.15.83:0/2581798577 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3400ae020 con 0x563290166490 2024-09-21T03:30:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.601+0000 7ff33b7fe640 1 -- 172.21.15.83:0/2581798577 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400b7340 con 0x563290166490 2024-09-21T03:30:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.602+0000 7ff35da54740 1 -- 172.21.15.83:0/2581798577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 msgr2=0x563290536970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.602+0000 7ff35da54740 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3400a9ef0 tx=0x7ff3400ad150 comp rx=0 tx=0).stop 2024-09-21T03:30:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 -- 172.21.15.83:0/2581798577 shutdown_connections 2024-09-21T03:30:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329016e670 0x563290536eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563290166490 0x563290536970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 --2- 172.21.15.83:0/2581798577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x563290536430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 -- 172.21.15.83:0/2581798577 >> 172.21.15.83:0/2581798577 conn(0x563290183740 msgr2=0x563290534320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 -- 172.21.15.83:0/2581798577 shutdown_connections 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 -- 172.21.15.83:0/2581798577 wait complete. 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.603+0000 7ff35da54740 1 -- start start 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 Processor -- start 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x563290546430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290546970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290552240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632902fd890 con 0x563290166490 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563290436d40 con 0x56329018f350 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563290548f10 con 0x56329016e670 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35da54740 10 client.0 _ll_register_callbacks cb 0x56329049ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35527c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x563290546430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290546970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35527c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x563290546430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54194/0 (socket says 172.21.15.83:54194) 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff35527c640 1 -- 172.21.15.83:0/3279375352 learned_addr learned my addr 172.21.15.83:0/3279375352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.604+0000 7ff355a7d640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290552240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 -- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 msgr2=0x563290552240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290552240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 -- 172.21.15.83:0/3279375352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 msgr2=0x563290546430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x563290546430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 -- 172.21.15.83:0/3279375352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563290552830 con 0x56329016e670 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff354a7b640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290546970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3400ad8b0 tx=0x7ff3400ad1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4738309ad71e6bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400c7070 con 0x56329016e670 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7ff3400a9270 con 0x56329016e670 2024-09-21T03:30:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3400ae020 con 0x56329016e670 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff3400cc020 con 0x56329016e670 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff35da54740 10 client.27160 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563290555140 con 0x56329016e670 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.605+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff340098070 con 0x56329016e670 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7ff3400ac5f0 con 0x56329016e670 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 10 client.27160 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 10 client.27160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 20 client.27160 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56329014dcd0 0x5632905584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 4 client.27160 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff3400ae020 con 0x56329014dcd0 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35527c640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56329014dcd0 0x5632905584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff338ff9640 10 client.27160 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.606+0000 7ff35527c640 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56329014dcd0 0x5632905584a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b71cbb12a2b559 server_cookie=8000707c39b246af in_seq=0 out_seq=0 2024-09-21T03:30:19.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff338ff9640 1 -- 172.21.15.83:0/3279375352 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff3400ae020 con 0x56329014dcd0 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff338ff9640 10 client.27160 handle_command_reply: tid=0 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 client.27160 shutdown 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 msgr2=0x563290546970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290546970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3400ad8b0 tx=0x7ff3400ad1c0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 shutdown_connections 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56329014dcd0 0x5632905584a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff338ff9640 0 client.27160 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290552240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290546970 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 --2- 172.21.15.83:0/3279375352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x563290546430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.607+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 >> 172.21.15.83:0/3279375352 conn(0x563290183740 msgr2=0x563290537e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.608+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 shutdown_connections 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.608+0000 7ff35da54740 1 -- 172.21.15.83:0/3279375352 wait complete. 2024-09-21T03:30:19.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.608+0000 7ff35da54740 20 client.27160 trim_cache size 0 max 16384 2024-09-21T03:30:19.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 Processor -- start 2024-09-21T03:30:19.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 -- start start 2024-09-21T03:30:19.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x5632905521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290543420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290543960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ad410 con 0x563290166490 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501aef50 con 0x56329018f350 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35da54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501adae0 con 0x56329016e670 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35527c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x5632905521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35527c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x5632905521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54204/0 (socket says 172.21.15.83:54204) 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290543420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff35527c640 1 -- 172.21.15.83:0/385676329 learned_addr learned my addr 172.21.15.83:0/385676329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.630+0000 7ff355a7d640 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290543960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 -- 172.21.15.83:0/385676329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 msgr2=0x563290543960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290543960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 -- 172.21.15.83:0/385676329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 msgr2=0x5632905521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x5632905521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 -- 172.21.15.83:0/385676329 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632905391f0 con 0x56329016e670 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff355a7d640 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290543960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff354a7b640 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290543420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff3400aab90 tx=0x7ff340010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cd6c81adea561e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff33affd640 1 -- 172.21.15.83:0/385676329 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400c7070 con 0x56329016e670 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff33affd640 1 -- 172.21.15.83:0/385676329 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff340098070 con 0x56329016e670 2024-09-21T03:30:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.631+0000 7ff33affd640 1 -- 172.21.15.83:0/385676329 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400b3590 con 0x56329016e670 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- 172.21.15.83:0/385676329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 msgr2=0x563290543420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290543420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff3400aab90 tx=0x7ff340010ca0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- 172.21.15.83:0/385676329 shutdown_connections 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56329018f350 0x563290543960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290543420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 --2- 172.21.15.83:0/385676329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563290166490 0x5632905521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- 172.21.15.83:0/385676329 >> 172.21.15.83:0/385676329 conn(0x563290183740 msgr2=0x563290544ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- 172.21.15.83:0/385676329 shutdown_connections 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- 172.21.15.83:0/385676329 wait complete. 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.632+0000 7ff35da54740 1 -- start start 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 Processor -- start 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563290166490 0x563290558f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x5632905bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff355a7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x5632905bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35527c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563290166490 0x563290558f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff354a7b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:51194/0 (socket says 172.21.15.83:51194) 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff354a7b640 1 -- 172.21.15.83:0/2322368963 learned_addr learned my addr 172.21.15.83:0/2322368963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ae880 con 0x56329018f350 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ae1b0 con 0x563290166490 2024-09-21T03:30:19.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.633+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ab410 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff35da54740 10 client.0 _ll_register_callbacks cb 0x7ff340005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 -- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563290166490 msgr2=0x563290558f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563290166490 0x563290558f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 -- 172.21.15.83:0/2322368963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 msgr2=0x5632905bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x5632905bbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632905bc220 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff340098070 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff354a7b640 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff340000c00 tx=0x7ff3400b3840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ad1eeb8edf252a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff338ff9640 10 client.27166 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff338ff9640 1 -- 172.21.15.83:0/2322368963 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400c7070 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff338ff9640 1 -- 172.21.15.83:0/2322368963 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7ff3400c3ad0 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff338ff9640 1 -- 172.21.15.83:0/2322368963 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff3400a9090 con 0x56329016e670 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff35da54740 10 client.27166 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.634+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=50}) -- 0x5632905bc220 con 0x56329016e670 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff338ff9640 1 -- 172.21.15.83:0/2322368963 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7ff3400cf310 con 0x56329016e670 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 10 client.27166 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 20 client.27166 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563290123080 0x5632905c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 4 client.27166 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632904aedb0 con 0x563290123080 2024-09-21T03:30:19.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.635+0000 7ff35527c640 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563290123080 0x5632905c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:19.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.636+0000 7ff338ff9640 10 client.27166 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:19.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.636+0000 7ff35527c640 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563290123080 0x5632905c5640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316c847cd12af921 server_cookie=e7d187e73b203db2 in_seq=0 out_seq=0 2024-09-21T03:30:19.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.637+0000 7ff338ff9640 1 -- 172.21.15.83:0/2322368963 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632904aedb0 con 0x563290123080 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.637+0000 7ff338ff9640 10 client.27166 handle_command_reply: tid=0 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.637+0000 7ff35da54740 1 client.27166 shutdown 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 msgr2=0x563290559500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff340000c00 tx=0x7ff3400b3840 comp rx=0 tx=0).stop 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 shutdown_connections 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x563290123080 0x5632905c5640 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56329018f350 0x5632905bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff338ff9640 0 client.27166 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329016e670 0x563290559500 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 --2- 172.21.15.83:0/2322368963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563290166490 0x563290558f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 >> 172.21.15.83:0/2322368963 conn(0x563290183740 msgr2=0x563290532850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 shutdown_connections 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 1 -- 172.21.15.83:0/2322368963 wait complete. 2024-09-21T03:30:19.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.638+0000 7ff35da54740 20 client.27166 trim_cache size 0 max 16384 2024-09-21T03:30:19.643 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:30:19.643 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:30:19.643 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "fec9277f-ee8e-4d8f-a1da-951b855d3d0e", 2024-09-21T03:30:19.643 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:30:19.643 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 msgr2=0x7ff33c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 0x7ff33c0b8ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff35006b9e0 tx=0x7ff34c013820 comp rx=0 tx=0).stop 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 msgr2=0x7ff35019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff34000e9c0 tx=0x7ff34000ee80 comp rx=0 tx=0).stop 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 shutdown_connections 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff33c0b67d0 0x7ff33c0b8ce0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350108cf0 0x7ff35019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3501083a0 0x7ff35019b7b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 --2- 172.21.15.83:0/268755390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff350102a60 0x7ff35019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:19.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 >> 172.21.15.83:0/268755390 conn(0x7ff350079730 msgr2=0x7ff350104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:19.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 shutdown_connections 2024-09-21T03:30:19.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:19.639+0000 7ff3574e0640 1 -- 172.21.15.83:0/268755390 wait complete. 2024-09-21T03:30:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:19 smithi142 ceph-mon[34512]: pgmap v749: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.7 MiB/s wr, 750 op/s 2024-09-21T03:30:19.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:19 smithi142 ceph-mon[34512]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:30:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:20 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:20 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:20 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:20 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:20.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:20 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:20 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:20.819 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 780.0 s, 1169.1 tps, lat 267.731 ms stddev 1465.191 2024-09-21T03:30:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 -- 172.21.15.83:0/3415119672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 msgr2=0x7f2654110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 --2- 172.21.15.83:0/3415119672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f2654110330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f264c009ca0 tx=0x7f264c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 -- 172.21.15.83:0/3415119672 shutdown_connections 2024-09-21T03:30:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 --2- 172.21.15.83:0/3415119672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f2654110330 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 --2- 172.21.15.83:0/3415119672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 0x7f265410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 --2- 172.21.15.83:0/3415119672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654104a30 0x7f2654104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 -- 172.21.15.83:0/3415119672 >> 172.21.15.83:0/3415119672 conn(0x7f2654079790 msgr2=0x7f2654079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 -- 172.21.15.83:0/3415119672 shutdown_connections 2024-09-21T03:30:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.399+0000 7f2658ed2640 1 -- 172.21.15.83:0/3415119672 wait complete. 2024-09-21T03:30:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 Processor -- start 2024-09-21T03:30:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 -- start start 2024-09-21T03:30:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654104a30 0x7f265419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 0x7f265419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2654113ac0 con 0x7f265410de60 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2654113940 con 0x7f2654105380 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2658ed2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2654113c40 con 0x7f2654104a30 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2652d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2652d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54228/0 (socket says 172.21.15.83:54228) 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2652d9d640 1 -- 172.21.15.83:0/2344805012 learned_addr learned my addr 172.21.15.83:0/2344805012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.400+0000 7f2651d9b640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 0x7f265419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 -- 172.21.15.83:0/2344805012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654104a30 msgr2=0x7f265419a430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654104a30 0x7f265419a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 -- 172.21.15.83:0/2344805012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 msgr2=0x7f265419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 0x7f265419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 -- 172.21.15.83:0/2344805012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26541a03a0 con 0x7f265410de60 2024-09-21T03:30:21.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2652d9d640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f264c0098d0 tx=0x7f264c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f26437fe640 1 -- 172.21.15.83:0/2344805012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c003b70 con 0x7f265410de60 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26541a0630 con 0x7f265410de60 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f26437fe640 1 -- 172.21.15.83:0/2344805012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c003d10 con 0x7f265410de60 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.401+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541a6df0 con 0x7f265410de60 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.402+0000 7f26437fe640 1 -- 172.21.15.83:0/2344805012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c02fa20 con 0x7f265410de60 2024-09-21T03:30:21.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.404+0000 7f265f446740 1 Processor -- start 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.404+0000 7f26437fe640 1 -- 172.21.15.83:0/2344805012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f264c02fd00 con 0x7f265410de60 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.404+0000 7f265f446740 1 -- start start 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.404+0000 7f265f446740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.404+0000 7f265f446740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265f446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563025649390 0x563025a11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265f446740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563025987e50 con 0x5630256411b0 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265f446740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563025977280 con 0x56302566a070 2024-09-21T03:30:21.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265f446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56302597b090 con 0x563025649390 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54236/0 (socket says 172.21.15.83:54236) 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 -- 172.21.15.83:0/640254160 learned_addr learned my addr 172.21.15.83:0/640254160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f2651d9b640 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563025649390 0x563025a11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f26437fe640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 0x7f263c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 -- 172.21.15.83:0/640254160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563025649390 msgr2=0x563025a11cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563025649390 0x563025a11cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 -- 172.21.15.83:0/640254160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 msgr2=0x563025a11270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a11270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 -- 172.21.15.83:0/640254160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630256d5650 con 0x5630256411b0 2024-09-21T03:30:21.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f2652d9d640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 0x7f263c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f26437fe640 1 -- 172.21.15.83:0/2344805012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f264c1089b0 con 0x7f265410de60 2024-09-21T03:30:21.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.405+0000 7f265259c640 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2644009bd0 tx=0x7f264400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f539b52b6af687 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f2640ff9640 1 -- 172.21.15.83:0/640254160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644005020 con 0x5630256411b0 2024-09-21T03:30:21.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f2640ff9640 1 -- 172.21.15.83:0/640254160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440047a0 con 0x5630256411b0 2024-09-21T03:30:21.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f2640ff9640 1 -- 172.21.15.83:0/640254160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440186d0 con 0x5630256411b0 2024-09-21T03:30:21.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f265f446740 1 -- 172.21.15.83:0/640254160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 msgr2=0x563025a117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f265f446740 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2644009bd0 tx=0x7f264400e720 comp rx=0 tx=0).stop 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.406+0000 7f2652d9d640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 0x7f263c0b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f264c0ce7e0 tx=0x7f264c03e040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 -- 172.21.15.83:0/640254160 shutdown_connections 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563025649390 0x563025a11cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a117b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 --2- 172.21.15.83:0/640254160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 -- 172.21.15.83:0/640254160 >> 172.21.15.83:0/640254160 conn(0x56302565e460 msgr2=0x563025a0f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 -- 172.21.15.83:0/640254160 shutdown_connections 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 -- 172.21.15.83:0/640254160 wait complete. 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 -- start start 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.407+0000 7f265f446740 1 Processor -- start 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 0x563025a1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a1d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630257d8700 con 0x563025649390 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563025912110 con 0x5630256411b0 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563025a1fa50 con 0x56302566a070 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265f446740 10 client.0 _ll_register_callbacks cb 0x56302592b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 0x563025a1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54238/0 (socket says 172.21.15.83:54238) 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265259c640 1 -- 172.21.15.83:0/735083037 learned_addr learned my addr 172.21.15.83:0/735083037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a1d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 -- 172.21.15.83:0/735083037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 msgr2=0x563025a1d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a1d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 -- 172.21.15.83:0/735083037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 msgr2=0x563025a1ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f2651d9b640 1 -- 172.21.15.83:0/735083037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302593fc40 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.408+0000 7f265259c640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2651d9b640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 0x563025a1c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f265410fb50 tx=0x7f264800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82549e98f9e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800ebb0 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f264800ed50 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26440047a0 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2648004020 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f265f446740 10 client.27215 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563025a2b700 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.409+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440047a0 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f26480047a0 con 0x5630256411b0 2024-09-21T03:30:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 10 client.27215 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 10 client.27215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 20 client.27215 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630256289f0 0x563025a2eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 4 client.27215 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563025989da0 con 0x5630256289f0 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265259c640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630256289f0 0x563025a2eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f2639ffb640 10 client.27215 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.410+0000 7f265259c640 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630256289f0 0x563025a2eba0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73b8cf15058c964 server_cookie=fb641cd4e5b7fac in_seq=0 out_seq=0 2024-09-21T03:30:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f2639ffb640 1 -- 172.21.15.83:0/735083037 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563025989da0 con 0x5630256289f0 2024-09-21T03:30:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f2639ffb640 10 client.27215 handle_command_reply: tid=0 2024-09-21T03:30:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 client.27215 shutdown 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 msgr2=0x563025a1c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 0x563025a1c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f265410fb50 tx=0x7f264800bbb0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 shutdown_connections 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630256289f0 0x563025a2eba0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a1d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 --2- 172.21.15.83:0/735083037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5630256411b0 0x563025a1c8f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 >> 172.21.15.83:0/735083037 conn(0x56302565e460 msgr2=0x563025a14400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.411+0000 7f2639ffb640 0 client.27215 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.412+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 shutdown_connections 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.412+0000 7f265f446740 1 -- 172.21.15.83:0/735083037 wait complete. 2024-09-21T03:30:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.412+0000 7f265f446740 20 client.27215 trim_cache size 0 max 16384 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.431+0000 7f265f446740 1 Processor -- start 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 -- start start 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26541b1810 con 0x5630256411b0 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26541b3350 con 0x563025649390 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265f446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26541b1ee0 con 0x56302566a070 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54250/0 (socket says 172.21.15.83:54250) 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:50496/0 (socket says 172.21.15.83:50496) 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 -- 172.21.15.83:0/3638769208 learned_addr learned my addr 172.21.15.83:0/3638769208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 -- 172.21.15.83:0/3638769208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 msgr2=0x563025a0e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a0e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 -- 172.21.15.83:0/3638769208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 msgr2=0x563025a129b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f2651d9b640 1 -- 172.21.15.83:0/3638769208 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563025a28c70 con 0x5630256411b0 2024-09-21T03:30:21.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.432+0000 7f265259c640 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:21.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.433+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f264800b670 tx=0x7f2648001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b049d372bdfc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.433+0000 7f263bfff640 1 -- 172.21.15.83:0/3638769208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480043c0 con 0x5630256411b0 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.433+0000 7f263bfff640 1 -- 172.21.15.83:0/3638769208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264800ebb0 con 0x5630256411b0 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.433+0000 7f263bfff640 1 -- 172.21.15.83:0/3638769208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648003830 con 0x5630256411b0 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- 172.21.15.83:0/3638769208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 msgr2=0x563025a1e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f264800b670 tx=0x7f2648001e50 comp rx=0 tx=0).stop 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- 172.21.15.83:0/3638769208 shutdown_connections 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56302566a070 0x563025a0e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563025649390 0x563025a129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 --2- 172.21.15.83:0/3638769208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630256411b0 0x563025a1e7d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- 172.21.15.83:0/3638769208 >> 172.21.15.83:0/3638769208 conn(0x56302565e460 msgr2=0x563025a122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- 172.21.15.83:0/3638769208 shutdown_connections 2024-09-21T03:30:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- 172.21.15.83:0/3638769208 wait complete. 2024-09-21T03:30:21.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.434+0000 7f265f446740 1 -- start start 2024-09-21T03:30:21.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 Processor -- start 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630256411b0 0x563025a91720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26541b2c80 con 0x563025649390 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26541b25b0 con 0x56302566a070 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26541af810 con 0x5630256411b0 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265f446740 10 client.0 _ll_register_callbacks cb 0x7f2648012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.435+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54260/0 (socket says 172.21.15.83:54260) 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 -- 172.21.15.83:0/3681905224 learned_addr learned my addr 172.21.15.83:0/3681905224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630256411b0 0x563025a91720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 -- 172.21.15.83:0/3681905224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630256411b0 msgr2=0x563025a91720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630256411b0 0x563025a91720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 -- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 msgr2=0x563025a24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a24b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.436+0000 7f265259c640 1 -- 172.21.15.83:0/3681905224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563025a25170 con 0x563025649390 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.437+0000 7f265259c640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f264400ec20 tx=0x7f26440058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9f9a9b904356d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.438+0000 7f2639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.438+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264402f020 con 0x563025649390 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.438+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264800ebb0 con 0x563025649390 2024-09-21T03:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.438+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f2644004040 con 0x563025649390 2024-09-21T03:30:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.446+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f264403a050 con 0x563025649390 2024-09-21T03:30:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.446+0000 7f265f446740 10 client.17691 fetch_fsmap learned FSMap version 50 2024-09-21T03:30:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.446+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563025a98fa0 con 0x563025649390 2024-09-21T03:30:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2644009830 con 0x563025649390 2024-09-21T03:30:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7f2644003be0 con 0x563025649390 2024-09-21T03:30:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f265f446740 10 client.17691 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T03:30:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f265f446740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f265f446740 20 client.17691 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630255fdda0 0x563025a9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.449+0000 7f265f446740 4 client.17691 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:21.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.451+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f264800ebb0 con 0x5630255fdda0 2024-09-21T03:30:21.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.464+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630255fdda0 0x563025a9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.476+0000 7f2651d9b640 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630255fdda0 0x563025a9b570 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb60fb294acfb2d server_cookie=5c6499da2fd0ae08 in_seq=0 out_seq=0 2024-09-21T03:30:21.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.477+0000 7f2639ffb640 10 client.17691 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:21.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.479+0000 7f2639ffb640 1 -- 172.21.15.83:0/3681905224 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5120 (crc 0 0 0) 0x7f264800ebb0 con 0x5630255fdda0 2024-09-21T03:30:21.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.479+0000 7f2639ffb640 10 client.17691 handle_command_reply: tid=0 2024-09-21T03:30:21.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.479+0000 7f265f446740 1 client.17691 shutdown 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 msgr2=0x563025a91d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f264400ec20 tx=0x7f26440058f0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 shutdown_connections 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5630255fdda0 0x563025a9b570 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302566a070 0x563025a24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563025649390 0x563025a91d00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f2639ffb640 0 client.17691 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:21.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 --2- 172.21.15.83:0/3681905224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630256411b0 0x563025a91720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 >> 172.21.15.83:0/3681905224 conn(0x56302565e460 msgr2=0x563025a12c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.481+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 shutdown_connections 2024-09-21T03:30:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.482+0000 7f265f446740 1 -- 172.21.15.83:0/3681905224 wait complete. 2024-09-21T03:30:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.482+0000 7f265f446740 20 client.17691 trim_cache size 0 max 16384 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "fec9277f-ee8e-4d8f-a1da-951b855d3d0e", 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.487 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.488 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.490 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:30:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.493 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:30:21.495 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:21.496 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:30:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 msgr2=0x7f263c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 0x7f263c0b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f264c0ce7e0 tx=0x7f264c03e040 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 msgr2=0x7f265419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f264c0098d0 tx=0x7f264c0317b0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 shutdown_connections 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f263c0b6680 0x7f263c0b8b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f265410de60 0x7f265419fc70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2654105380 0x7f265419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 --2- 172.21.15.83:0/2344805012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654104a30 0x7f265419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 >> 172.21.15.83:0/2344805012 conn(0x7f2654079790 msgr2=0x7f26540ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 shutdown_connections 2024-09-21T03:30:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:21.485+0000 7f2658ed2640 1 -- 172.21.15.83:0/2344805012 wait complete. 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: pgmap v751: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:21.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:21.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:21 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: pgmap v751: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:21.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:21 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: pgmap v751: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:21.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:21 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:22.283 INFO:tasks.cephfs.filesystem:scrub status for tag:fec9277f-ee8e-4d8f-a1da-951b855d3d0e - {'path': '/', 'tag': 'fec9277f-ee8e-4d8f-a1da-951b855d3d0e', 'options': 'recursive,force'} 2024-09-21T03:30:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:22 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:30:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:22 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:22.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:22 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:30:22.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:22 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:22 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:30:22.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:22 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:23 smithi193 ceph-mon[35224]: pgmap v752: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-09-21T03:30:23.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:23 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:23 smithi083 ceph-mon[31219]: pgmap v752: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-09-21T03:30:23.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:23 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:23 smithi142 ceph-mon[34512]: pgmap v752: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-09-21T03:30:23.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:23 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:25.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:25 smithi193 ceph-mon[35224]: pgmap v753: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T03:30:25.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:25 smithi083 ceph-mon[31219]: pgmap v753: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T03:30:25.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:25 smithi142 ceph-mon[34512]: pgmap v753: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T03:30:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 785.0 s, 1413.0 tps, lat 71.388 ms stddev 81.179 2024-09-21T03:30:27.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:27 smithi193 ceph-mon[35224]: pgmap v754: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-09-21T03:30:27.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:27 smithi083 ceph-mon[31219]: pgmap v754: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-09-21T03:30:27.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:27 smithi142 ceph-mon[34512]: pgmap v754: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: pgmap v755: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:30:29.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:29 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:30:29.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: pgmap v755: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-21T03:30:29.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:30:29.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:30:29.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:30:29.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:30:29.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:29 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:30:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: pgmap v755: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-21T03:30:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:30:29.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:30:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:30:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:30:29.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:29 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:30:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 790.0 s, 2061.0 tps, lat 48.738 ms stddev 80.273 2024-09-21T03:30:31.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:31 smithi193 ceph-mon[35224]: pgmap v756: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-09-21T03:30:31.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:31 smithi083 ceph-mon[31219]: pgmap v756: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-09-21T03:30:31.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:31 smithi142 ceph-mon[34512]: pgmap v756: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-09-21T03:30:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:33 smithi193 ceph-mon[35224]: pgmap v757: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T03:30:33.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:33 smithi083 ceph-mon[31219]: pgmap v757: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T03:30:33.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:33 smithi142 ceph-mon[34512]: pgmap v757: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T03:30:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:35 smithi193 ceph-mon[35224]: pgmap v758: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:30:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:35 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:30:35.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:35 smithi083 ceph-mon[31219]: pgmap v758: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:30:35.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:35 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:30:35.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:35 smithi142 ceph-mon[34512]: pgmap v758: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-09-21T03:30:35.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:35 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:30:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 795.0 s, 1272.2 tps, lat 78.610 ms stddev 95.289 2024-09-21T03:30:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:37 smithi083 ceph-mon[31219]: pgmap v759: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-09-21T03:30:37.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:37 smithi193 ceph-mon[35224]: pgmap v759: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-09-21T03:30:37.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:37 smithi142 ceph-mon[34512]: pgmap v759: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-09-21T03:30:39.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:39 smithi083 ceph-mon[31219]: pgmap v760: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T03:30:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:39 smithi193 ceph-mon[35224]: pgmap v760: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T03:30:39.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:39 smithi142 ceph-mon[34512]: pgmap v760: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T03:30:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 800.0 s, 1480.6 tps, lat 66.788 ms stddev 81.826 2024-09-21T03:30:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:41 smithi193 ceph-mon[35224]: pgmap v761: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:41.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:41 smithi083 ceph-mon[31219]: pgmap v761: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:41.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:41 smithi142 ceph-mon[34512]: pgmap v761: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:43 smithi193 ceph-mon[35224]: pgmap v762: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-09-21T03:30:43.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:43 smithi083 ceph-mon[31219]: pgmap v762: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-09-21T03:30:43.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:43 smithi142 ceph-mon[34512]: pgmap v762: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-09-21T03:30:45.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:45 smithi083 ceph-mon[31219]: pgmap v763: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:45.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:45 smithi142 ceph-mon[34512]: pgmap v763: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:45.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:45 smithi193 ceph-mon[35224]: pgmap v763: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T03:30:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 805.0 s, 1225.0 tps, lat 81.127 ms stddev 90.322 2024-09-21T03:30:47.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:47 smithi083 ceph-mon[31219]: pgmap v764: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 1.33k op/s 2024-09-21T03:30:47.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:47 smithi142 ceph-mon[34512]: pgmap v764: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 1.33k op/s 2024-09-21T03:30:47.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:47 smithi193 ceph-mon[35224]: pgmap v764: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 1.33k op/s 2024-09-21T03:30:49.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:49 smithi083 ceph-mon[31219]: pgmap v765: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:49.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:49 smithi142 ceph-mon[34512]: pgmap v765: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:49.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:49 smithi193 ceph-mon[35224]: pgmap v765: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 810.0 s, 1349.8 tps, lat 74.931 ms stddev 86.808 2024-09-21T03:30:51.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:51 smithi083 ceph-mon[31219]: pgmap v766: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-21T03:30:51.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:51 smithi142 ceph-mon[34512]: pgmap v766: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-21T03:30:51.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:51 smithi193 ceph-mon[35224]: pgmap v766: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 1.28k op/s 2024-09-21T03:30:52.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:53.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:53 smithi083 ceph-mon[31219]: pgmap v767: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-09-21T03:30:53.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:53 smithi142 ceph-mon[34512]: pgmap v767: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-09-21T03:30:53.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:53 smithi193 ceph-mon[35224]: pgmap v767: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 1.30k op/s 2024-09-21T03:30:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.556+0000 7f98477fe640 1 -- 172.21.15.83:0/1663294871 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380318a0 con 0x7f9848111f40 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 -- 172.21.15.83:0/1663294871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 msgr2=0x7f984810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 --2- 172.21.15.83:0/1663294871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f984810a3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 -- 172.21.15.83:0/1663294871 shutdown_connections 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 --2- 172.21.15.83:0/1663294871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984811cc30 0x7f984811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 --2- 172.21.15.83:0/1663294871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984810a9e0 0x7f984810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 --2- 172.21.15.83:0/1663294871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f984810a3c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 -- 172.21.15.83:0/1663294871 >> 172.21.15.83:0/1663294871 conn(0x7f9848070ec0 msgr2=0x7f98480712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.557+0000 7f984eb51640 1 -- 172.21.15.83:0/1663294871 shutdown_connections 2024-09-21T03:30:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.558+0000 7f984eb51640 1 -- 172.21.15.83:0/1663294871 wait complete. 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 Processor -- start 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 -- start start 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f98481a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 0x7f98481aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9848122900 con 0x7f984810a9e0 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9848122780 con 0x7f984811cc30 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984eb51640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9848122a80 con 0x7f9848111f40 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.559+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 0x7f98481aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51066/0 (socket says 172.21.15.83:51066) 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984c8ed640 1 -- 172.21.15.83:0/4221435437 learned_addr learned my addr 172.21.15.83:0/4221435437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f9847fff640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f98481a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 -- 172.21.15.83:0/4221435437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 msgr2=0x7f98481a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f98481a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 -- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 msgr2=0x7f98481a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 -- 172.21.15.83:0/4221435437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98481af220 con 0x7f984811cc30 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984c8ed640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f9847fff640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f98481a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984d0ee640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 0x7f98481aeaf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f984000e940 tx=0x7f984000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f9845ffb640 1 -- 172.21.15.83:0/4221435437 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840004d20 con 0x7f984811cc30 2024-09-21T03:30:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f9845ffb640 1 -- 172.21.15.83:0/4221435437 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9840005020 con 0x7f984811cc30 2024-09-21T03:30:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.560+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481af510 con 0x7f984811cc30 2024-09-21T03:30:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.561+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481b5c70 con 0x7f984811cc30 2024-09-21T03:30:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.561+0000 7f9845ffb640 1 -- 172.21.15.83:0/4221435437 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400116d0 con 0x7f984811cc30 2024-09-21T03:30:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.566+0000 7f98550c5740 1 Processor -- start 2024-09-21T03:30:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.566+0000 7f9845ffb640 1 -- 172.21.15.83:0/4221435437 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9840011870 con 0x7f984811cc30 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 -- start start 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d044648e50 con 0x55d04430a390 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d044638280 con 0x55d04432b070 2024-09-21T03:30:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.567+0000 7f98550c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d04463c090 con 0x55d0443021b0 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9845ffb640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 0x7f98300b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60560/0 (socket says 172.21.15.83:60560) 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 -- 172.21.15.83:0/3811441501 learned_addr learned my addr 172.21.15.83:0/3811441501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9845ffb640 1 -- 172.21.15.83:0/4221435437 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f98400e75f0 con 0x7f984811cc30 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f984d0ee640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 0x7f98300b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 -- 172.21.15.83:0/3811441501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 msgr2=0x55d0446d27b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:30:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f984c8ed640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 -- 172.21.15.83:0/3811441501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 msgr2=0x55d0446d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.568+0000 7f9847fff640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f9847fff640 1 -- 172.21.15.83:0/3811441501 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d044396650 con 0x55d04432b070 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f984d0ee640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 0x7f98300b8c90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f98400a9ab0 tx=0x7f9840005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f984c8ed640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f9847fff640 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f983c011800 tx=0x7f983c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94058ae59faffa37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f982effd640 1 -- 172.21.15.83:0/3811441501 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c004020 con 0x55d04432b070 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f982effd640 1 -- 172.21.15.83:0/3811441501 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c00fd70 con 0x55d04432b070 2024-09-21T03:30:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.569+0000 7f982effd640 1 -- 172.21.15.83:0/3811441501 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0055c0 con 0x55d04432b070 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 -- 172.21.15.83:0/3811441501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 msgr2=0x55d0446d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f983c011800 tx=0x7f983c005850 comp rx=0 tx=0).stop 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 -- 172.21.15.83:0/3811441501 shutdown_connections 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 --2- 172.21.15.83:0/3811441501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446d2270 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 -- 172.21.15.83:0/3811441501 >> 172.21.15.83:0/3811441501 conn(0x55d04431f460 msgr2=0x55d0446d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 -- 172.21.15.83:0/3811441501 shutdown_connections 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.570+0000 7f98550c5740 1 -- 172.21.15.83:0/3811441501 wait complete. 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 -- start start 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 Processor -- start 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d044499700 con 0x55d0443021b0 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d0445d3110 con 0x55d04430a390 2024-09-21T03:30:54.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0446e0a50 con 0x55d04432b070 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.571+0000 7f98550c5740 10 client.0 _ll_register_callbacks cb 0x55d0445ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51088/0 (socket says 172.21.15.83:51088) 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:60568/0 (socket says 172.21.15.83:60568) 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9847fff640 1 -- 172.21.15.83:0/3267580649 learned_addr learned my addr 172.21.15.83:0/3267580649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9847fff640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 -- 172.21.15.83:0/3267580649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 msgr2=0x55d0446e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 -- 172.21.15.83:0/3267580649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 msgr2=0x55d0446dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 -- 172.21.15.83:0/3267580649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d044600c40 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9847fff640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f984c8ed640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf824923a729dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9827fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035070 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c00fd70 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f9838003c50 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.572+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9838003e50 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 10 client.27233 fetch_fsmap learned FSMap version 51 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d0446ec7b0 con 0x55d04430a390 2024-09-21T03:30:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c00fd70 con 0x55d04430a390 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f98380314a0 con 0x55d04430a390 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 10 client.27233 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 10 client.27233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 20 client.27233 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442e99f0 0x55d0446f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 4 client.27233 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04464ada0 con 0x55d0442e99f0 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.573+0000 7f9847fff640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442e99f0 0x55d0446f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.574+0000 7f9847fff640 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442e99f0 0x55d0446f00b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7e13a6ec83f65a server_cookie=b80962ea0058264 in_seq=0 out_seq=0 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.574+0000 7f9827fff640 10 client.27233 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.574+0000 7f9827fff640 1 -- 172.21.15.83:0/3267580649 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d04464ada0 con 0x55d0442e99f0 2024-09-21T03:30:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.574+0000 7f9827fff640 10 client.27233 handle_command_reply: tid=0 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.574+0000 7f98550c5740 1 client.27233 shutdown 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 msgr2=0x55d0446dded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838003820 comp rx=0 tx=0).stop 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 shutdown_connections 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442e99f0 0x55d0446f00b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f9827fff640 0 client.27233 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446dded0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 --2- 172.21.15.83:0/3267580649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 >> 172.21.15.83:0/3267580649 conn(0x55d04431f460 msgr2=0x55d0446d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 shutdown_connections 2024-09-21T03:30:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 1 -- 172.21.15.83:0/3267580649 wait complete. 2024-09-21T03:30:54.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.575+0000 7f98550c5740 20 client.27233 trim_cache size 0 max 16384 2024-09-21T03:30:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.597+0000 7f98550c5740 1 Processor -- start 2024-09-21T03:30:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.597+0000 7f98550c5740 1 -- start start 2024-09-21T03:30:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.597+0000 7f98550c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f98550c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f98550c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f98550c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f983400e6a0 con 0x55d04430a390 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f98550c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98340101e0 con 0x55d04432b070 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f98550c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f983400ed70 con 0x55d0443021b0 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51096/0 (socket says 172.21.15.83:51096) 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 -- 172.21.15.83:0/1234137665 learned_addr learned my addr 172.21.15.83:0/1234137665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f9847fff640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f9847fff640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 -- 172.21.15.83:0/1234137665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 msgr2=0x55d0446dde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446dde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 -- 172.21.15.83:0/1234137665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 msgr2=0x55d0446dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f984c8ed640 1 -- 172.21.15.83:0/1234137665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0446de4d0 con 0x55d04430a390 2024-09-21T03:30:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.598+0000 7f9847fff640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.599+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9838004630 tx=0x7f983802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5502ca5b03d125 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.599+0000 7f982e7fc640 1 -- 172.21.15.83:0/1234137665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035070 con 0x55d04430a390 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.599+0000 7f982e7fc640 1 -- 172.21.15.83:0/1234137665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9838031bf0 con 0x55d04430a390 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.599+0000 7f982e7fc640 1 -- 172.21.15.83:0/1234137665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035070 con 0x55d04430a390 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 -- 172.21.15.83:0/1234137665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 msgr2=0x55d0446e9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9838004630 tx=0x7f983802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 -- 172.21.15.83:0/1234137665 shutdown_connections 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04432b070 0x55d0446dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d04430a390 0x55d0446e9cd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 --2- 172.21.15.83:0/1234137665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0443021b0 0x55d0446dde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 -- 172.21.15.83:0/1234137665 >> 172.21.15.83:0/1234137665 conn(0x55d04431f460 msgr2=0x55d0446dd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 -- 172.21.15.83:0/1234137665 shutdown_connections 2024-09-21T03:30:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.601+0000 7f98550c5740 1 -- 172.21.15.83:0/1234137665 wait complete. 2024-09-21T03:30:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 -- start start 2024-09-21T03:30:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 Processor -- start 2024-09-21T03:30:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446e2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f9847fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51106/0 (socket says 172.21.15.83:51106) 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f9847fff640 1 -- 172.21.15.83:0/1564517046 learned_addr learned my addr 172.21.15.83:0/1564517046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f983400fb10 con 0x55d0443021b0 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f983400f440 con 0x55d04430a390 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.602+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f983400c6a0 con 0x55d04432b070 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f98550c5740 10 client.0 _ll_register_callbacks cb 0x7f9838009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 -- 172.21.15.83:0/1564517046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 msgr2=0x55d0446e27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 -- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 msgr2=0x55d0446e2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446e2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d044759aa0 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446e2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9847fff640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f983c005130 tx=0x7f9838004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2403fc4473cd78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9827fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.603+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c027040 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f983c0035f0 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838031bf0 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f983c003810 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 10 client.17709 fetch_fsmap learned FSMap version 51 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d04475acf0 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c02f450 con 0x55d0443021b0 2024-09-21T03:30:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f983c041480 con 0x55d0443021b0 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 10 client.17709 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 20 client.17709 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442beda0 0x55d04475d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 4 client.17709 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.604+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9838031bf0 con 0x55d0442beda0 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.605+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442beda0 0x55d04475d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.605+0000 7f9827fff640 10 client.17709 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.605+0000 7f984c8ed640 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442beda0 0x55d04475d210 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb741d5e1a9aa64a server_cookie=e930d3d1e4857209 in_seq=0 out_seq=0 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f9827fff640 1 -- 172.21.15.83:0/1564517046 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x7f9838031bf0 con 0x55d0442beda0 2024-09-21T03:30:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f9827fff640 10 client.17709 handle_command_reply: tid=0 2024-09-21T03:30:54.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 client.17709 shutdown 2024-09-21T03:30:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 msgr2=0x55d0446e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f983c005130 tx=0x7f9838004430 comp rx=0 tx=0).stop 2024-09-21T03:30:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 shutdown_connections 2024-09-21T03:30:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d0442beda0 0x55d04475d210 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f9827fff640 0 client.17709 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d04432b070 0x55d0446e27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d04430a390 0x55d0446e2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 --2- 172.21.15.83:0/1564517046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0443021b0 0x55d0446e1d30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 >> 172.21.15.83:0/1564517046 conn(0x55d04431f460 msgr2=0x55d0446d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.606+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 shutdown_connections 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f98550c5740 1 -- 172.21.15.83:0/1564517046 wait complete. 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f98550c5740 20 client.17709 trim_cache size 0 max 16384 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:30:54.613 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.618 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:30:54.619 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.621 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.622 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.623 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.624 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:30:54.625 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:30:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 msgr2=0x7f98300b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 0x7f98300b8c90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f98400a9ab0 tx=0x7f9840005f50 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 msgr2=0x7f98481aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 0x7f98481aeaf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f984000e940 tx=0x7f984000ee00 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 shutdown_connections 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f98300b6780 0x7f98300b8c90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f984811cc30 0x7f98481aeaf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9848111f40 0x7f98481a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 --2- 172.21.15.83:0/4221435437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984810a9e0 0x7f98481a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.607+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 >> 172.21.15.83:0/4221435437 conn(0x7f9848070ec0 msgr2=0x7f9848071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.608+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 shutdown_connections 2024-09-21T03:30:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:54.608+0000 7f984eb51640 1 -- 172.21.15.83:0/4221435437 wait complete. 2024-09-21T03:30:55.498 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:30:55.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:55.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:55 smithi083 ceph-mon[31219]: pgmap v768: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-09-21T03:30:55.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:55 smithi142 ceph-mon[34512]: pgmap v768: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-09-21T03:30:55.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:55 smithi193 ceph-mon[35224]: pgmap v768: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-09-21T03:30:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:55.942 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 815.0 s, 1364.5 tps, lat 72.285 ms stddev 109.481 2024-09-21T03:30:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.530+0000 7f2e77755640 1 -- 172.21.15.83:0/1045204554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e70111d40 msgr2=0x7f2e7010a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.530+0000 7f2e77755640 1 --2- 172.21.15.83:0/1045204554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e70111d40 0x7f2e7010a640 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009be0 tx=0x7f2e6802a6f0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.535+0000 7f2e77755640 1 -- 172.21.15.83:0/1045204554 shutdown_connections 2024-09-21T03:30:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.535+0000 7f2e77755640 1 --2- 172.21.15.83:0/1045204554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e7011cba0 0x7f2e7011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.535+0000 7f2e77755640 1 --2- 172.21.15.83:0/1045204554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7010ab80 0x7f2e7010af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.535+0000 7f2e77755640 1 --2- 172.21.15.83:0/1045204554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e70111d40 0x7f2e7010a640 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.535+0000 7f2e77755640 1 -- 172.21.15.83:0/1045204554 >> 172.21.15.83:0/1045204554 conn(0x7f2e7006d950 msgr2=0x7f2e7006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.537+0000 7f2e77755640 1 -- 172.21.15.83:0/1045204554 shutdown_connections 2024-09-21T03:30:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.537+0000 7f2e77755640 1 -- 172.21.15.83:0/1045204554 wait complete. 2024-09-21T03:30:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.538+0000 7f2e77755640 1 Processor -- start 2024-09-21T03:30:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.538+0000 7f2e77755640 1 -- start start 2024-09-21T03:30:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.538+0000 7f2e77755640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e7010ab80 0x7f2e7011c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.538+0000 7f2e77755640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e77755640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7011cba0 0x7f2e70115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e77755640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70122720 con 0x7f2e7010ab80 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e74cf0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e74cf0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38872/0 (socket says 172.21.15.83:38872) 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e74cf0640 1 -- 172.21.15.83:0/491414670 learned_addr learned my addr 172.21.15.83:0/491414670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e75cf2640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7011cba0 0x7f2e70115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e77755640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701225a0 con 0x7f2e7011cba0 2024-09-21T03:30:56.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.539+0000 7f2e77755640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2e701228a0 con 0x7f2e70111d40 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 -- 172.21.15.83:0/491414670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7011cba0 msgr2=0x7f2e70115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7011cba0 0x7f2e70115e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 -- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e7010ab80 msgr2=0x7f2e7011c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e7010ab80 0x7f2e7011c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7011b170 con 0x7f2e70111d40 2024-09-21T03:30:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.542+0000 7f2e74cf0640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00d770 tx=0x7f2e6c00db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.546+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c017070 con 0x7f2e70111d40 2024-09-21T03:30:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.546+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6c010460 con 0x7f2e70111d40 2024-09-21T03:30:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.546+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00f620 con 0x7f2e70111d40 2024-09-21T03:30:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.546+0000 7f2e77755640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7011b460 con 0x7f2e70111d40 2024-09-21T03:30:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.546+0000 7f2e77755640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7011b890 con 0x7f2e70111d40 2024-09-21T03:30:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.548+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e540052c0 con 0x7f2e70111d40 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.553+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2e6c01a070 con 0x7f2e70111d40 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.554+0000 7f2e667fc640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 0x7f2e5c0b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.554+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f2e6c0e72f0 con 0x7f2e70111d40 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.557+0000 7f2e754f1640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 0x7f2e5c0b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.559+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2e6c0ad8b0 con 0x7f2e70111d40 2024-09-21T03:30:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.561+0000 7f2e754f1640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 0x7f2e5c0b8dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2e680099a0 tx=0x7f2e68000f40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.901+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e540055b0 con 0x7f2e70111d40 2024-09-21T03:30:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.902+0000 7f2e667fc640 1 -- 172.21.15.83:0/491414670 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11983 (secure 0 0 0) 0x7f2e6c0b1c90 con 0x7f2e70111d40 2024-09-21T03:30:56.904 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:30:56.904 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"btime":"2024-09-21T03:30:34:276749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:30:33.901771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:30:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-09-21T03:30:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 msgr2=0x7f2e5c0b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 0x7f2e5c0b8dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2e680099a0 tx=0x7f2e68000f40 comp rx=0 tx=0).stop 2024-09-21T03:30:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 msgr2=0x7f2e701157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00d770 tx=0x7f2e6c00db70 comp rx=0 tx=0).stop 2024-09-21T03:30:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 shutdown_connections 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2e5c0b68c0 0x7f2e5c0b8dd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7011cba0 0x7f2e70115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e70111d40 0x7f2e701157d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 --2- 172.21.15.83:0/491414670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e7010ab80 0x7f2e7011c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 >> 172.21.15.83:0/491414670 conn(0x7f2e7006d950 msgr2=0x7f2e701109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 shutdown_connections 2024-09-21T03:30:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:56.904+0000 7f2e5bfff640 1 -- 172.21.15.83:0/491414670 wait complete. 2024-09-21T03:30:57.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:57 smithi083 ceph-mon[31219]: pgmap v769: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:57.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:57 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/491414670' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:57 smithi142 ceph-mon[34512]: pgmap v769: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:57.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:57 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/491414670' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:57.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:57 smithi193 ceph-mon[35224]: pgmap v769: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:30:57.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:57 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/491414670' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:58.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:30:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 -- 172.21.15.83:0/2384502364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60441083a0 msgr2=0x7f60441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 --2- 172.21.15.83:0/2384502364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60441083a0 0x7f60441087b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6034009900 tx=0x7f603402a410 comp rx=0 tx=0).stop 2024-09-21T03:30:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 -- 172.21.15.83:0/2384502364 shutdown_connections 2024-09-21T03:30:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 --2- 172.21.15.83:0/2384502364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6044108cf0 0x7f604410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 --2- 172.21.15.83:0/2384502364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60441083a0 0x7f60441087b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 --2- 172.21.15.83:0/2384502364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044102a60 0x7f6044102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 -- 172.21.15.83:0/2384502364 >> 172.21.15.83:0/2384502364 conn(0x7f6044079730 msgr2=0x7f6044079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 -- 172.21.15.83:0/2384502364 shutdown_connections 2024-09-21T03:30:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.119+0000 7f604b786640 1 -- 172.21.15.83:0/2384502364 wait complete. 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 Processor -- start 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 -- start start 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441083a0 0x7f60441999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044108cf0 0x7f604419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112960 con 0x7f6044102a60 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60441127e0 con 0x7f6044108cf0 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f604b786640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112ae0 con 0x7f60441083a0 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f6049522640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f6049522640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51148/0 (socket says 172.21.15.83:51148) 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.120+0000 7f6049522640 1 -- 172.21.15.83:0/1628270289 learned_addr learned my addr 172.21.15.83:0/1628270289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6048d21640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441083a0 0x7f60441999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049d23640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044108cf0 0x7f604419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 -- 172.21.15.83:0/1628270289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441083a0 msgr2=0x7f60441999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441083a0 0x7f60441999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 -- 172.21.15.83:0/1628270289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044108cf0 msgr2=0x7f604419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044108cf0 0x7f604419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 -- 172.21.15.83:0/1628270289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419f2e0 con 0x7f6044102a60 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f6049522640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f604000ea60 tx=0x7f604000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f603a7fc640 1 -- 172.21.15.83:0/1628270289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000cde0 con 0x7f6044102a60 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419f5d0 con 0x7f6044102a60 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f603a7fc640 1 -- 172.21.15.83:0/1628270289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60400035f0 con 0x7f6044102a60 2024-09-21T03:30:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.121+0000 7f603a7fc640 1 -- 172.21.15.83:0/1628270289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040010690 con 0x7f6044102a60 2024-09-21T03:30:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.122+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604406ae60 con 0x7f6044102a60 2024-09-21T03:30:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.125+0000 7f603a7fc640 1 -- 172.21.15.83:0/1628270289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f60400037a0 con 0x7f6044102a60 2024-09-21T03:30:59.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.127+0000 7f6051cfa740 1 Processor -- start 2024-09-21T03:30:59.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.127+0000 7f6051cfa740 1 -- start start 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f0b1b0 0x5652a62db800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f13390 0x5652a62dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652a6251ea0 con 0x5652a5f34070 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5652a62412d0 con 0x5652a5f0b1b0 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6051cfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652a62450e0 con 0x5652a5f13390 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6049d23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6049d23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51154/0 (socket says 172.21.15.83:51154) 2024-09-21T03:30:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6049d23640 1 -- 172.21.15.83:0/313301953 learned_addr learned my addr 172.21.15.83:0/313301953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.128+0000 7f6049522640 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f0b1b0 0x5652a62db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 -- 172.21.15.83:0/313301953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f13390 msgr2=0x5652a62dbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f13390 0x5652a62dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 -- 172.21.15.83:0/313301953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f0b1b0 msgr2=0x5652a62db800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f0b1b0 0x5652a62db800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 -- 172.21.15.83:0/313301953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a5f9f650 con 0x5652a5f34070 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f603a7fc640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 0x7f60300b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f603a7fc640 1 -- 172.21.15.83:0/1628270289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f60400e7970 con 0x7f6044102a60 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f6049d23640 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f603c007ea0 tx=0x7f603c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753145ba16efe1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f602f7fe640 1 -- 172.21.15.83:0/313301953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c010840 con 0x5652a5f34070 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.129+0000 7f602f7fe640 1 -- 172.21.15.83:0/313301953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c0183f0 con 0x5652a5f34070 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.130+0000 7f602f7fe640 1 -- 172.21.15.83:0/313301953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c017600 con 0x5652a5f34070 2024-09-21T03:30:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.130+0000 7f6049522640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 0x7f60300b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.130+0000 7f6051cfa740 1 -- 172.21.15.83:0/313301953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 msgr2=0x5652a62db2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.130+0000 7f6051cfa740 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f603c007ea0 tx=0x7f603c007f90 comp rx=0 tx=0).stop 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 -- 172.21.15.83:0/313301953 shutdown_connections 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6049522640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 0x7f60300b8bc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5652a62d9f40 tx=0x7f6040003e90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f13390 0x5652a62dbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f0b1b0 0x5652a62db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 --2- 172.21.15.83:0/313301953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f34070 0x5652a62db2c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 -- 172.21.15.83:0/313301953 >> 172.21.15.83:0/313301953 conn(0x5652a5f28460 msgr2=0x5652a62d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 -- 172.21.15.83:0/313301953 shutdown_connections 2024-09-21T03:30:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 -- 172.21.15.83:0/313301953 wait complete. 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 -- start start 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.131+0000 7f6051cfa740 1 Processor -- start 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652a60a2700 con 0x5652a5f13390 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5652a61dc160 con 0x5652a5f34070 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652a62f22f0 con 0x5652a5f0b1b0 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6051cfa740 10 client.0 _ll_register_callbacks cb 0x5652a61f59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51164/0 (socket says 172.21.15.83:51164) 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 -- 172.21.15.83:0/1732629163 learned_addr learned my addr 172.21.15.83:0/1732629163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6049d23640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6049d23640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 -- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 msgr2=0x5652a62e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 -- 172.21.15.83:0/1732629163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 msgr2=0x5652a62e3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62e3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6048d21640 1 -- 172.21.15.83:0/1732629163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a6209c90 con 0x5652a5f13390 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.132+0000 7f6049d23640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62e3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f6048d21640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f604406aaa0 tx=0x7f6034032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caf2af90da0881c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f602cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034040070 con 0x5652a5f13390 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603c0183f0 con 0x5652a5f13390 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f60340318b0 con 0x5652a5f13390 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6034035040 con 0x5652a5f13390 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f6051cfa740 10 client.17724 fetch_fsmap learned FSMap version 51 2024-09-21T03:30:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5652a62fdf50 con 0x5652a5f13390 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.133+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c0183f0 con 0x5652a5f13390 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f6034004750 con 0x5652a5f13390 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 10 client.17724 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 10 client.17724 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 20 client.17724 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ef29f0 0x5652a63017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 4 client.17724 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652a6253df0 con 0x5652a5ef29f0 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.134+0000 7f6049d23640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ef29f0 0x5652a63017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.135+0000 7f6049d23640 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ef29f0 0x5652a63017f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c21cf6bceeb4c43 server_cookie=10840d31d8ecb90c in_seq=0 out_seq=0 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.135+0000 7f602cff9640 10 client.17724 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.135+0000 7f602cff9640 1 -- 172.21.15.83:0/1732629163 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652a6253df0 con 0x5652a5ef29f0 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.135+0000 7f602cff9640 10 client.17724 handle_command_reply: tid=0 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.135+0000 7f6051cfa740 1 client.17724 shutdown 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 msgr2=0x5652a62e2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f604406aaa0 tx=0x7f6034032e70 comp rx=0 tx=0).stop 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 shutdown_connections 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ef29f0 0x5652a63017f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62e3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62e2d50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 --2- 172.21.15.83:0/1732629163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 >> 172.21.15.83:0/1732629163 conn(0x5652a5f28460 msgr2=0x5652a62d7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f602cff9640 0 client.17724 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:59.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 shutdown_connections 2024-09-21T03:30:59.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 1 -- 172.21.15.83:0/1732629163 wait complete. 2024-09-21T03:30:59.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.136+0000 7f6051cfa740 20 client.17724 trim_cache size 0 max 16384 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 Processor -- start 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 -- start start 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60441ad410 con 0x5652a5f13390 2024-09-21T03:30:59.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60441aef50 con 0x5652a5f34070 2024-09-21T03:30:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6051cfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60441adae0 con 0x5652a5f0b1b0 2024-09-21T03:30:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51170/0 (socket says 172.21.15.83:51170) 2024-09-21T03:30:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6048d21640 1 -- 172.21.15.83:0/900856103 learned_addr learned my addr 172.21.15.83:0/900856103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6049d23640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.152+0000 7f6049d23640 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6048d21640 1 -- 172.21.15.83:0/900856103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 msgr2=0x5652a62d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6048d21640 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62d91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6048d21640 1 -- 172.21.15.83:0/900856103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 msgr2=0x5652a62dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6048d21640 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62dfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6048d21640 1 -- 172.21.15.83:0/900856103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a62dd7e0 con 0x5652a5f13390 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.153+0000 7f6049d23640 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.154+0000 7f6048d21640 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6034004b50 tx=0x7f6034001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7172d4748e1762a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.154+0000 7f602effd640 1 -- 172.21.15.83:0/900856103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034040070 con 0x5652a5f13390 2024-09-21T03:30:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.154+0000 7f602effd640 1 -- 172.21.15.83:0/900856103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60340318b0 con 0x5652a5f13390 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.154+0000 7f602effd640 1 -- 172.21.15.83:0/900856103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603404f480 con 0x5652a5f13390 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 -- 172.21.15.83:0/900856103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 msgr2=0x5652a62fb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6034004b50 tx=0x7f6034001e70 comp rx=0 tx=0).stop 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 -- 172.21.15.83:0/900856103 shutdown_connections 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62fb4e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 --2- 172.21.15.83:0/900856103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 -- 172.21.15.83:0/900856103 >> 172.21.15.83:0/900856103 conn(0x5652a5f28460 msgr2=0x5652a62d8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.155+0000 7f6051cfa740 1 -- 172.21.15.83:0/900856103 shutdown_connections 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.156+0000 7f6051cfa740 1 -- 172.21.15.83:0/900856103 wait complete. 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.156+0000 7f6051cfa740 1 -- start start 2024-09-21T03:30:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.156+0000 7f6051cfa740 1 Processor -- start 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60441ae880 con 0x5652a5f13390 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51172/0 (socket says 172.21.15.83:51172) 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 -- 172.21.15.83:0/3060261067 learned_addr learned my addr 172.21.15.83:0/3060261067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60441ae1b0 con 0x5652a5f34070 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60441ab410 con 0x5652a5f0b1b0 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6049d23640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6049d23640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6051cfa740 10 client.0 _ll_register_callbacks cb 0x7f6034009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 -- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 msgr2=0x5652a62f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 -- 172.21.15.83:0/3060261067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 msgr2=0x5652a62f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a62f4170 con 0x5652a5f13390 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6049d23640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f6048d21640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6034009230 tx=0x7f6034031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80898de80b630c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.157+0000 7f602cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034040040 con 0x5652a5f13390 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f6034031d20 con 0x5652a5f13390 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60340318b0 con 0x5652a5f13390 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6034032e50 con 0x5652a5f13390 2024-09-21T03:30:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 10 client.17730 fetch_fsmap learned FSMap version 51 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5652a636c1c0 con 0x5652a5f13390 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60340318b0 con 0x5652a5f13390 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f6034057070 con 0x5652a5f13390 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 10 client.17730 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 10 client.17730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 20 client.17730 populate_metadata read hostname 'smithi083' 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.158+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ec7da0 0x5652a636e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.159+0000 7f6051cfa740 4 client.17730 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.159+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652a624f8a0 con 0x5652a5ec7da0 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.159+0000 7f6049d23640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ec7da0 0x5652a636e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.159+0000 7f602cff9640 10 client.17730 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.159+0000 7f6049d23640 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ec7da0 0x5652a636e800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85732c92031beca2 server_cookie=e4615ac5d59b09fc in_seq=0 out_seq=0 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f602cff9640 1 -- 172.21.15.83:0/3060261067 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652a624f8a0 con 0x5652a5ec7da0 2024-09-21T03:30:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f602cff9640 10 client.17730 handle_command_reply: tid=0 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 client.17730 shutdown 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 msgr2=0x5652a62f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6034009230 tx=0x7f6034031400 comp rx=0 tx=0).stop 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 shutdown_connections 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x5652a5ec7da0 0x5652a636e800 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652a5f34070 0x5652a62f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652a5f13390 0x5652a62f3360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 --2- 172.21.15.83:0/3060261067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652a5f0b1b0 0x5652a62f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 >> 172.21.15.83:0/3060261067 conn(0x5652a5f28460 msgr2=0x5652a62dc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f602cff9640 0 client.17730 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 shutdown_connections 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 1 -- 172.21.15.83:0/3060261067 wait complete. 2024-09-21T03:30:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.160+0000 7f6051cfa740 20 client.17730 trim_cache size 0 max 16384 2024-09-21T03:30:59.167 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:30:59.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 msgr2=0x7f60300b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 0x7f60300b8bc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5652a62d9f40 tx=0x7f6040003e90 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 msgr2=0x7f6044199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f604000ea60 tx=0x7f604000ef20 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 shutdown_connections 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f60300b66b0 0x7f60300b8bc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6044108cf0 0x7f604419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441083a0 0x7f60441999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 --2- 172.21.15.83:0/1628270289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044102a60 0x7f6044199430 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 >> 172.21.15.83:0/1628270289 conn(0x7f6044079730 msgr2=0x7f6044104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 shutdown_connections 2024-09-21T03:30:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:30:59.161+0000 7f604b786640 1 -- 172.21.15.83:0/1628270289 wait complete. 2024-09-21T03:30:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:30:59 smithi083 ceph-mon[31219]: pgmap v770: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:30:59.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:30:59 smithi142 ceph-mon[34512]: pgmap v770: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:30:59.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:30:59 smithi193 ceph-mon[35224]: pgmap v770: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 1.22k op/s 2024-09-21T03:30:59.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:31:00.325 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 -- 172.21.15.83:0/3431223009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c102ae0 msgr2=0x7f2b8c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 --2- 172.21.15.83:0/3431223009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c102ef0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009900 tx=0x7f2b8802a410 comp rx=0 tx=0).stop 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 -- 172.21.15.83:0/3431223009 shutdown_connections 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 --2- 172.21.15.83:0/3431223009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 0x7f2b8c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 --2- 172.21.15.83:0/3431223009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 --2- 172.21.15.83:0/3431223009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c102ef0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 -- 172.21.15.83:0/3431223009 >> 172.21.15.83:0/3431223009 conn(0x7f2b8c079790 msgr2=0x7f2b8c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 -- 172.21.15.83:0/3431223009 shutdown_connections 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.918+0000 7f2b942d0640 1 -- 172.21.15.83:0/3431223009 wait complete. 2024-09-21T03:31:00.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 Processor -- start 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 -- start start 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 0x7f2b8c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1129e0 con 0x7f2b8c1083d0 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c112860 con 0x7f2b8c102ae0 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b942d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c112b60 con 0x7f2b8c108d20 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49678/0 (socket says 172.21.15.83:49678) 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b9186b640 1 -- 172.21.15.83:0/2466967453 learned_addr learned my addr 172.21.15.83:0/2466967453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b9206c640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.919+0000 7f2b9286d640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 0x7f2b8c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 -- 172.21.15.83:0/2466967453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 msgr2=0x7f2b8c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 0x7f2b8c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 -- 172.21.15.83:0/2466967453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c102ae0 msgr2=0x7f2b8c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 -- 172.21.15.83:0/2466967453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c1a8810 con 0x7f2b8c1083d0 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b9186b640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00e9f0 tx=0x7f2b7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b837fe640 1 -- 172.21.15.83:0/2466967453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00ccc0 con 0x7f2b8c1083d0 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b837fe640 1 -- 172.21.15.83:0/2466967453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7c0108e0 con 0x7f2b8c1083d0 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c1a8b00 con 0x7f2b8c1083d0 2024-09-21T03:31:00.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b837fe640 1 -- 172.21.15.83:0/2466967453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c018860 con 0x7f2b8c1083d0 2024-09-21T03:31:00.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.920+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c1a9040 con 0x7f2b8c1083d0 2024-09-21T03:31:00.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 Processor -- start 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 -- start start 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b837fe640 1 -- 172.21.15.83:0/2466967453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2b7c018a00 con 0x7f2b8c1083d0 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fe4129ea0 con 0x562fe3deb390 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fe41192d0 con 0x562fe3de31b0 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.923+0000 7f2b9a844740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fe411d0e0 con 0x562fe3e0c070 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9206c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57582/0 (socket says 172.21.15.83:57582) 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9206c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33398/0 (socket says 172.21.15.83:33398) 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 -- 172.21.15.83:0/728217027 learned_addr learned my addr 172.21.15.83:0/728217027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 -- 172.21.15.83:0/728217027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 msgr2=0x562fe41b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 -- 172.21.15.83:0/728217027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 msgr2=0x562fe41b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 -- 172.21.15.83:0/728217027 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe3e77650 con 0x562fe3e0c070 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9206c640 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b9286d640 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009f50 tx=0x7f2b840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2575ea382d926f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b837fe640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 0x7f2b780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.924+0000 7f2b80d9a640 1 -- 172.21.15.83:0/728217027 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84004d20 con 0x562fe3e0c070 2024-09-21T03:31:00.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.925+0000 7f2b80d9a640 1 -- 172.21.15.83:0/728217027 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b84004050 con 0x562fe3e0c070 2024-09-21T03:31:00.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.925+0000 7f2b80d9a640 1 -- 172.21.15.83:0/728217027 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84005280 con 0x562fe3e0c070 2024-09-21T03:31:00.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.925+0000 7f2b9206c640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 0x7f2b780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.925+0000 7f2b837fe640 1 -- 172.21.15.83:0/2466967453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f2b7c0e79d0 con 0x7f2b8c1083d0 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 -- 172.21.15.83:0/728217027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 msgr2=0x562fe41b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9206c640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 0x7f2b780b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009f50 tx=0x7f2b840118d0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 -- 172.21.15.83:0/728217027 shutdown_connections 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 --2- 172.21.15.83:0/728217027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41b32c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 -- 172.21.15.83:0/728217027 >> 172.21.15.83:0/728217027 conn(0x562fe3e00460 msgr2=0x562fe41b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 -- 172.21.15.83:0/728217027 shutdown_connections 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.926+0000 7f2b9a844740 1 -- 172.21.15.83:0/728217027 wait complete. 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 -- start start 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 Processor -- start 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fe3f7a700 con 0x562fe3deb390 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562fe40b4160 con 0x562fe3de31b0 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562fe41c1aa0 con 0x562fe3e0c070 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49692/0 (socket says 172.21.15.83:49692) 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9186b640 1 -- 172.21.15.83:0/1724501458 learned_addr learned my addr 172.21.15.83:0/1724501458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.927+0000 7f2b9a844740 10 client.0 _ll_register_callbacks cb 0x562fe40cd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9286d640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 -- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 msgr2=0x562fe41bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 -- 172.21.15.83:0/1724501458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 msgr2=0x562fe41be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 -- 172.21.15.83:0/1724501458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe40e1c90 con 0x562fe3deb390 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9286d640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c6a0 tx=0x7f2b7c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9289f87a760ae4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0a97c0 con 0x562fe3deb390 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b84004050 con 0x562fe3deb390 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f2b7c0a9270 con 0x562fe3deb390 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c0ad090 con 0x562fe3deb390 2024-09-21T03:31:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9a844740 10 client.17742 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.928+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562fe41cd7a0 con 0x562fe3deb390 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b84004050 con 0x562fe3deb390 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f2b7c0a9470 con 0x562fe3deb390 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 10 client.17742 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 10 client.17742 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 20 client.17742 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3dca9f0 0x562fe41d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 4 client.17742 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe412bdf0 con 0x562fe3dca9f0 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.929+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3dca9f0 0x562fe41d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.930+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3dca9f0 0x562fe41d0c30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d157b3e864853b4 server_cookie=a8f0f21aa18c45a4 in_seq=0 out_seq=0 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.930+0000 7f2b75ffb640 10 client.17742 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.930+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1724501458 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fe412bdf0 con 0x562fe3dca9f0 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.930+0000 7f2b75ffb640 10 client.17742 handle_command_reply: tid=0 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.930+0000 7f2b9a844740 1 client.17742 shutdown 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 msgr2=0x562fe41bee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c6a0 tx=0x7f2b7c00ce60 comp rx=0 tx=0).stop 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 shutdown_connections 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3dca9f0 0x562fe41d0c30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41bee30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1724501458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 >> 172.21.15.83:0/1724501458 conn(0x562fe3e00460 msgr2=0x562fe41b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b75ffb640 0 client.17742 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 shutdown_connections 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 1 -- 172.21.15.83:0/1724501458 wait complete. 2024-09-21T03:31:00.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.931+0000 7f2b9a844740 20 client.17742 trim_cache size 0 max 16384 2024-09-21T03:31:00.938 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 820.0 s, 962.8 tps, lat 101.318 ms stddev 135.951 2024-09-21T03:31:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.944+0000 7f2b9a844740 1 Processor -- start 2024-09-21T03:31:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.944+0000 7f2b9a844740 1 -- start start 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3deb390 0x562fe41bbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b1920 con 0x562fe3de31b0 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b3460 con 0x562fe3deb390 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9a844740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b1ff0 con 0x562fe3e0c070 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49702/0 (socket says 172.21.15.83:49702) 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 -- 172.21.15.83:0/567499673 learned_addr learned my addr 172.21.15.83:0/567499673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9286d640 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9186b640 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3deb390 0x562fe41bbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 -- 172.21.15.83:0/567499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 msgr2=0x562fe41bc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 -- 172.21.15.83:0/567499673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3deb390 msgr2=0x562fe41bbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3deb390 0x562fe41bbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 -- 172.21.15.83:0/567499673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe41b6120 con 0x562fe3de31b0 2024-09-21T03:31:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b9206c640 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b880096c0 tx=0x7f2b8803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fcd64f10c329c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.945+0000 7f2b77fff640 1 -- 172.21.15.83:0/567499673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88058070 con 0x562fe3de31b0 2024-09-21T03:31:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.946+0000 7f2b77fff640 1 -- 172.21.15.83:0/567499673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b88005040 con 0x562fe3de31b0 2024-09-21T03:31:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.946+0000 7f2b77fff640 1 -- 172.21.15.83:0/567499673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88035040 con 0x562fe3de31b0 2024-09-21T03:31:00.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.946+0000 7f2b9a844740 1 -- 172.21.15.83:0/567499673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 msgr2=0x562fe41c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.946+0000 7f2b9a844740 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b880096c0 tx=0x7f2b8803f000 comp rx=0 tx=0).stop 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.946+0000 7f2b9a844740 1 -- 172.21.15.83:0/567499673 shutdown_connections 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3deb390 0x562fe41bbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- 172.21.15.83:0/567499673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3de31b0 0x562fe41c0520 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- 172.21.15.83:0/567499673 >> 172.21.15.83:0/567499673 conn(0x562fe3e00460 msgr2=0x562fe41bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- 172.21.15.83:0/567499673 shutdown_connections 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- 172.21.15.83:0/567499673 wait complete. 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- start start 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 Processor -- start 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b2d90 con 0x562fe3deb390 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b26c0 con 0x562fe3de31b0 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1af920 con 0x562fe3e0c070 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9a844740 10 client.0 _ll_register_callbacks cb 0x7f2b7c0eb110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9186b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49718/0 (socket says 172.21.15.83:49718) 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.947+0000 7f2b9186b640 1 -- 172.21.15.83:0/1613899148 learned_addr learned my addr 172.21.15.83:0/1613899148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9286d640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 -- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 msgr2=0x562fe41c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 -- 172.21.15.83:0/1613899148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 msgr2=0x562fe41c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 -- 172.21.15.83:0/1613899148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe41c3920 con 0x562fe3deb390 2024-09-21T03:31:00.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9186b640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0a9ef0 tx=0x7f2b7c0a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7d0a584c70a802 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0ad270 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f2b7c0aedb0 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b88005040 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c098030 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9a844740 10 client.17748 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.948+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562fe423b6a0 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7c0ad430 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f2b7c0ac5f0 con 0x562fe3deb390 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 10 client.17748 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 10 client.17748 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 20 client.17748 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3d9fda0 0x562fe423de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 4 client.17748 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2b88005040 con 0x562fe3d9fda0 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3d9fda0 0x562fe423de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b75ffb640 10 client.17748 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.949+0000 7f2b9206c640 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3d9fda0 0x562fe423de40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e45480a13e64e5e server_cookie=1a5e5c0de86222bf in_seq=0 out_seq=0 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b75ffb640 1 -- 172.21.15.83:0/1613899148 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2b88005040 con 0x562fe3d9fda0 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b75ffb640 10 client.17748 handle_command_reply: tid=0 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 client.17748 shutdown 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 msgr2=0x562fe41c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0a9ef0 tx=0x7f2b7c0a9da0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 shutdown_connections 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x562fe3d9fda0 0x562fe423de40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b75ffb640 0 client.17748 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562fe3e0c070 0x562fe41c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fe3deb390 0x562fe41c2af0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 --2- 172.21.15.83:0/1613899148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562fe3de31b0 0x562fe41c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 >> 172.21.15.83:0/1613899148 conn(0x562fe3e00460 msgr2=0x562fe41c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 shutdown_connections 2024-09-21T03:31:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 1 -- 172.21.15.83:0/1613899148 wait complete. 2024-09-21T03:31:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b9a844740 20 client.17748 trim_cache size 0 max 16384 2024-09-21T03:31:00.957 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 msgr2=0x7f2b780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.950+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 0x7f2b780b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88031400 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 msgr2=0x7f2b8c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00e9f0 tx=0x7f2b7c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 shutdown_connections 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2b780b6680 0x7f2b780b8b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b8c108d20 0x7f2b8c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b8c1083d0 0x7f2b8c1996a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 --2- 172.21.15.83:0/2466967453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c102ae0 0x7f2b8c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 >> 172.21.15.83:0/2466967453 conn(0x7f2b8c079790 msgr2=0x7f2b8c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 shutdown_connections 2024-09-21T03:31:00.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:00.951+0000 7f2b942d0640 1 -- 172.21.15.83:0/2466967453 wait complete. 2024-09-21T03:31:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:01 smithi083 ceph-mon[31219]: pgmap v771: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-09-21T03:31:01.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:01 smithi142 ceph-mon[34512]: pgmap v771: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-09-21T03:31:01.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:01 smithi193 ceph-mon[35224]: pgmap v771: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-09-21T03:31:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:03 smithi083 ceph-mon[31219]: pgmap v772: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:31:03.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:03 smithi142 ceph-mon[34512]: pgmap v772: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:31:03.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:31:03.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:03 smithi193 ceph-mon[35224]: pgmap v772: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-09-21T03:31:04.134 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.744+0000 7f58101c7640 1 -- 172.21.15.83:0/201013965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808111f40 msgr2=0x7f580810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.744+0000 7f58101c7640 1 --2- 172.21.15.83:0/201013965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808111f40 0x7f580810a3c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f580002a420 comp rx=0 tx=0).stop 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.750+0000 7f58101c7640 1 -- 172.21.15.83:0/201013965 shutdown_connections 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.750+0000 7f58101c7640 1 --2- 172.21.15.83:0/201013965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f580811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.750+0000 7f58101c7640 1 --2- 172.21.15.83:0/201013965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f580810a9e0 0x7f580810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.750+0000 7f58101c7640 1 --2- 172.21.15.83:0/201013965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808111f40 0x7f580810a3c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.750+0000 7f58101c7640 1 -- 172.21.15.83:0/201013965 >> 172.21.15.83:0/201013965 conn(0x7f5808070ec0 msgr2=0x7f58080712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.751+0000 7f58101c7640 1 -- 172.21.15.83:0/201013965 shutdown_connections 2024-09-21T03:31:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.751+0000 7f58101c7640 1 -- 172.21.15.83:0/201013965 wait complete. 2024-09-21T03:31:04.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.752+0000 7f58101c7640 1 Processor -- start 2024-09-21T03:31:04.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.752+0000 7f58101c7640 1 -- start start 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f5808113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5808122900 con 0x7f580810a9e0 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5808122780 con 0x7f5808111f40 2024-09-21T03:31:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.755+0000 7f58101c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5808122a80 con 0x7f580811cc30 2024-09-21T03:31:04.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.756+0000 7f580df63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.756+0000 7f580df63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49732/0 (socket says 172.21.15.83:49732) 2024-09-21T03:31:04.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.756+0000 7f580d762640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.756+0000 7f580d762640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33452/0 (socket says 172.21.15.83:33452) 2024-09-21T03:31:04.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.756+0000 7f580d762640 1 -- 172.21.15.83:0/2220328047 learned_addr learned my addr 172.21.15.83:0/2220328047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:04.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.758+0000 7f580d762640 1 -- 172.21.15.83:0/2220328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 msgr2=0x7f5808113cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.758+0000 7f580e764640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f5808113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.761+0000 7f580d762640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f5808113cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.761+0000 7f580d762640 1 -- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 msgr2=0x7f580811a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.761+0000 7f580d762640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.761+0000 7f580d762640 1 -- 172.21.15.83:0/2220328047 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58081144e0 con 0x7f5808111f40 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.762+0000 7f580e764640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f5808113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.762+0000 7f580df63640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.762+0000 7f580d762640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c970 tx=0x7f57fc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.763+0000 7f5806ffd640 1 -- 172.21.15.83:0/2220328047 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc004220 con 0x7f5808111f40 2024-09-21T03:31:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.763+0000 7f5806ffd640 1 -- 172.21.15.83:0/2220328047 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57fc0043c0 con 0x7f5808111f40 2024-09-21T03:31:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.763+0000 7f5806ffd640 1 -- 172.21.15.83:0/2220328047 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00f450 con 0x7f5808111f40 2024-09-21T03:31:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.764+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5808119160 con 0x7f5808111f40 2024-09-21T03:31:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.764+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5808119650 con 0x7f5808111f40 2024-09-21T03:31:04.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.769+0000 7f581673b740 1 Processor -- start 2024-09-21T03:31:04.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.771+0000 7f5806ffd640 1 -- 172.21.15.83:0/2220328047 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f57fc00f5f0 con 0x7f5808111f40 2024-09-21T03:31:04.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.773+0000 7f5806ffd640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 0x7f57f00b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.773+0000 7f5806ffd640 1 -- 172.21.15.83:0/2220328047 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f57fc0e7740 con 0x7f5808111f40 2024-09-21T03:31:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.774+0000 7f580e764640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 0x7f57f00b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.784+0000 7f580e764640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 0x7f57f00b8cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005fd0 tx=0x7f57f80093c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 -- start start 2024-09-21T03:31:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564543828ea0 con 0x5645434e21b0 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645438182d0 con 0x56454350b070 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.786+0000 7f581673b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56454381c0e0 con 0x5645434ea390 2024-09-21T03:31:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.787+0000 7f580df63640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.787+0000 7f580df63640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33462/0 (socket says 172.21.15.83:33462) 2024-09-21T03:31:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.787+0000 7f580df63640 1 -- 172.21.15.83:0/4292244416 learned_addr learned my addr 172.21.15.83:0/4292244416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.788+0000 7f580d762640 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580e764640 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580df63640 1 -- 172.21.15.83:0/4292244416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 msgr2=0x5645438b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580df63640 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580df63640 1 -- 172.21.15.83:0/4292244416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 msgr2=0x5645438b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580df63640 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.789+0000 7f580df63640 1 -- 172.21.15.83:0/4292244416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564543576650 con 0x56454350b070 2024-09-21T03:31:04.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.791+0000 7f580df63640 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5800009230 tx=0x7f58000314f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6ec2d079b97d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.791+0000 7f57effff640 1 -- 172.21.15.83:0/4292244416 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000318e0 con 0x56454350b070 2024-09-21T03:31:04.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.792+0000 7f57effff640 1 -- 172.21.15.83:0/4292244416 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580003d420 con 0x56454350b070 2024-09-21T03:31:04.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.792+0000 7f57effff640 1 -- 172.21.15.83:0/4292244416 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580003c680 con 0x56454350b070 2024-09-21T03:31:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 -- 172.21.15.83:0/4292244416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 msgr2=0x5645438b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5800009230 tx=0x7f58000314f0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 -- 172.21.15.83:0/4292244416 shutdown_connections 2024-09-21T03:31:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 --2- 172.21.15.83:0/4292244416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56454350b070 0x5645438b22c0 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5800009230 tx=0x7f58000314f0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.795+0000 7f581673b740 1 -- 172.21.15.83:0/4292244416 >> 172.21.15.83:0/4292244416 conn(0x5645434ff460 msgr2=0x5645438b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.796+0000 7f581673b740 1 -- 172.21.15.83:0/4292244416 shutdown_connections 2024-09-21T03:31:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.797+0000 7f581673b740 1 -- 172.21.15.83:0/4292244416 wait complete. 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.797+0000 7f581673b740 1 -- start start 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.798+0000 7f581673b740 1 Processor -- start 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.798+0000 7f581673b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438bda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434c99f0 0x5645438c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564543679700 con 0x5645434e21b0 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645437b3160 con 0x5645434c99f0 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5645438c0a40 con 0x5645434ea390 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f580d762640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f580d762640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57630/0 (socket says 172.21.15.83:57630) 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f580d762640 1 -- 172.21.15.83:0/2039443790 learned_addr learned my addr 172.21.15.83:0/2039443790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.799+0000 7f581673b740 10 client.0 _ll_register_callbacks cb 0x5645437cc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.800+0000 7f580e764640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434c99f0 0x5645438c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.801+0000 7f580d762640 1 -- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434c99f0 msgr2=0x5645438c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.801+0000 7f580d762640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434c99f0 0x5645438c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.801+0000 7f580d762640 1 -- 172.21.15.83:0/2039443790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 msgr2=0x5645438bda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.801+0000 7f580d762640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.801+0000 7f580d762640 1 -- 172.21.15.83:0/2039443790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645437e0c90 con 0x5645434ea390 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.802+0000 7f580d762640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57fc003b20 tx=0x7f57fc0a9530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20187867ccb068a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.802+0000 7f57ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.802+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc01d070 con 0x5645434ea390 2024-09-21T03:31:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.802+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580003d420 con 0x5645434ea390 2024-09-21T03:31:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.802+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f57fc0e7110 con 0x5645434ea390 2024-09-21T03:31:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.805+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0d4020 con 0x5645434ea390 2024-09-21T03:31:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.805+0000 7f581673b740 10 client.27238 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.805+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5645438cc8e0 con 0x5645434ea390 2024-09-21T03:31:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580003d420 con 0x5645434ea390 2024-09-21T03:31:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f57fc0ac250 con 0x5645434ea390 2024-09-21T03:31:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 10 client.27238 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 10 client.27238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:31:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 20 client.27238 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454349eda0 0x5645438d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 4 client.27238 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.806+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454382adf0 con 0x56454349eda0 2024-09-21T03:31:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.809+0000 7f580df63640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454349eda0 0x5645438d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.816+0000 7f580df63640 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454349eda0 0x5645438d0250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f76eab30a79b5f1b server_cookie=d869f4a546d4d6d1 in_seq=0 out_seq=0 2024-09-21T03:31:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.816+0000 7f57ed7fa640 10 client.27238 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.817+0000 7f57ed7fa640 1 -- 172.21.15.83:0/2039443790 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56454382adf0 con 0x56454349eda0 2024-09-21T03:31:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.817+0000 7f57ed7fa640 10 client.27238 handle_command_reply: tid=0 2024-09-21T03:31:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.817+0000 7f581673b740 1 client.27238 shutdown 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 msgr2=0x5645438bdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57fc003b20 tx=0x7f57fc0a9530 comp rx=0 tx=0).stop 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 shutdown_connections 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454349eda0 0x5645438d0250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434c99f0 0x5645438c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434ea390 0x5645438bdfa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 --2- 172.21.15.83:0/2039443790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434e21b0 0x5645438bda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 >> 172.21.15.83:0/2039443790 conn(0x5645434ff460 msgr2=0x5645438b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f57ed7fa640 0 client.27238 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.819+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 shutdown_connections 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.820+0000 7f581673b740 1 -- 172.21.15.83:0/2039443790 wait complete. 2024-09-21T03:31:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.821+0000 7f581673b740 20 client.27238 trim_cache size 0 max 16384 2024-09-21T03:31:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.849+0000 7f581673b740 1 Processor -- start 2024-09-21T03:31:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.851+0000 7f581673b740 1 -- start start 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x5645438b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400e6a0 con 0x5645434c99f0 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40101e0 con 0x5645434ea390 2024-09-21T03:31:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f581673b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400ed70 con 0x5645434e21b0 2024-09-21T03:31:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f580df63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f580df63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49762/0 (socket says 172.21.15.83:49762) 2024-09-21T03:31:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.852+0000 7f580df63640 1 -- 172.21.15.83:0/4284798059 learned_addr learned my addr 172.21.15.83:0/4284798059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.854+0000 7f580e764640 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x5645438b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.854+0000 7f580d762640 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.857+0000 7f580df63640 1 -- 172.21.15.83:0/4284798059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 msgr2=0x5645438b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.857+0000 7f580df63640 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.857+0000 7f580df63640 1 -- 172.21.15.83:0/4284798059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 msgr2=0x5645438b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.857+0000 7f580df63640 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x5645438b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.857+0000 7f580df63640 1 -- 172.21.15.83:0/4284798059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645438d0820 con 0x5645434c99f0 2024-09-21T03:31:04.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.860+0000 7f580df63640 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6a121b73fba61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.861+0000 7f57ef7fe640 1 -- 172.21.15.83:0/4284798059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580005c070 con 0x5645434c99f0 2024-09-21T03:31:04.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.861+0000 7f57ef7fe640 1 -- 172.21.15.83:0/4284798059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580003d420 con 0x5645434c99f0 2024-09-21T03:31:04.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.861+0000 7f57ef7fe640 1 -- 172.21.15.83:0/4284798059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800057430 con 0x5645434c99f0 2024-09-21T03:31:04.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.862+0000 7f581673b740 1 -- 172.21.15.83:0/4284798059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 msgr2=0x5645438c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.862+0000 7f581673b740 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800000bc0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.863+0000 7f581673b740 1 -- 172.21.15.83:0/4284798059 shutdown_connections 2024-09-21T03:31:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.863+0000 7f581673b740 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x5645438b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.863+0000 7f581673b740 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.863+0000 7f581673b740 1 --2- 172.21.15.83:0/4284798059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c9c90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.863+0000 7f581673b740 1 -- 172.21.15.83:0/4284798059 >> 172.21.15.83:0/4284798059 conn(0x5645434ff460 msgr2=0x5645438bc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.866+0000 7f581673b740 1 -- 172.21.15.83:0/4284798059 shutdown_connections 2024-09-21T03:31:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.866+0000 7f581673b740 1 -- 172.21.15.83:0/4284798059 wait complete. 2024-09-21T03:31:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.867+0000 7f581673b740 1 -- start start 2024-09-21T03:31:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.867+0000 7f581673b740 1 Processor -- start 2024-09-21T03:31:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x564543938c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400fb10 con 0x5645434c99f0 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400f440 con 0x5645434ea390 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400c6a0 con 0x5645434e21b0 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.870+0000 7f581673b740 10 client.0 _ll_register_callbacks cb 0x7f57fc00a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.872+0000 7f580d762640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.872+0000 7f580d762640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57644/0 (socket says 172.21.15.83:57644) 2024-09-21T03:31:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.872+0000 7f580d762640 1 -- 172.21.15.83:0/3435306943 learned_addr learned my addr 172.21.15.83:0/3435306943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.872+0000 7f580e764640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x564543938c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.873+0000 7f580d762640 1 -- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 msgr2=0x564543938c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.873+0000 7f580d762640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x564543938c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.873+0000 7f580d762640 1 -- 172.21.15.83:0/3435306943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 msgr2=0x5645438c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.873+0000 7f580d762640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.873+0000 7f580d762640 1 -- 172.21.15.83:0/3435306943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564543939180 con 0x5645434e21b0 2024-09-21T03:31:04.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.876+0000 7f580d762640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00afd0 tx=0x7f57fc0acc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeceea4be9b75e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.877+0000 7f57ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.878+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc01d040 con 0x5645434e21b0 2024-09-21T03:31:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.878+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f57fc0a8ed0 con 0x5645434e21b0 2024-09-21T03:31:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.880+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580003d420 con 0x5645434e21b0 2024-09-21T03:31:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.882+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0b2060 con 0x5645434e21b0 2024-09-21T03:31:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.882+0000 7f581673b740 10 client.27244 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.882+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56454393a360 con 0x5645434e21b0 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.884+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580003d420 con 0x5645434e21b0 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.884+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f57fc0b64e0 con 0x5645434e21b0 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 10 client.27244 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 10 client.27244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 20 client.27244 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454393a920 0x56454393cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 4 client.27244 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.885+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645438268a0 con 0x56454393a920 2024-09-21T03:31:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.886+0000 7f580df63640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454393a920 0x56454393cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.900+0000 7f580df63640 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454393a920 0x56454393cd40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f35c341e8758855 server_cookie=1b9a75c004e7a777 in_seq=0 out_seq=0 2024-09-21T03:31:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.900+0000 7f57ed7fa640 10 client.27244 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.900+0000 7f57ed7fa640 1 -- 172.21.15.83:0/3435306943 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645438268a0 con 0x56454393a920 2024-09-21T03:31:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.900+0000 7f57ed7fa640 10 client.27244 handle_command_reply: tid=0 2024-09-21T03:31:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.900+0000 7f581673b740 1 client.27244 shutdown 2024-09-21T03:31:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.905+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 msgr2=0x5645438c1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.905+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00afd0 tx=0x7f57fc0acc20 comp rx=0 tx=0).stop 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 shutdown_connections 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x56454393a920 0x56454393cd40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645434ea390 0x564543938c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645434e21b0 0x5645438c1a00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 --2- 172.21.15.83:0/3435306943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645434c99f0 0x5645438c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.906+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 >> 172.21.15.83:0/3435306943 conn(0x5645434ff460 msgr2=0x5645438afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.909+0000 7f57ed7fa640 0 client.27244 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:04.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.914+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 shutdown_connections 2024-09-21T03:31:04.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.914+0000 7f581673b740 1 -- 172.21.15.83:0/3435306943 wait complete. 2024-09-21T03:31:04.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.914+0000 7f581673b740 20 client.27244 trim_cache size 0 max 16384 2024-09-21T03:31:04.917 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:04.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.916+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 msgr2=0x7f57f00b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.916+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 0x7f57f00b8cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005fd0 tx=0x7f57f80093c0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.916+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 msgr2=0x7f5808113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.916+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c970 tx=0x7f57fc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 shutdown_connections 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57f00b67b0 0x7f57f00b8cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f580811cc30 0x7f5808113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5808111f40 0x7f5808113700 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 --2- 172.21.15.83:0/2220328047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580810a9e0 0x7f580811a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.917+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 >> 172.21.15.83:0/2220328047 conn(0x7f5808070ec0 msgr2=0x7f5808071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.919+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 shutdown_connections 2024-09-21T03:31:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:04.919+0000 7f58101c7640 1 -- 172.21.15.83:0/2220328047 wait complete. 2024-09-21T03:31:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:05 smithi083 ceph-mon[31219]: pgmap v773: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-09-21T03:31:05.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:05 smithi142 ceph-mon[34512]: pgmap v773: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-09-21T03:31:05.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:05 smithi193 ceph-mon[35224]: pgmap v773: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-09-21T03:31:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 825.0 s, 1290.8 tps, lat 80.134 ms stddev 108.314 2024-09-21T03:31:06.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:31:06.369 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:06 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:06.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:06 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:06.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:06 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.224+0000 7f58b4c38640 1 -- 172.21.15.83:0/3366687679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010baf0 msgr2=0x7f58b010bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.224+0000 7f58b4c38640 1 --2- 172.21.15.83:0/3366687679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010baf0 0x7f58b010bf00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58a0008570 tx=0x7f58a002e3b0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.226+0000 7f58b4c38640 1 -- 172.21.15.83:0/3366687679 shutdown_connections 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.226+0000 7f58b4c38640 1 --2- 172.21.15.83:0/3366687679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b0110490 0x7f58b007b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.226+0000 7f58b4c38640 1 --2- 172.21.15.83:0/3366687679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b010fb40 0x7f58b010ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.226+0000 7f58b4c38640 1 --2- 172.21.15.83:0/3366687679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010baf0 0x7f58b010bf00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.226+0000 7f58b4c38640 1 -- 172.21.15.83:0/3366687679 >> 172.21.15.83:0/3366687679 conn(0x7f58b0070ec0 msgr2=0x7f58b00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.227+0000 7f58b4c38640 1 -- 172.21.15.83:0/3366687679 shutdown_connections 2024-09-21T03:31:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.227+0000 7f58b4c38640 1 -- 172.21.15.83:0/3366687679 wait complete. 2024-09-21T03:31:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.228+0000 7f58b4c38640 1 Processor -- start 2024-09-21T03:31:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.228+0000 7f58b4c38640 1 -- start start 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010fb40 0x7f58b0080b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b0110490 0x7f58b0081060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58b007ec60 con 0x7f58b0110490 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58b007eae0 con 0x7f58b010fb40 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58b4c38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58b007ede0 con 0x7f58b010baf0 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57664/0 (socket says 172.21.15.83:57664) 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58ae59c640 1 -- 172.21.15.83:0/2665128787 learned_addr learned my addr 172.21.15.83:0/2665128787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b0110490 0x7f58b0081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.229+0000 7f58add9b640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010fb40 0x7f58b0080b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.232+0000 7f58ae59c640 1 -- 172.21.15.83:0/2665128787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010fb40 msgr2=0x7f58b0080b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.232+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010fb40 0x7f58b0080b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.232+0000 7f58ae59c640 1 -- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b0110490 msgr2=0x7f58b0081060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.232+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b0110490 0x7f58b0081060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.232+0000 7f58ae59c640 1 -- 172.21.15.83:0/2665128787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b01bdbd0 con 0x7f58b010baf0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.233+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a0005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.234+0000 7f589f7fe640 1 -- 172.21.15.83:0/2665128787 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a004d020 con 0x7f58b010baf0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.234+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b01bde60 con 0x7f58b010baf0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.234+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b01be360 con 0x7f58b010baf0 2024-09-21T03:31:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.234+0000 7f589f7fe640 1 -- 172.21.15.83:0/2665128787 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a000a580 con 0x7f58b010baf0 2024-09-21T03:31:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.234+0000 7f589f7fe640 1 -- 172.21.15.83:0/2665128787 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0036ae0 con 0x7f58b010baf0 2024-09-21T03:31:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.240+0000 7f589f7fe640 1 -- 172.21.15.83:0/2665128787 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f58a0050020 con 0x7f58b010baf0 2024-09-21T03:31:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.241+0000 7f589f7fe640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 0x7f58980b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.241+0000 7f589f7fe640 1 -- 172.21.15.83:0/2665128787 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f58a0116dd0 con 0x7f58b010baf0 2024-09-21T03:31:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.245+0000 7f58bb1ac740 1 Processor -- start 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.246+0000 7f58bb1ac740 1 -- start start 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.246+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.246+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc7671be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc7672120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57672/0 (socket says 172.21.15.83:57672) 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58ae59c640 1 -- 172.21.15.83:0/164886382 learned_addr learned my addr 172.21.15.83:0/164886382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cc75e7ea0 con 0x555cc72a9390 2024-09-21T03:31:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555cc75d72d0 con 0x555cc72a11b0 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cc75db0e0 con 0x555cc72ca070 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.247+0000 7f58aed9d640 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc7672120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 -- 172.21.15.83:0/164886382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 msgr2=0x555cc7671be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc7671be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 -- 172.21.15.83:0/164886382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 msgr2=0x555cc7672120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc7672120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 -- 172.21.15.83:0/164886382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc7335650 con 0x555cc72ca070 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58aed9d640 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc7672120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.248+0000 7f58ae59c640 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a00d93f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e779904a212c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.249+0000 7f589cff9640 1 -- 172.21.15.83:0/164886382 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00d83e0 con 0x555cc72ca070 2024-09-21T03:31:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.249+0000 7f589cff9640 1 -- 172.21.15.83:0/164886382 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a00dc780 con 0x555cc72ca070 2024-09-21T03:31:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.249+0000 7f589cff9640 1 -- 172.21.15.83:0/164886382 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00e6690 con 0x555cc72ca070 2024-09-21T03:31:07.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.250+0000 7f58add9b640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 0x7f58980b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.251+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 msgr2=0x555cc76716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.251+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a00d93f0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.252+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 shutdown_connections 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.252+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc7672120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.252+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc7671be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.252+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/164886382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc76716a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.252+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 >> 172.21.15.83:0/164886382 conn(0x555cc72be460 msgr2=0x555cc766f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.259+0000 7f58add9b640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 0x7f58980b8c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x555cc7670320 tx=0x7f58a8009910 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.260+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 shutdown_connections 2024-09-21T03:31:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.261+0000 7f58bb1ac740 1 -- 172.21.15.83:0/164886382 wait complete. 2024-09-21T03:31:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.261+0000 7f58bb1ac740 1 -- start start 2024-09-21T03:31:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.262+0000 7f58bb1ac740 1 Processor -- start 2024-09-21T03:31:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.262+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.262+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc7688d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58bb1ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cc7438700 con 0x555cc72a9390 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58bb1ac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555cc7572160 con 0x555cc72a11b0 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58bb1ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cc75db0e0 con 0x555cc72ca070 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58bb1ac740 10 client.0 _ll_register_callbacks cb 0x555cc758b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58add9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58add9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49798/0 (socket says 172.21.15.83:49798) 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.263+0000 7f58add9b640 1 -- 172.21.15.83:0/40064995 learned_addr learned my addr 172.21.15.83:0/40064995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.264+0000 7f58aed9d640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc7688d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.265+0000 7f58add9b640 1 -- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 msgr2=0x555cc7688d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.265+0000 7f58add9b640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc7688d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.265+0000 7f58add9b640 1 -- 172.21.15.83:0/40064995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 msgr2=0x555cc767cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.265+0000 7f58add9b640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.265+0000 7f58add9b640 1 -- 172.21.15.83:0/40064995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc759fc90 con 0x555cc72a9390 2024-09-21T03:31:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.271+0000 7f58add9b640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005fd0 tx=0x7f58a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2445efc64667f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.271+0000 7f5895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.272+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8023020 con 0x555cc72a9390 2024-09-21T03:31:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.272+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f58a801f890 con 0x555cc72a9390 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.272+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a00dc780 con 0x555cc72a9390 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.273+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f58a802e050 con 0x555cc72a9390 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.273+0000 7f58bb1ac740 10 client.17772 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.273+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555cc768b820 con 0x555cc72a9390 2024-09-21T03:31:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.273+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a801e3e0 con 0x555cc72a9390 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.273+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f58a800f040 con 0x555cc72a9390 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 10 client.17772 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 10 client.17772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 20 client.17772 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc72889f0 0x555cc768edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 4 client.17772 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.274+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58a00dc780 con 0x555cc72889f0 2024-09-21T03:31:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.276+0000 7f58ae59c640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc72889f0 0x555cc768edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.289+0000 7f58ae59c640 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc72889f0 0x555cc768edf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2d0ed4619d22cd8 server_cookie=2df8698ae7d155a2 in_seq=0 out_seq=0 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.289+0000 7f5895ffb640 10 client.17772 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.290+0000 7f5895ffb640 1 -- 172.21.15.83:0/40064995 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f58a00dc780 con 0x555cc72889f0 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.290+0000 7f5895ffb640 10 client.17772 handle_command_reply: tid=0 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.290+0000 7f58bb1ac740 1 client.17772 shutdown 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.290+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 msgr2=0x555cc767d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.290+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005fd0 tx=0x7f58a8004a40 comp rx=0 tx=0).stop 2024-09-21T03:31:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 shutdown_connections 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc72889f0 0x555cc768edf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72ca070 0x555cc7688d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72a9390 0x555cc767d2f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/40064995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 >> 172.21.15.83:0/40064995 conn(0x555cc72be460 msgr2=0x555cc7674830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f5895ffb640 0 client.17772 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 shutdown_connections 2024-09-21T03:31:07.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.293+0000 7f58bb1ac740 1 -- 172.21.15.83:0/40064995 wait complete. 2024-09-21T03:31:07.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.294+0000 7f58bb1ac740 20 client.17772 trim_cache size 0 max 16384 2024-09-21T03:31:07.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.321+0000 7f58bb1ac740 1 Processor -- start 2024-09-21T03:31:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.323+0000 7f58bb1ac740 1 -- start start 2024-09-21T03:31:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.323+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.324+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.324+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.324+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:33526/0 (socket says 172.21.15.83:33526) 2024-09-21T03:31:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.324+0000 7f58add9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.324+0000 7f58add9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57690/0 (socket says 172.21.15.83:57690) 2024-09-21T03:31:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.325+0000 7f58ae59c640 1 -- 172.21.15.83:0/2028312808 learned_addr learned my addr 172.21.15.83:0/2028312808 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.325+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72ca070 0x555cc7674ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.325+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f589000e7d0 con 0x555cc72ca070 2024-09-21T03:31:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.325+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5890010310 con 0x555cc72a11b0 2024-09-21T03:31:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.325+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f589000eea0 con 0x555cc72a9390 2024-09-21T03:31:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.326+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72ca070 0x555cc7674ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.326+0000 7f58ae59c640 1 -- 172.21.15.83:0/2028312808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 msgr2=0x555cc76747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.326+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.327+0000 7f58ae59c640 1 -- 172.21.15.83:0/2028312808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72ca070 msgr2=0x555cc7674ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.327+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72ca070 0x555cc7674ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.327+0000 7f58ae59c640 1 -- 172.21.15.83:0/2028312808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc7678640 con 0x555cc72a11b0 2024-09-21T03:31:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.335+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f58a000acc0 tx=0x7f58a002e930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b56a99effe5af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.336+0000 7f5897fff640 1 -- 172.21.15.83:0/2028312808 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00d83e0 con 0x555cc72a11b0 2024-09-21T03:31:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.336+0000 7f5897fff640 1 -- 172.21.15.83:0/2028312808 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a00dc780 con 0x555cc72a11b0 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.337+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 msgr2=0x555cc767d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f58a000acc0 tx=0x7f58a002e930 comp rx=0 tx=0).stop 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 shutdown_connections 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc72ca070 0x555cc7674ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2028312808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72a11b0 0x555cc767d270 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.338+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 >> 172.21.15.83:0/2028312808 conn(0x555cc72be460 msgr2=0x555cc767cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.342+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 shutdown_connections 2024-09-21T03:31:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.342+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2028312808 wait complete. 2024-09-21T03:31:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.342+0000 7f58bb1ac740 1 -- start start 2024-09-21T03:31:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.342+0000 7f58bb1ac740 1 Processor -- start 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.343+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.343+0000 7f58bb1ac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72ca070 0x555cc7680960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58ae59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76854b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57692/0 (socket says 172.21.15.83:57692) 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58ae59c640 1 -- 172.21.15.83:0/2721979843 learned_addr learned my addr 172.21.15.83:0/2721979843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58add9b640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72ca070 0x555cc7680960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 0x555cc7680ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f589000fc40 con 0x555cc725dda0 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f589000f570 con 0x555cc72ca070 2024-09-21T03:31:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f589000c7d0 con 0x555cc72a9390 2024-09-21T03:31:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.344+0000 7f58bb1ac740 10 client.0 _ll_register_callbacks cb 0x7f58a8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.345+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 0x555cc7680ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.346+0000 7f58aed9d640 1 -- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 msgr2=0x555cc76854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.346+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.346+0000 7f58aed9d640 1 -- 172.21.15.83:0/2721979843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72ca070 msgr2=0x555cc7680960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.346+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72ca070 0x555cc7680960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.346+0000 7f58aed9d640 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc76815e0 con 0x555cc725dda0 2024-09-21T03:31:07.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.347+0000 7f58aed9d640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 0x555cc7680ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f58a4017590 tx=0x7f58a400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a77ce163517d547 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.348+0000 7f5895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:07.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.348+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4017800 con 0x555cc725dda0 2024-09-21T03:31:07.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.348+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cc75e9df0 con 0x555cc725dda0 2024-09-21T03:31:07.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.348+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8441+0+0 (secure 0 0 0) 0x7f58a4005020 con 0x555cc725dda0 2024-09-21T03:31:07.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.349+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f58a4005220 con 0x555cc725dda0 2024-09-21T03:31:07.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.350+0000 7f58bb1ac740 10 client.17778 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:07.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.350+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555cc76f97f0 con 0x555cc725dda0 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555cc75e9df0 con 0x555cc725dda0 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f58a401e070 con 0x555cc725dda0 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 10 client.17778 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 10 client.17778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 20 client.17778 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc76f9fd0 0x555cc76fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 4 client.17778 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.354+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cc75e58a0 con 0x555cc76f9fd0 2024-09-21T03:31:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.355+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc76f9fd0 0x555cc76fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.357+0000 7f58ae59c640 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc76f9fd0 0x555cc76fc3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c257f21cae53e993 server_cookie=744bce0c685bb4a6 in_seq=0 out_seq=0 2024-09-21T03:31:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.357+0000 7f5895ffb640 10 client.17778 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.358+0000 7f5895ffb640 1 -- 172.21.15.83:0/2721979843 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cc75e58a0 con 0x555cc76f9fd0 2024-09-21T03:31:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.358+0000 7f5895ffb640 10 client.17778 handle_command_reply: tid=0 2024-09-21T03:31:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.358+0000 7f58bb1ac740 1 client.17778 shutdown 2024-09-21T03:31:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.359+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 msgr2=0x555cc7680ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.359+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 0x555cc7680ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f58a4017590 tx=0x7f58a400b9d0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.360+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 shutdown_connections 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.360+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x555cc76f9fd0 0x555cc76fc3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cc725dda0 0x555cc7680ea0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555cc72ca070 0x555cc7680960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 --2- 172.21.15.83:0/2721979843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cc72a9390 0x555cc76854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 >> 172.21.15.83:0/2721979843 conn(0x555cc72be460 msgr2=0x555cc7674520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f5895ffb640 0 client.17778 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 shutdown_connections 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 1 -- 172.21.15.83:0/2721979843 wait complete. 2024-09-21T03:31:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.361+0000 7f58bb1ac740 20 client.17778 trim_cache size 0 max 16384 2024-09-21T03:31:07.364 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.363+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 msgr2=0x7f58980b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.363+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 0x7f58980b8c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x555cc7670320 tx=0x7f58a8009910 comp rx=0 tx=0).stop 2024-09-21T03:31:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.363+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 msgr2=0x7f58b00805e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.363+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a0005470 comp rx=0 tx=0).stop 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.363+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 shutdown_connections 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f58980b6730 0x7f58980b8c40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b0110490 0x7f58b0081060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b010fb40 0x7f58b0080b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 --2- 172.21.15.83:0/2665128787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58b010baf0 0x7f58b00805e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 >> 172.21.15.83:0/2665128787 conn(0x7f58b0070ec0 msgr2=0x7f58b010d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 shutdown_connections 2024-09-21T03:31:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:07.364+0000 7f58b4c38640 1 -- 172.21.15.83:0/2665128787 wait complete. 2024-09-21T03:31:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:07 smithi083 ceph-mon[31219]: pgmap v774: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-21T03:31:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:07.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:31:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:07 smithi142 ceph-mon[34512]: pgmap v774: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-21T03:31:07.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:07.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:31:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:07 smithi193 ceph-mon[35224]: pgmap v774: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-09-21T03:31:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:07.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:31:09.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:31:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:09 smithi142 ceph-mon[34512]: pgmap v775: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-21T03:31:09.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:09 smithi142 ceph-mon[34512]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:31:09.792 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:09 smithi193 ceph-mon[35224]: pgmap v775: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-21T03:31:09.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:09 smithi193 ceph-mon[35224]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:31:09.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:09 smithi083 ceph-mon[31219]: pgmap v775: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-21T03:31:09.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:09 smithi083 ceph-mon[31219]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:31:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 830.0 s, 985.3 tps, lat 89.709 ms stddev 114.293 2024-09-21T03:31:11.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.400+0000 7f6d8054d640 1 -- 172.21.15.83:0/2506829691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 msgr2=0x7f6d7810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.400+0000 7f6d8054d640 1 --2- 172.21.15.83:0/2506829691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7810adf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009900 tx=0x7f6d6802a440 comp rx=0 tx=0).stop 2024-09-21T03:31:11.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.407+0000 7f6d8054d640 1 -- 172.21.15.83:0/2506829691 shutdown_connections 2024-09-21T03:31:11.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.407+0000 7f6d8054d640 1 --2- 172.21.15.83:0/2506829691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7811cc30 0x7f6d7811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.407+0000 7f6d8054d640 1 --2- 172.21.15.83:0/2506829691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7810adf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.407+0000 7f6d8054d640 1 --2- 172.21.15.83:0/2506829691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d78111f40 0x7f6d7810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.407+0000 7f6d8054d640 1 -- 172.21.15.83:0/2506829691 >> 172.21.15.83:0/2506829691 conn(0x7f6d78070ec0 msgr2=0x7f6d780712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.410+0000 7f6d8054d640 1 -- 172.21.15.83:0/2506829691 shutdown_connections 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.410+0000 7f6d8054d640 1 -- 172.21.15.83:0/2506829691 wait complete. 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.411+0000 7f6d8054d640 1 Processor -- start 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 -- start start 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78111f40 0x7f6d78113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d7811cc30 0x7f6d78113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78122960 con 0x7f6d78111f40 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781227e0 con 0x7f6d7810a9e0 2024-09-21T03:31:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.413+0000 7f6d8054d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78122ae0 con 0x7f6d7811cc30 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.418+0000 7f6d7eaea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d7811cc30 0x7f6d78113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.418+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.418+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38264/0 (socket says 172.21.15.83:38264) 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.418+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/2242986638 learned_addr learned my addr 172.21.15.83:0/2242986638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.419+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78111f40 0x7f6d78113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.420+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/2242986638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d7811cc30 msgr2=0x7f6d78113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.420+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d7811cc30 0x7f6d78113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.420+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78111f40 msgr2=0x7f6d78113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.420+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78111f40 0x7f6d78113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.420+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/2242986638 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d78114500 con 0x7f6d7810a9e0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.425+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e9f0 tx=0x7f6d7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.425+0000 7f6d6f7fe640 1 -- 172.21.15.83:0/2242986638 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400cd80 con 0x7f6d7810a9e0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.425+0000 7f6d8054d640 1 -- 172.21.15.83:0/2242986638 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d78119150 con 0x7f6d7810a9e0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.426+0000 7f6d8054d640 1 -- 172.21.15.83:0/2242986638 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d78119730 con 0x7f6d7810a9e0 2024-09-21T03:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.429+0000 7f6d86ac1740 1 Processor -- start 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.430+0000 7f6d6f7fe640 1 -- 172.21.15.83:0/2242986638 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d740108e0 con 0x7f6d7810a9e0 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.430+0000 7f6d6f7fe640 1 -- 172.21.15.83:0/2242986638 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74018980 con 0x7f6d7810a9e0 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 -- start start 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba974390 0x55fcbad3cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fcbacb2ea0 con 0x55fcba974390 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fcbaca22d0 con 0x55fcba995070 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.431+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcbaca60e0 con 0x55fcba96c1b0 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.432+0000 7f6d6f7fe640 1 -- 172.21.15.83:0/2242986638 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6d74018be0 con 0x7f6d7810a9e0 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.432+0000 7f6d7eaea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.433+0000 7f6d7eaea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38266/0 (socket says 172.21.15.83:38266) 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.433+0000 7f6d7eaea640 1 -- 172.21.15.83:0/770653823 learned_addr learned my addr 172.21.15.83:0/770653823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:11.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.433+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba974390 0x55fcbad3cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.433+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.434+0000 7f6d6f7fe640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 0x7f6d640b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.435+0000 7f6d6f7fe640 1 -- 172.21.15.83:0/2242986638 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f6d740e8a20 con 0x7f6d7810a9e0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.435+0000 7f6d7eaea640 1 -- 172.21.15.83:0/770653823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 msgr2=0x55fcbad3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.435+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad3c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.435+0000 7f6d7eaea640 1 -- 172.21.15.83:0/770653823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba974390 msgr2=0x55fcbad3cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.435+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba974390 0x55fcbad3cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.436+0000 7f6d7eaea640 1 -- 172.21.15.83:0/770653823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbaa00650 con 0x55fcba995070 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.438+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007d90 tx=0x7f6d7000c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa2bf7f6d09fb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.438+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 0x7f6d640b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.442+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 0x7f6d640b8cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6d7001b000 tx=0x7f6d7001c7a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.443+0000 7f6d8054d640 1 -- 172.21.15.83:0/770653823 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70017030 con 0x55fcba995070 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.444+0000 7f6d8054d640 1 -- 172.21.15.83:0/770653823 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7001d9b0 con 0x55fcba995070 2024-09-21T03:31:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.444+0000 7f6d8054d640 1 -- 172.21.15.83:0/770653823 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000d040 con 0x55fcba995070 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.445+0000 7f6d86ac1740 1 -- 172.21.15.83:0/770653823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 msgr2=0x55fcbad3f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.445+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007d90 tx=0x7f6d7000c540 comp rx=0 tx=0).stop 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.449+0000 7f6d86ac1740 1 -- 172.21.15.83:0/770653823 shutdown_connections 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.449+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba974390 0x55fcbad3cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.449+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.449+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/770653823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba995070 0x55fcbad3f3a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.449+0000 7f6d86ac1740 1 -- 172.21.15.83:0/770653823 >> 172.21.15.83:0/770653823 conn(0x55fcba989460 msgr2=0x55fcbad3a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.450+0000 7f6d86ac1740 1 -- 172.21.15.83:0/770653823 shutdown_connections 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.451+0000 7f6d86ac1740 1 -- 172.21.15.83:0/770653823 wait complete. 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.451+0000 7f6d86ac1740 1 -- start start 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.451+0000 7f6d86ac1740 1 Processor -- start 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad47fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fcbab03700 con 0x55fcba995070 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fcbac3d160 con 0x55fcba974390 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcbad4ae50 con 0x55fcba96c1b0 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.457+0000 7f6d86ac1740 10 client.0 _ll_register_callbacks cb 0x55fcbac569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7eaea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7dae8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7eaea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad48a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56698/0 (socket says 172.21.15.83:56698) 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7dae8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38270/0 (socket says 172.21.15.83:38270) 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7eaea640 1 -- 172.21.15.83:0/1396454079 learned_addr learned my addr 172.21.15.83:0/1396454079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.461+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad47fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.464+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 msgr2=0x55fcbad484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.464+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.464+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1396454079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 msgr2=0x55fcbad48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.464+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad48a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.464+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1396454079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbac6ac90 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.465+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad47fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6d740e6130 tx=0x7f6d740b8180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=366b4676a011bcdf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.469+0000 7f6d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.469+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740aae80 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.469+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d7001d9b0 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.469+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f6d740ad500 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.472+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6d740adfa0 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.475+0000 7f6d86ac1740 10 client.27271 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.475+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fcbad56b70 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.477+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7001d9b0 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.477+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f6d740ad700 con 0x55fcba96c1b0 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 10 client.27271 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 10 client.27271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 20 client.27271 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:11.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba9539f0 0x55fcbad5a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 4 client.27271 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcbacb4df0 con 0x55fcba9539f0 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.478+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba9539f0 0x55fcbad5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.479+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba9539f0 0x55fcbad5a100 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c6377883af0b58 server_cookie=27f8194102a43c16 in_seq=0 out_seq=0 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.479+0000 7f6d61ffb640 10 client.27271 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.482+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1396454079 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcbacb4df0 con 0x55fcba9539f0 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.482+0000 7f6d61ffb640 10 client.27271 handle_command_reply: tid=0 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.482+0000 7f6d86ac1740 1 client.27271 shutdown 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.484+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 msgr2=0x55fcbad47fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.484+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad47fa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6d740e6130 tx=0x7f6d740b8180 comp rx=0 tx=0).stop 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 shutdown_connections 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba9539f0 0x55fcbad5a100 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1396454079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad47fa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.486+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 >> 172.21.15.83:0/1396454079 conn(0x55fcba989460 msgr2=0x55fcbad3e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.489+0000 7f6d61ffb640 0 client.27271 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.492+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 shutdown_connections 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.492+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1396454079 wait complete. 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.493+0000 7f6d86ac1740 20 client.27271 trim_cache size 0 max 16384 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.523+0000 7f6d86ac1740 1 Processor -- start 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.523+0000 7f6d86ac1740 1 -- start start 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c00e7c0 con 0x55fcba995070 2024-09-21T03:31:11.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c010300 con 0x55fcba974390 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c00ee90 con 0x55fcba96c1b0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47672/0 (socket says 172.21.15.83:47672) 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.524+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/4207529389 learned_addr learned my addr 172.21.15.83:0/4207529389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/4207529389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 msgr2=0x55fcbad3e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad3e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/4207529389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 msgr2=0x55fcbad3e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad3e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/4207529389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbad4aed0 con 0x55fcba96c1b0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.525+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6d740abaa0 tx=0x7f6d740d2020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28831ca2cdc7ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.526+0000 7f6d63fff640 1 -- 172.21.15.83:0/4207529389 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740ae180 con 0x55fcba96c1b0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.526+0000 7f6d63fff640 1 -- 172.21.15.83:0/4207529389 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d740e8e50 con 0x55fcba96c1b0 2024-09-21T03:31:11.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.526+0000 7f6d63fff640 1 -- 172.21.15.83:0/4207529389 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740ae600 con 0x55fcba96c1b0 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 -- 172.21.15.83:0/4207529389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 msgr2=0x55fcbad49e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6d740abaa0 tx=0x7f6d740d2020 comp rx=0 tx=0).stop 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 -- 172.21.15.83:0/4207529389 shutdown_connections 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad3e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad3e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/4207529389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad49e80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.527+0000 7f6d86ac1740 1 -- 172.21.15.83:0/4207529389 >> 172.21.15.83:0/4207529389 conn(0x55fcba989460 msgr2=0x55fcbad3d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.528+0000 7f6d86ac1740 1 -- 172.21.15.83:0/4207529389 shutdown_connections 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.528+0000 7f6d86ac1740 1 -- 172.21.15.83:0/4207529389 wait complete. 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.528+0000 7f6d86ac1740 1 -- start start 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.528+0000 7f6d86ac1740 1 Processor -- start 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c00fc30 con 0x55fcba995070 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c00f560 con 0x55fcba974390 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d7e2e9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47686/0 (socket says 172.21.15.83:47686) 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1284184499 learned_addr learned my addr 172.21.15.83:0/1284184499 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c00c7c0 con 0x55fcba96c1b0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d86ac1740 10 client.0 _ll_register_callbacks cb 0x7f6d740aacd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.529+0000 7f6d7eaea640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 msgr2=0x55fcbad4bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1284184499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 msgr2=0x55fcbad4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 -- 172.21.15.83:0/1284184499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbad4c6e0 con 0x55fcba96c1b0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d7e2e9640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d740e8e20 tx=0x7f6d740ae680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3488995915f8abc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:11.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.530+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740aa6d0 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.531+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d740e8e50 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.531+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f6d740c6b20 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.531+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6d740ae340 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.531+0000 7f6d86ac1740 10 client.27277 fetch_fsmap learned FSMap version 51 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.531+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fcbadc4820 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d740e8e50 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f6d740b44a0 con 0x55fcba96c1b0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 10 client.27277 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 10 client.27277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 20 client.27277 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba928da0 0x55fcbadc6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 4 client.27277 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.533+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcbacb08a0 con 0x55fcba928da0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.535+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba928da0 0x55fcbadc6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d61ffb640 10 client.27277 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d7dae8640 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba928da0 0x55fcbadc6d90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13fea94f9840f5a4 server_cookie=7ea72e8d2927a790 in_seq=0 out_seq=0 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d61ffb640 1 -- 172.21.15.83:0/1284184499 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcbacb08a0 con 0x55fcba928da0 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d61ffb640 10 client.27277 handle_command_reply: tid=0 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 client.27277 shutdown 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 msgr2=0x55fcbad50760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d740e8e20 tx=0x7f6d740ae680 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 shutdown_connections 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55fcba928da0 0x55fcbadc6d90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fcba995070 0x55fcbad4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d61ffb640 0 client.27277 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcba974390 0x55fcbad4bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 --2- 172.21.15.83:0/1284184499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcba96c1b0 0x55fcbad50760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.536+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 >> 172.21.15.83:0/1284184499 conn(0x55fcba989460 msgr2=0x55fcbad49260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 shutdown_connections 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d86ac1740 1 -- 172.21.15.83:0/1284184499 wait complete. 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d86ac1740 20 client.27277 trim_cache size 0 max 16384 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 msgr2=0x7f6d640b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 0x7f6d640b8cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6d7001b000 tx=0x7f6d7001c7a0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 msgr2=0x7f6d7811a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e9f0 tx=0x7f6d7400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 shutdown_connections 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f6d640b67b0 0x7f6d640b8cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d7811cc30 0x7f6d78113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78111f40 0x7f6d78113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 --2- 172.21.15.83:0/2242986638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d7810a9e0 0x7f6d7811a420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 >> 172.21.15.83:0/2242986638 conn(0x7f6d78070ec0 msgr2=0x7f6d78071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 shutdown_connections 2024-09-21T03:31:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:10.537+0000 7f6d6d7fa640 1 -- 172.21.15.83:0/2242986638 wait complete. 2024-09-21T03:31:11.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:11 smithi142 ceph-mon[34512]: pgmap v777: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:11.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:11 smithi193 ceph-mon[35224]: pgmap v777: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:11 smithi083 ceph-mon[31219]: pgmap v777: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:13.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:31:13.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:31:13.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:13 smithi083 ceph-mon[31219]: pgmap v778: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-09-21T03:31:13.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:13 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:13.655 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:13.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:13 smithi142 ceph-mon[34512]: pgmap v778: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-09-21T03:31:13.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:13 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:13.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:13 smithi193 ceph-mon[35224]: pgmap v778: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.99k op/s 2024-09-21T03:31:13.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:13 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:15.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:15 smithi142 ceph-mon[34512]: pgmap v779: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 2.05k op/s 2024-09-21T03:31:15.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.692+0000 7f697f91f640 1 -- 172.21.15.83:0/3253425595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 msgr2=0x7f697810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.692+0000 7f697f91f640 1 --2- 172.21.15.83:0/3253425595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 0x7f697810bb00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f697001acb0 tx=0x7f697003b8e0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:15 smithi083 ceph-mon[31219]: pgmap v779: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 2.05k op/s 2024-09-21T03:31:15.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 -- 172.21.15.83:0/3253425595 shutdown_connections 2024-09-21T03:31:15.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 --2- 172.21.15.83:0/3253425595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697810c040 0x7f6978112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 --2- 172.21.15.83:0/3253425595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 0x7f697810bb00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 --2- 172.21.15.83:0/3253425595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f69781061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 -- 172.21.15.83:0/3253425595 >> 172.21.15.83:0/3253425595 conn(0x7f69781017c0 msgr2=0x7f6978103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.693+0000 7f697f91f640 1 -- 172.21.15.83:0/3253425595 shutdown_connections 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.697+0000 7f697f91f640 1 -- 172.21.15.83:0/3253425595 wait complete. 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 Processor -- start 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 -- start start 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 0x7f6978073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697810c040 0x7f69780739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f697806f670 con 0x7f697810c040 2024-09-21T03:31:15.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f697806f4f0 con 0x7f6978105db0 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697f91f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f697806f7f0 con 0x7f697810b6f0 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.698+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.699+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38312/0 (socket says 172.21.15.83:38312) 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.699+0000 7f697d6bb640 1 -- 172.21.15.83:0/1413700147 learned_addr learned my addr 172.21.15.83:0/1413700147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.700+0000 7f697d6bb640 1 -- 172.21.15.83:0/1413700147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 msgr2=0x7f6978073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.700+0000 7f697d6bb640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 0x7f6978073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.700+0000 7f697d6bb640 1 -- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697810c040 msgr2=0x7f69780739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.700+0000 7f697d6bb640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697810c040 0x7f69780739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.700+0000 7f697d6bb640 1 -- 172.21.15.83:0/1413700147 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6978073f10 con 0x7f6978105db0 2024-09-21T03:31:15.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.706+0000 7f697d6bb640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f696800c890 tx=0x7f696800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.712+0000 7f696e7fc640 1 -- 172.21.15.83:0/1413700147 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968004220 con 0x7f6978105db0 2024-09-21T03:31:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.712+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6978074200 con 0x7f6978105db0 2024-09-21T03:31:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.712+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697819c410 con 0x7f6978105db0 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.715+0000 7f6985e93740 1 Processor -- start 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.715+0000 7f696e7fc640 1 -- 172.21.15.83:0/1413700147 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69680043c0 con 0x7f6978105db0 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.715+0000 7f696e7fc640 1 -- 172.21.15.83:0/1413700147 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968004a00 con 0x7f6978105db0 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 -- start start 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382bff2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56382bf68e60 con 0x56382bc4b350 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56382bf58290 con 0x56382bc22490 2024-09-21T03:31:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.717+0000 7f6985e93740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56382bf5c0a0 con 0x56382bc2a670 2024-09-21T03:31:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697ceba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697ceba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47716/0 (socket says 172.21.15.83:47716) 2024-09-21T03:31:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697ceba640 1 -- 172.21.15.83:0/2140550275 learned_addr learned my addr 172.21.15.83:0/2140550275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697ceba640 1 -- 172.21.15.83:0/2140550275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 msgr2=0x56382bff2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.719+0000 7f697ceba640 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.720+0000 7f697ceba640 1 -- 172.21.15.83:0/2140550275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 msgr2=0x56382bff2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.720+0000 7f697ceba640 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382bff2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.720+0000 7f697ceba640 1 -- 172.21.15.83:0/2140550275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382bcb6930 con 0x56382bc2a670 2024-09-21T03:31:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.726+0000 7f697ceba640 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f697001ac00 tx=0x7f697001a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25853d78072eb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.727+0000 7f696e7fc640 1 -- 172.21.15.83:0/1413700147 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f69680079b0 con 0x7f6978105db0 2024-09-21T03:31:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.728+0000 7f696e7fc640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 0x7f69640b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.728+0000 7f696e7fc640 1 -- 172.21.15.83:0/1413700147 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f69680e7730 con 0x7f6978105db0 2024-09-21T03:31:15.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.730+0000 7f697debc640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 0x7f69640b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.733+0000 7f69637fe640 1 -- 172.21.15.83:0/2140550275 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970011020 con 0x56382bc2a670 2024-09-21T03:31:15.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.733+0000 7f69637fe640 1 -- 172.21.15.83:0/2140550275 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970047040 con 0x56382bc2a670 2024-09-21T03:31:15.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.733+0000 7f69637fe640 1 -- 172.21.15.83:0/2140550275 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970004040 con 0x56382bc2a670 2024-09-21T03:31:15.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.734+0000 7f6985e93740 1 -- 172.21.15.83:0/2140550275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 msgr2=0x56382bff2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.734+0000 7f6985e93740 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f697001ac00 tx=0x7f697001a480 comp rx=0 tx=0).stop 2024-09-21T03:31:15.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.739+0000 7f6985e93740 1 -- 172.21.15.83:0/2140550275 shutdown_connections 2024-09-21T03:31:15.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.739+0000 7f6985e93740 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382bff2eb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.739+0000 7f6985e93740 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.739+0000 7f6985e93740 1 --2- 172.21.15.83:0/2140550275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382bff2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.739+0000 7f6985e93740 1 -- 172.21.15.83:0/2140550275 >> 172.21.15.83:0/2140550275 conn(0x56382bc3f740 msgr2=0x56382bff0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.746+0000 7f697debc640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 0x7f69640b8d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6974009460 tx=0x7f69740093b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.749+0000 7f6985e93740 1 -- 172.21.15.83:0/2140550275 shutdown_connections 2024-09-21T03:31:15.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.749+0000 7f6985e93740 1 -- 172.21.15.83:0/2140550275 wait complete. 2024-09-21T03:31:15.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.750+0000 7f6985e93740 1 -- start start 2024-09-21T03:31:15.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.750+0000 7f6985e93740 1 Processor -- start 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.750+0000 7f6985e93740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 0x56382bffe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 0x56382bffe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382c02bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56382bdb9890 con 0x56382bc4b350 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56382bef2d40 con 0x56382bc2a670 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56382c022bc0 con 0x56382bc22490 2024-09-21T03:31:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.751+0000 7f6985e93740 10 client.0 _ll_register_callbacks cb 0x56382bf56c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:15.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.753+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 0x56382bffe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.754+0000 7f697debc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382c02bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.754+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 0x56382bffe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47726/0 (socket says 172.21.15.83:47726) 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.754+0000 7f697d6bb640 1 -- 172.21.15.83:0/2351210677 learned_addr learned my addr 172.21.15.83:0/2351210677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.754+0000 7f697ceba640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 0x56382bffe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.755+0000 7f697ceba640 1 -- 172.21.15.83:0/2351210677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 msgr2=0x56382bffe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.755+0000 7f697ceba640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 0x56382bffe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.755+0000 7f697ceba640 1 -- 172.21.15.83:0/2351210677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 msgr2=0x56382c02bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.755+0000 7f697ceba640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382c02bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.755+0000 7f697ceba640 1 -- 172.21.15.83:0/2351210677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382c02c520 con 0x56382bc2a670 2024-09-21T03:31:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.763+0000 7f697ceba640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 0x56382bffe600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f697003bed0 tx=0x7f69700426a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4df46e3ea3139 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.764+0000 7f6960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:31:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.765+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970050060 con 0x56382bc2a670 2024-09-21T03:31:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.765+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6970047040 con 0x56382bc2a670 2024-09-21T03:31:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.765+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f69700419f0 con 0x56382bc2a670 2024-09-21T03:31:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.769+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6970042c50 con 0x56382bc2a670 2024-09-21T03:31:15.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.770+0000 7f6985e93740 10 client.27329 fetch_fsmap learned FSMap version 52 2024-09-21T03:31:15.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.770+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56382c02ee50 con 0x56382bc2a670 2024-09-21T03:31:15.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.771+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970047040 con 0x56382bc2a670 2024-09-21T03:31:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.773+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f697000d450 con 0x56382bc2a670 2024-09-21T03:31:15.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 10 client.27329 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T03:31:15.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 10 client.27329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:15.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 20 client.27329 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:15.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bc09cd0 0x56382c0321b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 4 client.27329 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:15.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.775+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56382bf6adb0 con 0x56382bc09cd0 2024-09-21T03:31:15.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.778+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bc09cd0 0x56382c0321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.782+0000 7f6960ff9640 10 client.27329 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:15.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.782+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bc09cd0 0x56382c0321b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340324aca4ad9803 server_cookie=57a25905e87d2e39 in_seq=0 out_seq=0 2024-09-21T03:31:15.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.786+0000 7f6960ff9640 1 -- 172.21.15.83:0/2351210677 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56382bf6adb0 con 0x56382bc09cd0 2024-09-21T03:31:15.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.786+0000 7f6960ff9640 10 client.27329 handle_command_reply: tid=0 2024-09-21T03:31:15.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.786+0000 7f6985e93740 1 client.27329 shutdown 2024-09-21T03:31:15.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.789+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 msgr2=0x56382bffe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.789+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 0x56382bffe600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f697003bed0 tx=0x7f69700426a0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 shutdown_connections 2024-09-21T03:31:15.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bc09cd0 0x56382c0321b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc4b350 0x56382c02bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc2a670 0x56382bffe600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 --2- 172.21.15.83:0/2351210677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc22490 0x56382bffe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6960ff9640 0 client.27329 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 >> 172.21.15.83:0/2351210677 conn(0x56382bc3f740 msgr2=0x56382bff55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 shutdown_connections 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 1 -- 172.21.15.83:0/2351210677 wait complete. 2024-09-21T03:31:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.790+0000 7f6985e93740 20 client.27329 trim_cache size 0 max 16384 2024-09-21T03:31:15.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:15 smithi193 ceph-mon[35224]: pgmap v779: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 2.05k op/s 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.824+0000 7f6985e93740 1 Processor -- start 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.824+0000 7f6985e93740 1 -- start start 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc2a670 0x56382bff74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00e820 con 0x56382bc2a670 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f695c010390 con 0x56382bc22490 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f6985e93740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00eef0 con 0x56382bc4b350 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47740/0 (socket says 172.21.15.83:47740) 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 -- 172.21.15.83:0/4029047055 learned_addr learned my addr 172.21.15.83:0/4029047055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697d6bb640 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697ceba640 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc2a670 0x56382bff74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 -- 172.21.15.83:0/4029047055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 msgr2=0x56382bff5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 -- 172.21.15.83:0/4029047055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc2a670 msgr2=0x56382bff74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc2a670 0x56382bff74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.825+0000 7f697debc640 1 -- 172.21.15.83:0/4029047055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382bffcde0 con 0x56382bc4b350 2024-09-21T03:31:15.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.826+0000 7f697debc640 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f697400ed40 tx=0x7f6974011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f650d08a8e37c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.826+0000 7f6962ffd640 1 -- 172.21.15.83:0/4029047055 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974005020 con 0x56382bc4b350 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.826+0000 7f6962ffd640 1 -- 172.21.15.83:0/4029047055 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69740047a0 con 0x56382bc4b350 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.826+0000 7f6962ffd640 1 -- 172.21.15.83:0/4029047055 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740266e0 con 0x56382bc4b350 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 -- 172.21.15.83:0/4029047055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 msgr2=0x56382bff7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f697400ed40 tx=0x7f6974011000 comp rx=0 tx=0).stop 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 -- 172.21.15.83:0/4029047055 shutdown_connections 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc4b350 0x56382bff7a00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc2a670 0x56382bff74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 --2- 172.21.15.83:0/4029047055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc22490 0x56382bff5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.827+0000 7f6985e93740 1 -- 172.21.15.83:0/4029047055 >> 172.21.15.83:0/4029047055 conn(0x56382bc3f740 msgr2=0x56382bff5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.828+0000 7f6985e93740 1 -- 172.21.15.83:0/4029047055 shutdown_connections 2024-09-21T03:31:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.828+0000 7f6985e93740 1 -- 172.21.15.83:0/4029047055 wait complete. 2024-09-21T03:31:15.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.829+0000 7f6985e93740 1 -- start start 2024-09-21T03:31:15.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.829+0000 7f6985e93740 1 Processor -- start 2024-09-21T03:31:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.832+0000 7f6985e93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.832+0000 7f6985e93740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382c033de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.832+0000 7f6985e93740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc4b350 0x56382c034320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f6985e93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00fcc0 con 0x56382bc22490 2024-09-21T03:31:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f6985e93740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00f5f0 con 0x56382bc4b350 2024-09-21T03:31:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f6985e93740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00c820 con 0x56382bc2a670 2024-09-21T03:31:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f6985e93740 10 client.0 _ll_register_callbacks cb 0x7f696801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f697d6bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56764/0 (socket says 172.21.15.83:56764) 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f697d6bb640 1 -- 172.21.15.83:0/2286986106 learned_addr learned my addr 172.21.15.83:0/2286986106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f697ceba640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382c033de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.833+0000 7f697debc640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc4b350 0x56382c034320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.834+0000 7f697d6bb640 1 -- 172.21.15.83:0/2286986106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 msgr2=0x56382c033de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.834+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382c033de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.834+0000 7f697d6bb640 1 -- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc4b350 msgr2=0x56382c034320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.834+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc4b350 0x56382c034320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.834+0000 7f697d6bb640 1 -- 172.21.15.83:0/2286986106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382c0349e0 con 0x56382bc22490 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f697d6bb640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6968007590 tx=0x7f6968004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2eb2bf18f7b139e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6960ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696801d040 con 0x56382bc22490 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f69680ac830 con 0x56382bc22490 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69740047a0 con 0x56382bc22490 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f69680e52c0 con 0x56382bc22490 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.835+0000 7f6985e93740 10 client.17805 fetch_fsmap learned FSMap version 52 2024-09-21T03:31:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.836+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56382c09d850 con 0x56382bc22490 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.836+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69740047a0 con 0x56382bc22490 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.836+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 52) ==== 8370+0+0 (secure 0 0 0) 0x7f69680a8f20 con 0x56382bc22490 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.836+0000 7f6985e93740 10 client.17805 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.837+0000 7f6985e93740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.837+0000 7f6985e93740 20 client.17805 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.837+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bbdf080 0x56382c09fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.837+0000 7f6985e93740 4 client.17805 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:31:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.837+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56382bf66860 con 0x56382bbdf080 2024-09-21T03:31:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.839+0000 7f697ceba640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bbdf080 0x56382c09fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.839+0000 7f697ceba640 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bbdf080 0x56382c09fff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941be6e5973a924e server_cookie=35e5b0041d7931b in_seq=0 out_seq=0 2024-09-21T03:31:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.839+0000 7f6960ff9640 10 client.17805 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:15.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.841+0000 7f6960ff9640 1 -- 172.21.15.83:0/2286986106 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56382bf66860 con 0x56382bbdf080 2024-09-21T03:31:15.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.841+0000 7f6960ff9640 10 client.17805 handle_command_reply: tid=0 2024-09-21T03:31:15.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.841+0000 7f6985e93740 1 client.17805 shutdown 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 msgr2=0x56382c035900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6968007590 tx=0x7f6968004be0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 shutdown_connections 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x56382bbdf080 0x56382c09fff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6960ff9640 0 client.17805 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56382bc4b350 0x56382c034320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56382bc2a670 0x56382c033de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 --2- 172.21.15.83:0/2286986106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382bc22490 0x56382c035900 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 >> 172.21.15.83:0/2286986106 conn(0x56382bc3f740 msgr2=0x56382bff3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 shutdown_connections 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 1 -- 172.21.15.83:0/2286986106 wait complete. 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.842+0000 7f6985e93740 20 client.17805 trim_cache size 0 max 16384 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "20ac5392-b139-40a5-bf76-f9084f445257", 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:31:15.845 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:31:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 msgr2=0x7f69640b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 0x7f69640b8d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6974009460 tx=0x7f69740093b0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 msgr2=0x7f6978112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f696800c890 tx=0x7f696800cd50 comp rx=0 tx=0).stop 2024-09-21T03:31:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 shutdown_connections 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f69640b6880 0x7f69640b8d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697810c040 0x7f69780739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f697810b6f0 0x7f6978073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 --2- 172.21.15.83:0/1413700147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6978105db0 0x7f6978112680 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 >> 172.21.15.83:0/1413700147 conn(0x7f69781017c0 msgr2=0x7f6978102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 shutdown_connections 2024-09-21T03:31:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:15.843+0000 7f697f91f640 1 -- 172.21.15.83:0/1413700147 wait complete. 2024-09-21T03:31:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 835.0 s, 1042.1 tps, lat 104.565 ms stddev 151.097 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:16 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:31:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:16.677 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.678 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:16 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:16.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:16.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:16.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:16.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:16 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: pgmap v780: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:17 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.714+0000 7f5b365a0640 1 -- 172.21.15.83:0/3102566616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b301021f0 msgr2=0x7f5b30102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.714+0000 7f5b365a0640 1 --2- 172.21.15.83:0/3102566616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b301021f0 0x7f5b30102600 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009900 tx=0x7f5b2002a410 comp rx=0 tx=0).stop 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 -- 172.21.15.83:0/3102566616 shutdown_connections 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 --2- 172.21.15.83:0/3102566616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30108480 0x7f5b3010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 --2- 172.21.15.83:0/3102566616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b30107b30 0x7f5b30107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 --2- 172.21.15.83:0/3102566616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b301021f0 0x7f5b30102600 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 -- 172.21.15.83:0/3102566616 >> 172.21.15.83:0/3102566616 conn(0x7f5b30077c60 msgr2=0x7f5b300780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 -- 172.21.15.83:0/3102566616 shutdown_connections 2024-09-21T03:31:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.715+0000 7f5b365a0640 1 -- 172.21.15.83:0/3102566616 wait complete. 2024-09-21T03:31:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 Processor -- start 2024-09-21T03:31:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 -- start start 2024-09-21T03:31:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 0x7f5b3019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 0x7f5b3019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b30108480 0x7f5b3019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301118c0 con 0x7f5b30108480 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.716+0000 7f5b365a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30111740 con 0x7f5b30107b30 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 0x7f5b3019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b365a0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30111a40 con 0x7f5b301021f0 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 0x7f5b3019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47752/0 (socket says 172.21.15.83:47752) 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b2ffff640 1 -- 172.21.15.83:0/1008010271 learned_addr learned my addr 172.21.15.83:0/1008010271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 0x7f5b3019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b30108480 0x7f5b3019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 -- 172.21.15.83:0/1008010271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 msgr2=0x7f5b3019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 0x7f5b3019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 -- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b30108480 msgr2=0x7f5b3019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b30108480 0x7f5b3019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 -- 172.21.15.83:0/1008010271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301a0490 con 0x7f5b30107b30 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b27fff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 0x7f5b3019aac0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5b30075ab0 tx=0x7f5b1800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.717+0000 7f5b2dffb640 1 -- 172.21.15.83:0/1008010271 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18017440 con 0x7f5b30107b30 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.718+0000 7f5b2dffb640 1 -- 172.21.15.83:0/1008010271 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18016890 con 0x7f5b30107b30 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.718+0000 7f5b2dffb640 1 -- 172.21.15.83:0/1008010271 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180157c0 con 0x7f5b30107b30 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.718+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301a0780 con 0x7f5b30107b30 2024-09-21T03:31:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.718+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a6ee0 con 0x7f5b30107b30 2024-09-21T03:31:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b3cb14740 1 Processor -- start 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b3cb14740 1 -- start start 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b2dffb640 1 -- 172.21.15.83:0/1008010271 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5b180159a0 con 0x7f5b30107b30 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488ae1b0 0x55ce48c7e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.721+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c7ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.722+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce48bf4e50 con 0x55ce488ae1b0 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.722+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce48be4280 con 0x55ce488b6390 2024-09-21T03:31:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.722+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce48be8090 con 0x55ce488d7070 2024-09-21T03:31:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b27fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c7ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47762/0 (socket says 172.21.15.83:47762) 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 -- 172.21.15.83:0/1373391182 learned_addr learned my addr 172.21.15.83:0/1373391182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 -- 172.21.15.83:0/1373391182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 msgr2=0x55ce48c7ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c7ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 -- 172.21.15.83:0/1373391182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488ae1b0 msgr2=0x55ce48c7e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.723+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488ae1b0 0x55ce48c7e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.724+0000 7f5b34b3d640 1 -- 172.21.15.83:0/1373391182 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce48942650 con 0x55ce488d7070 2024-09-21T03:31:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.724+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009f50 tx=0x7f5b280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4137074c82a58b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.724+0000 7f5b267fc640 1 -- 172.21.15.83:0/1373391182 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004d20 con 0x55ce488d7070 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.725+0000 7f5b267fc640 1 -- 172.21.15.83:0/1373391182 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b28004050 con 0x55ce488d7070 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.725+0000 7f5b2dffb640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 0x7f5b1c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.725+0000 7f5b2dffb640 1 -- 172.21.15.83:0/1008010271 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f5b180e7a60 con 0x7f5b30107b30 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.726+0000 7f5b3cb14740 1 -- 172.21.15.83:0/1373391182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 msgr2=0x55ce48c7e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.726+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009f50 tx=0x7f5b280118d0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.730+0000 7f5b3cb14740 1 -- 172.21.15.83:0/1373391182 shutdown_connections 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.730+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c7ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.730+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488ae1b0 0x55ce48c7e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.730+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/1373391182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488d7070 0x55ce48c7e270 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.730+0000 7f5b3cb14740 1 -- 172.21.15.83:0/1373391182 >> 172.21.15.83:0/1373391182 conn(0x55ce488cb460 msgr2=0x55ce48c7c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.734+0000 7f5b3cb14740 1 -- 172.21.15.83:0/1373391182 shutdown_connections 2024-09-21T03:31:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.734+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 0x7f5b1c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.737+0000 7f5b3cb14740 1 -- 172.21.15.83:0/1373391182 wait complete. 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.737+0000 7f5b3cb14740 1 -- start start 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.737+0000 7f5b3cb14740 1 Processor -- start 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c85d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c86290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce48a45700 con 0x55ce488d7070 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce48b7f110 con 0x55ce488ae1b0 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce48c952a0 con 0x55ce488b6390 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.738+0000 7f5b3cb14740 10 client.0 _ll_register_callbacks cb 0x55ce48b98990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.741+0000 7f5b34b3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c86290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.741+0000 7f5b34b3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c86290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56802/0 (socket says 172.21.15.83:56802) 2024-09-21T03:31:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.741+0000 7f5b34b3d640 1 -- 172.21.15.83:0/661664533 learned_addr learned my addr 172.21.15.83:0/661664533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.743+0000 7f5b27fff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c85d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.744+0000 7f5b27fff640 1 -- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 msgr2=0x55ce48c85810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.744+0000 7f5b27fff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c85810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.744+0000 7f5b27fff640 1 -- 172.21.15.83:0/661664533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 msgr2=0x55ce48c86290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.744+0000 7f5b27fff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c86290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.744+0000 7f5b27fff640 1 -- 172.21.15.83:0/661664533 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce48bacc40 con 0x55ce488b6390 2024-09-21T03:31:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.745+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 0x7f5b1c0b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b20031440 tx=0x7f5b20009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.749+0000 7f5b27fff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c85d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b180e51d0 tx=0x7f5b180ad6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198d80a4cbbbfa8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.750+0000 7f5b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.751+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180ad3e0 con 0x55ce488b6390 2024-09-21T03:31:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.751+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b28004050 con 0x55ce488b6390 2024-09-21T03:31:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.751+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f5b180163e0 con 0x55ce488b6390 2024-09-21T03:31:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.752+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5b180adfd0 con 0x55ce488b6390 2024-09-21T03:31:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.752+0000 7f5b3cb14740 10 client.27304 fetch_fsmap learned FSMap version 53 2024-09-21T03:31:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.753+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ce48c8b230 con 0x55ce488b6390 2024-09-21T03:31:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b28004050 con 0x55ce488b6390 2024-09-21T03:31:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f5b18015070 con 0x55ce488b6390 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 10 client.27304 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 10 client.27304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 20 client.27304 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce488959f0 0x55ce48c8e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 4 client.27304 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.755+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce48bf6da0 con 0x55ce488959f0 2024-09-21T03:31:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.758+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce488959f0 0x55ce48c8e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.760+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce488959f0 0x55ce48c8e790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde33800a95a89da server_cookie=c00693195f2fa5af in_seq=0 out_seq=0 2024-09-21T03:31:17.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.760+0000 7f5b0f7fe640 10 client.27304 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.762+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/661664533 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce48bf6da0 con 0x55ce488959f0 2024-09-21T03:31:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.762+0000 7f5b0f7fe640 10 client.27304 handle_command_reply: tid=0 2024-09-21T03:31:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.762+0000 7f5b3cb14740 1 client.27304 shutdown 2024-09-21T03:31:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.763+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 msgr2=0x55ce48c85d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.763+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c85d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b180e51d0 tx=0x7f5b180ad6a0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 shutdown_connections 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce488959f0 0x55ce48c8e790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c86290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c85d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/661664533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 >> 172.21.15.83:0/661664533 conn(0x55ce488cb460 msgr2=0x55ce48c81400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b0f7fe640 0 client.27304 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 shutdown_connections 2024-09-21T03:31:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.764+0000 7f5b3cb14740 1 -- 172.21.15.83:0/661664533 wait complete. 2024-09-21T03:31:17.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.765+0000 7f5b3cb14740 20 client.27304 trim_cache size 0 max 16384 2024-09-21T03:31:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 Processor -- start 2024-09-21T03:31:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 -- start start 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c7bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c9d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301ad640 con 0x55ce488d7070 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301af1e0 con 0x55ce488ae1b0 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301add10 con 0x55ce488b6390 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b34b3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c9d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b27fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c7bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:38404/0 (socket says 172.21.15.83:38404) 2024-09-21T03:31:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.795+0000 7f5b2ffff640 1 -- 172.21.15.83:0/4040021801 learned_addr learned my addr 172.21.15.83:0/4040021801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 -- 172.21.15.83:0/4040021801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 msgr2=0x55ce48c7bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c7bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 -- 172.21.15.83:0/4040021801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 msgr2=0x55ce48c9d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c9d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 -- 172.21.15.83:0/4040021801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce48c822c0 con 0x55ce488ae1b0 2024-09-21T03:31:17.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b20006010 tx=0x7f5b20040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f79aed43dbaee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b25ffb640 1 -- 172.21.15.83:0/4040021801 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20035040 con 0x55ce488ae1b0 2024-09-21T03:31:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b25ffb640 1 -- 172.21.15.83:0/4040021801 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200318a0 con 0x55ce488ae1b0 2024-09-21T03:31:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.796+0000 7f5b25ffb640 1 -- 172.21.15.83:0/4040021801 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2003c440 con 0x55ce488ae1b0 2024-09-21T03:31:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.797+0000 7f5b3cb14740 1 -- 172.21.15.83:0/4040021801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 msgr2=0x55ce48c7b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b20006010 tx=0x7f5b20040000 comp rx=0 tx=0).stop 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 -- 172.21.15.83:0/4040021801 shutdown_connections 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48c9d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488b6390 0x55ce48c7bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/4040021801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488ae1b0 0x55ce48c7b710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 -- 172.21.15.83:0/4040021801 >> 172.21.15.83:0/4040021801 conn(0x55ce488cb460 msgr2=0x55ce48c876f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 -- 172.21.15.83:0/4040021801 shutdown_connections 2024-09-21T03:31:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.798+0000 7f5b3cb14740 1 -- 172.21.15.83:0/4040021801 wait complete. 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.799+0000 7f5b3cb14740 1 -- start start 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.799+0000 7f5b3cb14740 1 Processor -- start 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c9ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48cb3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301aeb10 con 0x55ce488d7070 2024-09-21T03:31:17.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301ae440 con 0x55ce488b6390 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301ab640 con 0x55ce488ae1b0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b3cb14740 10 client.0 _ll_register_callbacks cb 0x7f5b1800ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b27fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c9ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:47792/0 (socket says 172.21.15.83:47792) 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 -- 172.21.15.83:0/870600101 learned_addr learned my addr 172.21.15.83:0/870600101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48cb3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 -- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 msgr2=0x55ce48c9ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c9ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 -- 172.21.15.83:0/870600101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 msgr2=0x55ce48cb3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48cb3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b2ffff640 1 -- 172.21.15.83:0/870600101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce48cb4360 con 0x55ce488ae1b0 2024-09-21T03:31:17.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.800+0000 7f5b34b3d640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48cb3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b2ffff640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b20031a90 tx=0x7f5b20031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6198b4d35aadea58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2005c070 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8485+0+0 (secure 0 0 0) 0x7f5b20003c70 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b200318a0 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5b20035030 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b3cb14740 10 client.27310 fetch_fsmap learned FSMap version 53 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ce48d171f0 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.801+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200035f0 con 0x55ce488ae1b0 2024-09-21T03:31:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 53) ==== 8374+0+0 (secure 0 0 0) 0x7f5b2003c770 con 0x55ce488ae1b0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 10 client.27310 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 10 client.27310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 20 client.27310 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce4886ada0 0x55ce48d19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 4 client.27310 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5b200318a0 con 0x55ce4886ada0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.802+0000 7f5b27fff640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce4886ada0 0x55ce48d19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b0f7fe640 10 client.27310 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b27fff640 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce4886ada0 0x55ce48d19950 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721b2c4de38875d6 server_cookie=7c56ac244b26bb3 in_seq=0 out_seq=0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b0f7fe640 1 -- 172.21.15.83:0/870600101 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x7f5b200318a0 con 0x55ce4886ada0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b0f7fe640 10 client.27310 handle_command_reply: tid=0 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 client.27310 shutdown 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 msgr2=0x55ce48c9a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b20031a90 tx=0x7f5b20031730 comp rx=0 tx=0).stop 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 shutdown_connections 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55ce4886ada0 0x55ce48d19950 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b0f7fe640 0 client.27310 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce488d7070 0x55ce48cb3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce488b6390 0x55ce48c9ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 --2- 172.21.15.83:0/870600101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce488ae1b0 0x55ce48c9a6f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 >> 172.21.15.83:0/870600101 conn(0x55ce488cb460 msgr2=0x55ce48c9de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 shutdown_connections 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.803+0000 7f5b3cb14740 1 -- 172.21.15.83:0/870600101 wait complete. 2024-09-21T03:31:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.804+0000 7f5b3cb14740 20 client.27310 trim_cache size 0 max 16384 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "20ac5392-b139-40a5-bf76-f9084f445257": { 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "20ac5392-b139-40a5-bf76-f9084f445257", 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:31:17.809 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:31:17.810 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.813 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.814 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.815 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "20ac5392-b139-40a5-bf76-f9084f445257": { 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:31:15.839749+0000", 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.818 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 msgr2=0x7f5b1c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 0x7f5b1c0b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b20031440 tx=0x7f5b20009260 comp rx=0 tx=0).stop 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 msgr2=0x7f5b3019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 0x7f5b3019aac0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5b30075ab0 tx=0x7f5b1800eef0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 shutdown_connections 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f5b1c0b6680 0x7f5b1c0b8b90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b30108480 0x7f5b3019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b30107b30 0x7f5b3019aac0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 --2- 172.21.15.83:0/1008010271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b301021f0 0x7f5b3019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 >> 172.21.15.83:0/1008010271 conn(0x7f5b30077c60 msgr2=0x7f5b30105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 shutdown_connections 2024-09-21T03:31:17.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:17.806+0000 7f5b365a0640 1 -- 172.21.15.83:0/1008010271 wait complete. 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: pgmap v780: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:31:17.822 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.822 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:17 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: pgmap v780: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-09-21T03:31:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:17.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:31:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:17.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:17 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:18.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:18 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:18.615 INFO:tasks.cephfs.filesystem:scrub status for tag:20ac5392-b139-40a5-bf76-f9084f445257 - {'path': '/', 'tag': '20ac5392-b139-40a5-bf76-f9084f445257', 'options': 'recursive,force'} 2024-09-21T03:31:18.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:18 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:18.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:18 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:19 smithi193 ceph-mon[35224]: pgmap v781: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-21T03:31:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:19 smithi193 ceph-mon[35224]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:31:19.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:19 smithi083 ceph-mon[31219]: pgmap v781: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-21T03:31:19.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:19 smithi083 ceph-mon[31219]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:31:19.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:19 smithi142 ceph-mon[34512]: pgmap v781: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-09-21T03:31:19.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:19 smithi142 ceph-mon[34512]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:31:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 840.0 s, 1227.4 tps, lat 82.768 ms stddev 96.815 2024-09-21T03:31:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:21 smithi193 ceph-mon[35224]: pgmap v783: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:21.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:21 smithi083 ceph-mon[31219]: pgmap v783: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:21.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:21 smithi142 ceph-mon[34512]: pgmap v783: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-09-21T03:31:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:23 smithi193 ceph-mon[35224]: pgmap v784: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T03:31:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:31:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:23 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:31:23.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:23 smithi083 ceph-mon[31219]: pgmap v784: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T03:31:23.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:31:23.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:23 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:31:23.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:23 smithi142 ceph-mon[34512]: pgmap v784: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T03:31:23.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:31:23.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:23 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:31:24.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:24 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:31:24.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:24 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:31:24.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:24 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:31:24.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:24 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:31:24.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:24 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:31:24.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:24 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:31:24.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:24 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:31:24.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:24 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:31:24.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:24 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:31:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:25 smithi193 ceph-mon[35224]: pgmap v785: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:31:25.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:25 smithi083 ceph-mon[31219]: pgmap v785: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:31:25.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:25 smithi142 ceph-mon[34512]: pgmap v785: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T03:31:25.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 845.0 s, 1321.2 tps, lat 76.762 ms stddev 81.941 2024-09-21T03:31:27.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:27 smithi193 ceph-mon[35224]: pgmap v786: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T03:31:27.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:27 smithi083 ceph-mon[31219]: pgmap v786: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T03:31:27.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:27 smithi142 ceph-mon[34512]: pgmap v786: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-21T03:31:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:29 smithi193 ceph-mon[35224]: pgmap v787: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 715 op/s 2024-09-21T03:31:29.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:29 smithi083 ceph-mon[31219]: pgmap v787: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 715 op/s 2024-09-21T03:31:29.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:29 smithi142 ceph-mon[34512]: pgmap v787: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 715 op/s 2024-09-21T03:31:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:30 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:30 smithi193 ceph-mon[35224]: pgmap v788: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 604 op/s 2024-09-21T03:31:30.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:30 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:30.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:30 smithi083 ceph-mon[31219]: pgmap v788: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 604 op/s 2024-09-21T03:31:30.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:30 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:31:30.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:30 smithi142 ceph-mon[34512]: pgmap v788: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 604 op/s 2024-09-21T03:31:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 850.0 s, 1239.0 tps, lat 80.181 ms stddev 97.736 2024-09-21T03:31:32.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:32 smithi083 ceph-mon[31219]: pgmap v789: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 927 op/s 2024-09-21T03:31:32.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:32 smithi142 ceph-mon[34512]: pgmap v789: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 927 op/s 2024-09-21T03:31:33.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:32 smithi193 ceph-mon[35224]: pgmap v789: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 927 op/s 2024-09-21T03:31:34.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:34 smithi083 ceph-mon[31219]: pgmap v790: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 1.06k op/s 2024-09-21T03:31:34.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:34 smithi142 ceph-mon[34512]: pgmap v790: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 1.06k op/s 2024-09-21T03:31:35.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:34 smithi193 ceph-mon[35224]: pgmap v790: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 1.06k op/s 2024-09-21T03:31:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 855.0 s, 1595.2 tps, lat 62.766 ms stddev 69.117 2024-09-21T03:31:36.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:36 smithi083 ceph-mon[31219]: pgmap v791: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-21T03:31:36.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:36 smithi142 ceph-mon[34512]: pgmap v791: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-21T03:31:37.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:36 smithi193 ceph-mon[35224]: pgmap v791: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-21T03:31:38.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:38 smithi083 ceph-mon[31219]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T03:31:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:38 smithi142 ceph-mon[34512]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T03:31:39.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:38 smithi193 ceph-mon[35224]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T03:31:40.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:40 smithi083 ceph-mon[31219]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-21T03:31:40.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:40 smithi142 ceph-mon[34512]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-21T03:31:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 860.0 s, 1202.2 tps, lat 78.451 ms stddev 92.397 2024-09-21T03:31:41.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:40 smithi193 ceph-mon[35224]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-21T03:31:42.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:42 smithi083 ceph-mon[31219]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 1.40k op/s 2024-09-21T03:31:42.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:42 smithi142 ceph-mon[34512]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 1.40k op/s 2024-09-21T03:31:43.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:42 smithi193 ceph-mon[35224]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 1.40k op/s 2024-09-21T03:31:44.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:44 smithi142 ceph-mon[34512]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-21T03:31:45.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:44 smithi193 ceph-mon[35224]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-21T03:31:45.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:44 smithi083 ceph-mon[31219]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.24k op/s 2024-09-21T03:31:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 865.0 s, 1861.3 tps, lat 56.976 ms stddev 100.183 2024-09-21T03:31:46.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:46 smithi142 ceph-mon[34512]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-09-21T03:31:47.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:46 smithi193 ceph-mon[35224]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-09-21T03:31:47.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:46 smithi083 ceph-mon[31219]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 1.13k op/s 2024-09-21T03:31:48.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:31:48.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:48 smithi142 ceph-mon[34512]: pgmap v797: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-21T03:31:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:49.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:48 smithi193 ceph-mon[35224]: pgmap v797: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-21T03:31:49.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:48 smithi083 ceph-mon[31219]: pgmap v797: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 -- 172.21.15.83:0/2112197749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0841094b0 msgr2=0x7ff0841098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2112197749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0841094b0 0x7ff0841098c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 -- 172.21.15.83:0/2112197749 shutdown_connections 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2112197749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084109e00 0x7ff084110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2112197749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0841094b0 0x7ff0841098c0 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2112197749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084101f70 0x7ff084102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 -- 172.21.15.83:0/2112197749 >> 172.21.15.83:0/2112197749 conn(0x7ff084077c60 msgr2=0x7ff0840780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 -- 172.21.15.83:0/2112197749 shutdown_connections 2024-09-21T03:31:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.590+0000 7ff08a3da640 1 -- 172.21.15.83:0/2112197749 wait complete. 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.591+0000 7ff08a3da640 1 Processor -- start 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.591+0000 7ff08a3da640 1 -- start start 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 0x7ff084103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff084103990 0x7ff0841a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff084113ac0 con 0x7ff084101f70 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff084113940 con 0x7ff084103990 2024-09-21T03:31:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 0x7ff084103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51860/0 (socket says 172.21.15.83:51860) 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 0x7ff084103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40564/0 (socket says 172.21.15.83:40564) 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 -- 172.21.15.83:0/2971111635 learned_addr learned my addr 172.21.15.83:0/2971111635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff088977640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff084103990 0x7ff0841a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff084113c40 con 0x7ff084109e00 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 -- 172.21.15.83:0/2971111635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 msgr2=0x7ff084103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 0x7ff084103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 -- 172.21.15.83:0/2971111635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff084103990 msgr2=0x7ff0841a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff084103990 0x7ff0841a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.592+0000 7ff083fff640 1 -- 172.21.15.83:0/2971111635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0841a8cf0 con 0x7ff084101f70 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff083fff640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9f0 tx=0x7ff07400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff081ffb640 1 -- 172.21.15.83:0/2971111635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07400ccc0 con 0x7ff084101f70 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff081ffb640 1 -- 172.21.15.83:0/2971111635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07400ce60 con 0x7ff084101f70 2024-09-21T03:31:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0841a8fe0 con 0x7ff084101f70 2024-09-21T03:31:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff081ffb640 1 -- 172.21.15.83:0/2971111635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074010430 con 0x7ff084101f70 2024-09-21T03:31:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.593+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a9440 con 0x7ff084101f70 2024-09-21T03:31:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.596+0000 7ff09094e740 1 Processor -- start 2024-09-21T03:31:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.596+0000 7ff09094e740 1 -- start start 2024-09-21T03:31:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.596+0000 7ff09094e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff09094e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b8b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff09094e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b8bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff09094e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561767b01e50 con 0x5617677bb1b0 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff09094e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561767af1280 con 0x5617677c3390 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff09094e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561767af5090 con 0x5617677e4070 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40570/0 (socket says 172.21.15.83:40570) 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 -- 172.21.15.83:0/3256976938 learned_addr learned my addr 172.21.15.83:0/3256976938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b8bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff081ffb640 1 -- 172.21.15.83:0/2971111635 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff0740105f0 con 0x7ff084101f70 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff083fff640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b8b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 -- 172.21.15.83:0/3256976938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 msgr2=0x561767b8bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b8bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 -- 172.21.15.83:0/3256976938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 msgr2=0x561767b8b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b8b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff088977640 1 -- 172.21.15.83:0/3256976938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176784f650 con 0x5617677e4070 2024-09-21T03:31:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.597+0000 7ff083fff640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b8b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.598+0000 7ff088977640 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff07c011440 tx=0x7ff07c011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b31052131f7338 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.598+0000 7ff07a7fc640 1 -- 172.21.15.83:0/3256976938 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c003c30 con 0x5617677e4070 2024-09-21T03:31:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.599+0000 7ff07a7fc640 1 -- 172.21.15.83:0/3256976938 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07c003dd0 con 0x5617677e4070 2024-09-21T03:31:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.599+0000 7ff07a7fc640 1 -- 172.21.15.83:0/3256976938 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00fae0 con 0x5617677e4070 2024-09-21T03:31:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.599+0000 7ff081ffb640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 0x7ff0700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.599+0000 7ff083fff640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 0x7ff0700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.599+0000 7ff081ffb640 1 -- 172.21.15.83:0/2971111635 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7ff0740e7b60 con 0x7ff084101f70 2024-09-21T03:31:49.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 -- 172.21.15.83:0/3256976938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 msgr2=0x561767b8b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff083fff640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 0x7ff0700b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x561767b89ef0 tx=0x7ff0740a9840 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff07c011440 tx=0x7ff07c011900 comp rx=0 tx=0).stop 2024-09-21T03:31:49.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 -- 172.21.15.83:0/3256976938 shutdown_connections 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b8bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b8b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 --2- 172.21.15.83:0/3256976938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b8b270 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 -- 172.21.15.83:0/3256976938 >> 172.21.15.83:0/3256976938 conn(0x5617677d8460 msgr2=0x561767b89160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 -- 172.21.15.83:0/3256976938 shutdown_connections 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.600+0000 7ff09094e740 1 -- 172.21.15.83:0/3256976938 wait complete. 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 -- start start 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 Processor -- start 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b9fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561767952700 con 0x5617677bb1b0 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561767a8c110 con 0x5617677c3390 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561767ba2180 con 0x5617677e4070 2024-09-21T03:31:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff09094e740 10 client.0 _ll_register_callbacks cb 0x561767aa5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.601+0000 7ff088977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff088977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b9f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51882/0 (socket says 172.21.15.83:51882) 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff088977640 1 -- 172.21.15.83:0/708767327 learned_addr learned my addr 172.21.15.83:0/708767327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff088977640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b9fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 -- 172.21.15.83:0/708767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 msgr2=0x561767b9fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b9fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 -- 172.21.15.83:0/708767327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 msgr2=0x561767b9f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b9f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 -- 172.21.15.83:0/708767327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561767ab9c40 con 0x5617677c3390 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff088977640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff07bfff640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b9f5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff08406aa10 tx=0x7ff06c032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57c7bc7e9568b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff0637fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:31:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.602+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c035040 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07c003dd0 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7ff06c0316a0 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff06c0318a0 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 10 client.27362 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561767bade30 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c005020 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7ff06c004890 con 0x5617677c3390 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 10 client.27362 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 20 client.27362 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677a29f0 0x561767bb1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 4 client.27362 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff07c003dd0 con 0x5617677a29f0 2024-09-21T03:31:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.603+0000 7ff088977640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677a29f0 0x561767bb1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.604+0000 7ff0637fe640 10 client.27362 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.605+0000 7ff088977640 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677a29f0 0x561767bb1350 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4790c7c54614ddf3 server_cookie=c0fae3a72b0799b6 in_seq=0 out_seq=0 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff0637fe640 1 -- 172.21.15.83:0/708767327 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff07c003dd0 con 0x5617677a29f0 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff0637fe640 10 client.27362 handle_command_reply: tid=0 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 client.27362 shutdown 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 msgr2=0x561767b9f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b9f5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff08406aa10 tx=0x7ff06c032dc0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 shutdown_connections 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677a29f0 0x561767bb1350 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677e4070 0x561767b9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677c3390 0x561767b9f5f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff0637fe640 0 client.27362 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 --2- 172.21.15.83:0/708767327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767b9f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 >> 172.21.15.83:0/708767327 conn(0x5617677d8460 msgr2=0x561767b87700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 shutdown_connections 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.606+0000 7ff09094e740 1 -- 172.21.15.83:0/708767327 wait complete. 2024-09-21T03:31:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.607+0000 7ff09094e740 20 client.27362 trim_cache size 0 max 16384 2024-09-21T03:31:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 Processor -- start 2024-09-21T03:31:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 -- start start 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 0x561767ba10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677c3390 0x561767b89160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841ad900 con 0x5617677c3390 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841af4a0 con 0x5617677bb1b0 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff09094e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841adfd0 con 0x5617677a29f0 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff088977640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 0x561767ba10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55712/0 (socket says 172.21.15.83:55712) 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff07bfff640 1 -- 172.21.15.83:0/3653129245 learned_addr learned my addr 172.21.15.83:0/3653129245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.629+0000 7ff088977640 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677c3390 0x561767b89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 -- 172.21.15.83:0/3653129245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 msgr2=0x561767ba10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 0x561767ba10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 -- 172.21.15.83:0/3653129245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677c3390 msgr2=0x561767b89160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677c3390 0x561767b89160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 -- 172.21.15.83:0/3653129245 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561767b896a0 con 0x5617677bb1b0 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff088977640 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 0x561767ba10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff06c003690 tx=0x7ff06c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2baeeddd27d034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff079ffb640 1 -- 172.21.15.83:0/3653129245 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c035070 con 0x5617677bb1b0 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff079ffb640 1 -- 172.21.15.83:0/3653129245 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c005dd0 con 0x5617677bb1b0 2024-09-21T03:31:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.630+0000 7ff079ffb640 1 -- 172.21.15.83:0/3653129245 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c005340 con 0x5617677bb1b0 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.631+0000 7ff09094e740 1 -- 172.21.15.83:0/3653129245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 msgr2=0x561767b9e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.631+0000 7ff09094e740 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff06c003690 tx=0x7ff06c005a20 comp rx=0 tx=0).stop 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 -- 172.21.15.83:0/3653129245 shutdown_connections 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677c3390 0x561767b89160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677bb1b0 0x561767b9e5c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 --2- 172.21.15.83:0/3653129245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677a29f0 0x561767ba10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 -- 172.21.15.83:0/3653129245 >> 172.21.15.83:0/3653129245 conn(0x5617677d8460 msgr2=0x561767b8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 -- 172.21.15.83:0/3653129245 shutdown_connections 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 -- 172.21.15.83:0/3653129245 wait complete. 2024-09-21T03:31:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 -- start start 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.632+0000 7ff09094e740 1 Processor -- start 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 0x561767ba3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 0x561767c1ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51892/0 (socket says 172.21.15.83:51892) 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff088977640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 0x561767ba3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff088977640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 0x561767ba3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55726/0 (socket says 172.21.15.83:55726) 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff07bfff640 1 -- 172.21.15.83:0/805406552 learned_addr learned my addr 172.21.15.83:0/805406552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841aedd0 con 0x5617677bb1b0 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841ae700 con 0x5617677a29f0 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841ab900 con 0x5617677c3390 2024-09-21T03:31:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.633+0000 7ff088977640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 0x561767c1ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 -- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 msgr2=0x561767ba3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 0x561767ba3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 -- 172.21.15.83:0/805406552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 msgr2=0x561767ba3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561767c1b2a0 con 0x5617677c3390 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff07bfff640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff09094e740 10 client.0 _ll_register_callbacks cb 0x7ff06c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff088977640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 0x561767c1ad60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x561767bb1b00 tx=0x7ff07c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a0f196bc2996ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff0637fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c005dd0 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff0637fe640 1 -- 172.21.15.83:0/805406552 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c005250 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.634+0000 7ff0637fe640 1 -- 172.21.15.83:0/805406552 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7ff07c0053f0 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff0637fe640 1 -- 172.21.15.83:0/805406552 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff07c005650 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 10 client.27328 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=54}) -- 0x561767c1c110 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff0637fe640 1 -- 172.21.15.83:0/805406552 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7ff07c0057f0 con 0x5617677c3390 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 10 client.27328 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 10 client.27328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 20 client.27328 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677e4070 0x561767c1e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 4 client.27328 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff06c005dd0 con 0x5617677e4070 2024-09-21T03:31:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.635+0000 7ff07bfff640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677e4070 0x561767c1e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff0637fe640 10 client.27328 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff07bfff640 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677e4070 0x561767c1e690 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bab1d99af9dd18e server_cookie=f767a2d08c4991ae in_seq=0 out_seq=0 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff0637fe640 1 -- 172.21.15.83:0/805406552 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x7ff06c005dd0 con 0x5617677e4070 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff0637fe640 10 client.27328 handle_command_reply: tid=0 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff09094e740 1 client.27328 shutdown 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 msgr2=0x561767c1ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.637+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 0x561767c1ad60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x561767bb1b00 tx=0x7ff07c009500 comp rx=0 tx=0).stop 2024-09-21T03:31:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 shutdown_connections 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x5617677e4070 0x561767c1e690 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617677c3390 0x561767c1ad60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5617677bb1b0 0x561767ba3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 --2- 172.21.15.83:0/805406552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617677a29f0 0x561767ba3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff0637fe640 0 client.27328 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 >> 172.21.15.83:0/805406552 conn(0x5617677d8460 msgr2=0x561767b8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 shutdown_connections 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 1 -- 172.21.15.83:0/805406552 wait complete. 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff09094e740 20 client.27328 trim_cache size 0 max 16384 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:31:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:31:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "20ac5392-b139-40a5-bf76-f9084f445257": { 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:31:15.839749+0000", 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:31:49.652 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 msgr2=0x7ff0700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.638+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 0x7ff0700b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x561767b89ef0 tx=0x7ff0740a9840 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 msgr2=0x7ff084102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9f0 tx=0x7ff07400edf0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 shutdown_connections 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff0700b6700 0x7ff0700b8c10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff084103990 0x7ff0841a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109e00 0x7ff084103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 --2- 172.21.15.83:0/2971111635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff084101f70 0x7ff084102f10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 >> 172.21.15.83:0/2971111635 conn(0x7ff084077c60 msgr2=0x7ff0841055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 shutdown_connections 2024-09-21T03:31:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:49.639+0000 7ff08a3da640 1 -- 172.21.15.83:0/2971111635 wait complete. 2024-09-21T03:31:50.574 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:31:50.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:50.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:50 smithi083 ceph-mon[31219]: pgmap v798: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:31:50.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:50 smithi142 ceph-mon[34512]: pgmap v798: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:31:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 870.0 s, 1275.3 tps, lat 78.223 ms stddev 86.796 2024-09-21T03:31:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:51.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:50 smithi193 ceph-mon[35224]: pgmap v798: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-09-21T03:31:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.575+0000 7f10792b9640 1 -- 172.21.15.83:0/4084036987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074077380 msgr2=0x7f10740757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.575+0000 7f10792b9640 1 --2- 172.21.15.83:0/4084036987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074077380 0x7f10740757b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1064009900 tx=0x7f106402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 -- 172.21.15.83:0/4084036987 shutdown_connections 2024-09-21T03:31:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 --2- 172.21.15.83:0/4084036987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1074114280 0x7f1074116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 --2- 172.21.15.83:0/4084036987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1074075dd0 0x7f10740761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 --2- 172.21.15.83:0/4084036987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074077380 0x7f10740757b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 -- 172.21.15.83:0/4084036987 >> 172.21.15.83:0/4084036987 conn(0x7f10740ff440 msgr2=0x7f1074101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 -- 172.21.15.83:0/4084036987 shutdown_connections 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.577+0000 7f10792b9640 1 -- 172.21.15.83:0/4084036987 wait complete. 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.578+0000 7f10792b9640 1 Processor -- start 2024-09-21T03:31:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.578+0000 7f10792b9640 1 -- start start 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1074077380 0x7f1074073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1074114280 0x7f1074109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f107406d290 con 0x7f1074075dd0 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f107406d110 con 0x7f1074077380 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f10792b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f107406d410 con 0x7f1074114280 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f1073fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f1073fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51918/0 (socket says 172.21.15.83:51918) 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.581+0000 7f1073fff640 1 -- 172.21.15.83:0/181436378 learned_addr learned my addr 172.21.15.83:0/181436378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.582+0000 7f1073fff640 1 -- 172.21.15.83:0/181436378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1074114280 msgr2=0x7f1074109a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:31:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.582+0000 7f1073fff640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1074114280 0x7f1074109a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.582+0000 7f1073fff640 1 -- 172.21.15.83:0/181436378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1074077380 msgr2=0x7f1074073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.582+0000 7f1073fff640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1074077380 0x7f1074073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.582+0000 7f1073fff640 1 -- 172.21.15.83:0/181436378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107410a040 con 0x7f1074075dd0 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.583+0000 7f1073fff640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1064005ab0 tx=0x7f1064030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.583+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064040070 con 0x7f1074075dd0 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.583+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106403c920 con 0x7f1074075dd0 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.583+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064035030 con 0x7f1074075dd0 2024-09-21T03:31:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.584+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107410a2d0 con 0x7f1074075dd0 2024-09-21T03:31:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.584+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107410a760 con 0x7f1074075dd0 2024-09-21T03:31:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.587+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10740761e0 con 0x7f1074075dd0 2024-09-21T03:31:51.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.589+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1064043070 con 0x7f1074075dd0 2024-09-21T03:31:51.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.590+0000 7f10717fa640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 0x7f105c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.591+0000 7f10737fe640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 0x7f105c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.594+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f10640315b0 con 0x7f1074075dd0 2024-09-21T03:31:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.597+0000 7f10737fe640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 0x7f105c0b8d10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1074116590 tx=0x7f1068005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:51.602+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f106404a030 con 0x7f1074075dd0 2024-09-21T03:31:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.019+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1074074780 con 0x7f1074075dd0 2024-09-21T03:31:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.020+0000 7f10717fa640 1 -- 172.21.15.83:0/181436378 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11985 (secure 0 0 0) 0x7f1074074780 con 0x7f1074075dd0 2024-09-21T03:31:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:31:52.022 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":54,"btime":"2024-09-21T03:31:29:481628+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:31:28.906136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:31:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 54 2024-09-21T03:31:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 msgr2=0x7f105c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 0x7f105c0b8d10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1074116590 tx=0x7f1068005c70 comp rx=0 tx=0).stop 2024-09-21T03:31:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 msgr2=0x7f1074073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1064005ab0 tx=0x7f1064030890 comp rx=0 tx=0).stop 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 shutdown_connections 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f105c0b6800 0x7f105c0b8d10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1074114280 0x7f1074109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1074077380 0x7f1074073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 --2- 172.21.15.83:0/181436378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1074075dd0 0x7f1074073650 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 >> 172.21.15.83:0/181436378 conn(0x7f10740ff440 msgr2=0x7f10740ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 shutdown_connections 2024-09-21T03:31:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:52.022+0000 7f10792b9640 1 -- 172.21.15.83:0/181436378 wait complete. 2024-09-21T03:31:52.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:31:52.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:52 smithi142 ceph-mon[34512]: pgmap v799: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:31:52.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:52 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/181436378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:53.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:52 smithi193 ceph-mon[35224]: pgmap v799: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:31:53.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:52 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/181436378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:53.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:52 smithi083 ceph-mon[31219]: pgmap v799: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:31:53.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:52 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/181436378' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 -- 172.21.15.83:0/2331209126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128102ae0 msgr2=0x7fc128102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2331209126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128102ae0 0x7fc128102ef0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc124009860 tx=0x7fc12402a340 comp rx=0 tx=0).stop 2024-09-21T03:31:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 -- 172.21.15.83:0/2331209126 shutdown_connections 2024-09-21T03:31:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2331209126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128108d20 0x7fc12810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2331209126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc1281087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2331209126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128102ae0 0x7fc128102ef0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 -- 172.21.15.83:0/2331209126 >> 172.21.15.83:0/2331209126 conn(0x7fc128079760 msgr2=0x7fc128079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 -- 172.21.15.83:0/2331209126 shutdown_connections 2024-09-21T03:31:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.605+0000 7fc1304fb640 1 -- 172.21.15.83:0/2331209126 wait complete. 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.606+0000 7fc1304fb640 1 Processor -- start 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.606+0000 7fc1304fb640 1 -- start start 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.606+0000 7fc1304fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 0x7fc12819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc1304fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc1304fb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128108d20 0x7fc12819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc1304fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc128112990 con 0x7fc1281083d0 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc1304fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc128112810 con 0x7fc128102ae0 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc1304fb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc128112b10 con 0x7fc128108d20 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12da96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12da96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51928/0 (socket says 172.21.15.83:51928) 2024-09-21T03:31:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12da96640 1 -- 172.21.15.83:0/2671515114 learned_addr learned my addr 172.21.15.83:0/2671515114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 0x7fc12819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12ea98640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128108d20 0x7fc12819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 -- 172.21.15.83:0/2671515114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128108d20 msgr2=0x7fc12819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128108d20 0x7fc12819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 -- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 msgr2=0x7fc12819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 -- 172.21.15.83:0/2671515114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1281a65a0 con 0x7fc128102ae0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12da96640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.607+0000 7fc12e297640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 0x7fc12819b2e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc124035040 tx=0x7fc124032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.608+0000 7fc11f7fe640 1 -- 172.21.15.83:0/2671515114 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124040020 con 0x7fc128102ae0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.608+0000 7fc11f7fe640 1 -- 172.21.15.83:0/2671515114 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124030d50 con 0x7fc128102ae0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.608+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1281a6830 con 0x7fc128102ae0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.608+0000 7fc11f7fe640 1 -- 172.21.15.83:0/2671515114 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037040 con 0x7fc128102ae0 2024-09-21T03:31:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.608+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1281a6d10 con 0x7fc128102ae0 2024-09-21T03:31:53.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.610+0000 7fc136a6f740 1 Processor -- start 2024-09-21T03:31:53.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.610+0000 7fc136a6f740 1 -- start start 2024-09-21T03:31:53.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.610+0000 7fc11f7fe640 1 -- 172.21.15.83:0/2671515114 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc124043020 con 0x7fc128102ae0 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.611+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.611+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 0x5609730b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.611+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.611+0000 7fc136a6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560973027ea0 con 0x560972d0a070 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc136a6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5609730172d0 con 0x560972ce9390 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc136a6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56097301b0e0 con 0x560972ce11b0 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 0x5609730b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.611+0000 7fc12da96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12da96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55770/0 (socket says 172.21.15.83:55770) 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12da96640 1 -- 172.21.15.83:0/2678151078 learned_addr learned my addr 172.21.15.83:0/2678151078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc11f7fe640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 0x7fc1140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12ea98640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12ea98640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 0x7fc1140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 -- 172.21.15.83:0/2678151078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 msgr2=0x5609730b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 -- 172.21.15.83:0/2678151078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 msgr2=0x5609730b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12e297640 1 -- 172.21.15.83:0/2678151078 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560972d75650 con 0x560972ce11b0 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12ea98640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.614+0000 7fc12da96640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc11f7fe640 1 -- 172.21.15.83:0/2671515114 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fc1241074e0 con 0x7fc128102ae0 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc12e297640 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 0x5609730b1be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc124104c80 tx=0x7fc1240c1be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8884f33981da8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc12ea98640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 0x7fc1140b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc120005fd0 tx=0x7fc120004ab0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc11cff9640 1 -- 172.21.15.83:0/2678151078 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240cd150 con 0x560972ce11b0 2024-09-21T03:31:53.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc11cff9640 1 -- 172.21.15.83:0/2678151078 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1240cce90 con 0x560972ce11b0 2024-09-21T03:31:53.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.615+0000 7fc11cff9640 1 -- 172.21.15.83:0/2678151078 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12410ac60 con 0x560972ce11b0 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 -- 172.21.15.83:0/2678151078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 msgr2=0x5609730b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 0x5609730b1be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc124104c80 tx=0x7fc1240c1be0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 -- 172.21.15.83:0/2678151078 shutdown_connections 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce9390 0x5609730b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce11b0 0x5609730b1be0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 --2- 172.21.15.83:0/2678151078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 -- 172.21.15.83:0/2678151078 >> 172.21.15.83:0/2678151078 conn(0x560972cfe460 msgr2=0x5609730af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 -- 172.21.15.83:0/2678151078 shutdown_connections 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.616+0000 7fc136a6f740 1 -- 172.21.15.83:0/2678151078 wait complete. 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 -- start start 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 Processor -- start 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 0x5609730bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560972e78700 con 0x560972d0a070 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560972fb2160 con 0x560972ce11b0 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609730c4050 con 0x560972ce9390 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc136a6f740 10 client.0 _ll_register_callbacks cb 0x560972fcb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12ea98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12da96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12da96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40652/0 (socket says 172.21.15.83:40652) 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12ea98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51946/0 (socket says 172.21.15.83:51946) 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12da96640 1 -- 172.21.15.83:0/3736066653 learned_addr learned my addr 172.21.15.83:0/3736066653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.617+0000 7fc12e297640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 0x5609730bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 -- 172.21.15.83:0/3736066653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 msgr2=0x5609730bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 0x5609730bcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 -- 172.21.15.83:0/3736066653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 msgr2=0x5609730cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 -- 172.21.15.83:0/3736066653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560972fdfc90 con 0x560972ce9390 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12e297640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 0x5609730bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc12da96640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc118011d10 tx=0x7fc11800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda74a420d5ea084 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11801e020 con 0x560972ce9390 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1240cce90 con 0x560972ce9390 2024-09-21T03:31:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fc118017050 con 0x560972ce9390 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc118005020 con 0x560972ce9390 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc136a6f740 10 client.27340 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5609730cfcd0 con 0x560972ce9390 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.618+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1240cce90 con 0x560972ce9390 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fc118019070 con 0x560972ce9390 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 10 client.27340 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 10 client.27340 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 20 client.27340 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972cc89f0 0x5609730d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 4 client.27340 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560973029df0 con 0x560972cc89f0 2024-09-21T03:31:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.619+0000 7fc12e297640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972cc89f0 0x5609730d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc111ffb640 10 client.27340 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:31:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc12e297640 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972cc89f0 0x5609730d3590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e8c9398642a66a1 server_cookie=333415c0f430c49c in_seq=0 out_seq=0 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc111ffb640 1 -- 172.21.15.83:0/3736066653 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560973029df0 con 0x560972cc89f0 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc111ffb640 10 client.27340 handle_command_reply: tid=0 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc136a6f740 1 client.27340 shutdown 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 msgr2=0x5609730bd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.620+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc118011d10 tx=0x7fc11800f790 comp rx=0 tx=0).stop 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 shutdown_connections 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972cc89f0 0x5609730d3590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972d0a070 0x5609730cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730bd270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3736066653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972ce11b0 0x5609730bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 >> 172.21.15.83:0/3736066653 conn(0x560972cfe460 msgr2=0x5609730b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc111ffb640 0 client.27340 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 shutdown_connections 2024-09-21T03:31:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 1 -- 172.21.15.83:0/3736066653 wait complete. 2024-09-21T03:31:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.621+0000 7fc136a6f740 20 client.27340 trim_cache size 0 max 16384 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.636+0000 7fc136a6f740 1 Processor -- start 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.636+0000 7fc136a6f740 1 -- start start 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ad410 con 0x560972ce11b0 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281aefb0 con 0x560972d0a070 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc136a6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281adae0 con 0x560972ce9390 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51954/0 (socket says 172.21.15.83:51954) 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 -- 172.21.15.83:0/3850324688 learned_addr learned my addr 172.21.15.83:0/3850324688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12ea98640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12da96640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 -- 172.21.15.83:0/3850324688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 msgr2=0x5609730cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 -- 172.21.15.83:0/3850324688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 msgr2=0x5609730b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 -- 172.21.15.83:0/3850324688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609730b3df0 con 0x560972ce11b0 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12ea98640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.637+0000 7fc12e297640 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc124104d30 tx=0x7fc12402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4e455c061e804e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.638+0000 7fc113fff640 1 -- 172.21.15.83:0/3850324688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240ccaf0 con 0x560972ce11b0 2024-09-21T03:31:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.638+0000 7fc113fff640 1 -- 172.21.15.83:0/3850324688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1240cce90 con 0x560972ce11b0 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.638+0000 7fc113fff640 1 -- 172.21.15.83:0/3850324688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240cccd0 con 0x560972ce11b0 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 -- 172.21.15.83:0/3850324688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 msgr2=0x5609730cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc124104d30 tx=0x7fc12402a340 comp rx=0 tx=0).stop 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 -- 172.21.15.83:0/3850324688 shutdown_connections 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 --2- 172.21.15.83:0/3850324688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730cce40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.639+0000 7fc136a6f740 1 -- 172.21.15.83:0/3850324688 >> 172.21.15.83:0/3850324688 conn(0x560972cfe460 msgr2=0x5609730bb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.640+0000 7fc136a6f740 1 -- 172.21.15.83:0/3850324688 shutdown_connections 2024-09-21T03:31:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.640+0000 7fc136a6f740 1 -- 172.21.15.83:0/3850324688 wait complete. 2024-09-21T03:31:53.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.641+0000 7fc136a6f740 1 -- start start 2024-09-21T03:31:53.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.641+0000 7fc136a6f740 1 Processor -- start 2024-09-21T03:31:53.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.641+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.641+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.641+0000 7fc136a6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc136a6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ae8e0 con 0x560972ce11b0 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc12da96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc136a6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ae210 con 0x560972d0a070 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc136a6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ab410 con 0x560972ce9390 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc12da96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40668/0 (socket says 172.21.15.83:40668) 2024-09-21T03:31:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc12da96640 1 -- 172.21.15.83:0/1799505423 learned_addr learned my addr 172.21.15.83:0/1799505423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc12e297640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.642+0000 7fc12ea98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc136a6f740 10 client.0 _ll_register_callbacks cb 0x7fc118009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc12da96640 1 -- 172.21.15.83:0/1799505423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 msgr2=0x5609730c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc12da96640 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc12da96640 1 -- 172.21.15.83:0/1799505423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 msgr2=0x5609730c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc12da96640 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.643+0000 7fc12da96640 1 -- 172.21.15.83:0/1799505423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609730c5850 con 0x560972ce9390 2024-09-21T03:31:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.644+0000 7fc12da96640 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc118011ce0 tx=0x7fc11800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308397025c50a349 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.648+0000 7fc111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:31:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.648+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11801e040 con 0x560972ce9390 2024-09-21T03:31:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.648+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fc118011e00 con 0x560972ce9390 2024-09-21T03:31:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.648+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1240cce90 con 0x560972ce9390 2024-09-21T03:31:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.663+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc1180053d0 con 0x560972ce9390 2024-09-21T03:31:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.663+0000 7fc136a6f740 10 client.27346 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.663+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56097313dea0 con 0x560972ce9390 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1240cce90 con 0x560972ce9390 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fc118019040 con 0x560972ce9390 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 10 client.27346 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 10 client.27346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 20 client.27346 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972c9dda0 0x560973140600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 4 client.27346 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.672+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609730258a0 con 0x560972c9dda0 2024-09-21T03:31:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.673+0000 7fc12e297640 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972c9dda0 0x560973140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.674+0000 7fc12e297640 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972c9dda0 0x560973140600 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6bc992c74e968b5 server_cookie=5a23535ca1e012f1 in_seq=0 out_seq=0 2024-09-21T03:31:53.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.675+0000 7fc111ffb640 10 client.27346 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:31:53.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.675+0000 7fc111ffb640 1 -- 172.21.15.83:0/1799505423 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609730258a0 con 0x560972c9dda0 2024-09-21T03:31:53.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.675+0000 7fc111ffb640 10 client.27346 handle_command_reply: tid=0 2024-09-21T03:31:53.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.675+0000 7fc136a6f740 1 client.27346 shutdown 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 msgr2=0x5609730c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc118011ce0 tx=0x7fc11800edb0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 shutdown_connections 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x560972c9dda0 0x560973140600 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560972d0a070 0x5609730c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560972ce9390 0x5609730c4b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 --2- 172.21.15.83:0/1799505423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560972ce11b0 0x5609730c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 >> 172.21.15.83:0/1799505423 conn(0x560972cfe460 msgr2=0x5609730b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.685+0000 7fc111ffb640 0 client.27346 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:31:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.688+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 shutdown_connections 2024-09-21T03:31:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.688+0000 7fc136a6f740 1 -- 172.21.15.83:0/1799505423 wait complete. 2024-09-21T03:31:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.688+0000 7fc136a6f740 20 client.27346 trim_cache size 0 max 16384 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.689+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 msgr2=0x7fc1140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.689+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 0x7fc1140b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc120005fd0 tx=0x7fc120004ab0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.689+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 msgr2=0x7fc12819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.689+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 0x7fc12819b2e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc124035040 tx=0x7fc124032d50 comp rx=0 tx=0).stop 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 shutdown_connections 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fc1140b6680 0x7fc1140b8b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128108d20 0x7fc12819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1281083d0 0x7fc12819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 --2- 172.21.15.83:0/2671515114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc128102ae0 0x7fc12819b2e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 >> 172.21.15.83:0/2671515114 conn(0x7fc128079760 msgr2=0x7fc128101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 shutdown_connections 2024-09-21T03:31:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:53.691+0000 7fc1304fb640 1 -- 172.21.15.83:0/2671515114 wait complete. 2024-09-21T03:31:54.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:31:54.891 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:54.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:54 smithi142 ceph-mon[34512]: pgmap v800: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.7 MiB/s wr, 970 op/s 2024-09-21T03:31:55.017 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:54 smithi083 ceph-mon[31219]: pgmap v800: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.7 MiB/s wr, 970 op/s 2024-09-21T03:31:55.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:54 smithi193 ceph-mon[35224]: pgmap v800: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.7 MiB/s wr, 970 op/s 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 -- 172.21.15.83:0/3704909990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe628108cf0 msgr2=0x7fe62810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 --2- 172.21.15.83:0/3704909990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe628108cf0 0x7fe62810f520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe620009ca0 tx=0x7fe62002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 -- 172.21.15.83:0/3704909990 shutdown_connections 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 --2- 172.21.15.83:0/3704909990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe628108cf0 0x7fe62810f520 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 --2- 172.21.15.83:0/3704909990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6281083a0 0x7fe6281087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 --2- 172.21.15.83:0/3704909990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 -- 172.21.15.83:0/3704909990 >> 172.21.15.83:0/3704909990 conn(0x7fe628079730 msgr2=0x7fe628079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 -- 172.21.15.83:0/3704909990 shutdown_connections 2024-09-21T03:31:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.451+0000 7fe62cc98640 1 -- 172.21.15.83:0/3704909990 wait complete. 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 Processor -- start 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 -- start start 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 0x7fe628199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 0x7fe62819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281129c0 con 0x7fe628102a60 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe628112840 con 0x7fe6281083a0 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62cc98640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe628112b40 con 0x7fe628108cf0 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe626d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 0x7fe62819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51982/0 (socket says 172.21.15.83:51982) 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe62659c640 1 -- 172.21.15.83:0/924988572 learned_addr learned my addr 172.21.15.83:0/924988572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.452+0000 7fe61ffff640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 0x7fe628199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 -- 172.21.15.83:0/924988572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 msgr2=0x7fe628199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 0x7fe628199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 -- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 msgr2=0x7fe628199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 -- 172.21.15.83:0/924988572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62819f3b0 con 0x7fe628108cf0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe61ffff640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 0x7fe628199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe62659c640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe626d9d640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 0x7fe62819ecb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe620009ca0 tx=0x7fe6200316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.453+0000 7fe61f7fe640 1 -- 172.21.15.83:0/924988572 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620003b70 con 0x7fe628108cf0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.454+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62819f640 con 0x7fe628108cf0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.454+0000 7fe61f7fe640 1 -- 172.21.15.83:0/924988572 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe620003d10 con 0x7fe628108cf0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.454+0000 7fe61f7fe640 1 -- 172.21.15.83:0/924988572 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62002fac0 con 0x7fe628108cf0 2024-09-21T03:31:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.454+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62806ae60 con 0x7fe628108cf0 2024-09-21T03:31:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.456+0000 7fe63320c740 1 Processor -- start 2024-09-21T03:31:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.456+0000 7fe61f7fe640 1 -- 172.21.15.83:0/924988572 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe62002fc60 con 0x7fe628108cf0 2024-09-21T03:31:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 -- start start 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770ccb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770903390 0x55d770ccbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d770c41ea0 con 0x55d770903390 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d770c312d0 con 0x55d7708fb1b0 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe63320c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d770c350e0 con 0x55d770924070 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe61f7fe640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 0x7fe6140b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40682/0 (socket says 172.21.15.83:40682) 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe61ffff640 1 -- 172.21.15.83:0/3966829323 learned_addr learned my addr 172.21.15.83:0/3966829323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe62659c640 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770ccb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe61ffff640 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770903390 0x55d770ccbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.457+0000 7fe626d9d640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 0x7fe6140b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61f7fe640 1 -- 172.21.15.83:0/924988572 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fe6201087d0 con 0x7fe628108cf0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 -- 172.21.15.83:0/3966829323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 msgr2=0x55d770ccb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770ccb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 -- 172.21.15.83:0/3966829323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770903390 msgr2=0x55d770ccbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770903390 0x55d770ccbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 -- 172.21.15.83:0/3966829323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77098f650 con 0x55d770924070 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61ffff640 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe610012800 tx=0x7fe610009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e0b73938b821ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe626d9d640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 0x7fe6140b8c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe6200cabe0 tx=0x7fe6200ce800 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61cff9640 1 -- 172.21.15.83:0/3966829323 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610012d80 con 0x55d770924070 2024-09-21T03:31:55.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61cff9640 1 -- 172.21.15.83:0/3966829323 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe610010c80 con 0x55d770924070 2024-09-21T03:31:55.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.458+0000 7fe61cff9640 1 -- 172.21.15.83:0/3966829323 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61001eb60 con 0x55d770924070 2024-09-21T03:31:55.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.460+0000 7fe63320c740 1 -- 172.21.15.83:0/3966829323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 msgr2=0x55d770ccb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.460+0000 7fe63320c740 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe610012800 tx=0x7fe610009700 comp rx=0 tx=0).stop 2024-09-21T03:31:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 -- 172.21.15.83:0/3966829323 shutdown_connections 2024-09-21T03:31:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770903390 0x55d770ccbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770ccb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 --2- 172.21.15.83:0/3966829323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ccb2c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 -- 172.21.15.83:0/3966829323 >> 172.21.15.83:0/3966829323 conn(0x55d770918460 msgr2=0x55d770cc91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 -- 172.21.15.83:0/3966829323 shutdown_connections 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 -- 172.21.15.83:0/3966829323 wait complete. 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.461+0000 7fe63320c740 1 -- start start 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 Processor -- start 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770903390 0x55d770cd6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770924070 0x55d770ce2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55826/0 (socket says 172.21.15.83:55826) 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 -- 172.21.15.83:0/2452570883 learned_addr learned my addr 172.21.15.83:0/2452570883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d770a92700 con 0x55d770924070 2024-09-21T03:31:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d770bcc160 con 0x55d7708fb1b0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d770cd9aa0 con 0x55d770903390 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770924070 0x55d770ce2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe62659c640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770903390 0x55d770cd6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe63320c740 10 client.0 _ll_register_callbacks cb 0x55d770be59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 -- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770903390 msgr2=0x55d770cd6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770903390 0x55d770cd6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 -- 172.21.15.83:0/2452570883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770924070 msgr2=0x55d770ce2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770924070 0x55d770ce2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.462+0000 7fe61ffff640 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d770bf9c90 con 0x55d7708fb1b0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe61ffff640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55d770cd9610 tx=0x7fe61001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224c22e4dfddaa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610014e70 con 0x55d7708fb1b0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe610010c80 con 0x55d7708fb1b0 2024-09-21T03:31:55.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fe610017070 con 0x55d7708fb1b0 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe61001ec80 con 0x55d7708fb1b0 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe63320c740 10 client.27401 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d770ce5790 con 0x55d7708fb1b0 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe610010c80 con 0x55d7708fb1b0 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.463+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fe61002a040 con 0x55d7708fb1b0 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 10 client.27401 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 10 client.27401 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 20 client.27401 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708e29f0 0x55d770ce8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 4 client.27401 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d770c43df0 con 0x55d7708e29f0 2024-09-21T03:31:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe62659c640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708e29f0 0x55d770ce8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe60dffb640 10 client.27401 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.464+0000 7fe62659c640 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708e29f0 0x55d770ce8b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f156597bf2c41b88 server_cookie=521fe1d60dc924b2 in_seq=0 out_seq=0 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.465+0000 7fe60dffb640 1 -- 172.21.15.83:0/2452570883 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d770c43df0 con 0x55d7708e29f0 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.465+0000 7fe60dffb640 10 client.27401 handle_command_reply: tid=0 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.465+0000 7fe63320c740 1 client.27401 shutdown 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.465+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 msgr2=0x55d770cd69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.465+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55d770cd9610 tx=0x7fe61001b040 comp rx=0 tx=0).stop 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 shutdown_connections 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d770924070 0x55d770ce2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708e29f0 0x55d770ce8b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770903390 0x55d770cd6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 --2- 172.21.15.83:0/2452570883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7708fb1b0 0x55d770cd69a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe60dffb640 0 client.27401 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 >> 172.21.15.83:0/2452570883 conn(0x55d770918460 msgr2=0x55d770cce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 shutdown_connections 2024-09-21T03:31:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 1 -- 172.21.15.83:0/2452570883 wait complete. 2024-09-21T03:31:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.466+0000 7fe63320c740 20 client.27401 trim_cache size 0 max 16384 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.480+0000 7fe63320c740 1 Processor -- start 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 -- start start 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770ccca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ce2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ad380 con 0x55d7708fb1b0 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281aef20 con 0x55d770903390 2024-09-21T03:31:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe63320c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ada50 con 0x55d770924070 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cd8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52000/0 (socket says 172.21.15.83:52000) 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe61ffff640 1 -- 172.21.15.83:0/3601110241 learned_addr learned my addr 172.21.15.83:0/3601110241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe62659c640 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770ccca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.481+0000 7fe61ffff640 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ce2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 -- 172.21.15.83:0/3601110241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 msgr2=0x55d770ce2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ce2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 -- 172.21.15.83:0/3601110241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 msgr2=0x55d770cd8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cd8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 -- 172.21.15.83:0/3601110241 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d770cd8f80 con 0x55d770903390 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe62659c640 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770ccca00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55d770bcc1e0 tx=0x7fe6180091c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b8f044a868ab123 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe60ffff640 1 -- 172.21.15.83:0/3601110241 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618031070 con 0x55d770903390 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe60ffff640 1 -- 172.21.15.83:0/3601110241 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe618022d10 con 0x55d770903390 2024-09-21T03:31:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.482+0000 7fe60ffff640 1 -- 172.21.15.83:0/3601110241 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618024650 con 0x55d770903390 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- 172.21.15.83:0/3601110241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 msgr2=0x55d770ccca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770ccca00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55d770bcc1e0 tx=0x7fe6180091c0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- 172.21.15.83:0/3601110241 shutdown_connections 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770ce2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770ccca00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 --2- 172.21.15.83:0/3601110241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- 172.21.15.83:0/3601110241 >> 172.21.15.83:0/3601110241 conn(0x55d770918460 msgr2=0x55d770ccc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- 172.21.15.83:0/3601110241 shutdown_connections 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- 172.21.15.83:0/3601110241 wait complete. 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 -- start start 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.483+0000 7fe63320c740 1 Processor -- start 2024-09-21T03:31:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770cda780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770cdacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ae850 con 0x55d7708fb1b0 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ae180 con 0x55d770903390 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ab380 con 0x55d770924070 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe63320c740 10 client.0 _ll_register_callbacks cb 0x7fe610009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52006/0 (socket says 172.21.15.83:52006) 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 -- 172.21.15.83:0/4054916142 learned_addr learned my addr 172.21.15.83:0/4054916142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770cdacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe62659c640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770cda780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 -- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 msgr2=0x55d770cdacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.484+0000 7fe61ffff640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770cdacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe61ffff640 1 -- 172.21.15.83:0/4054916142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 msgr2=0x55d770cda780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe61ffff640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770cda780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe61ffff640 1 -- 172.21.15.83:0/4054916142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d770cdb280 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe61ffff640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe610006010 tx=0x7fe610009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6fec1d5c2dbaed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610039040 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fe610017040 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe618022d10 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe610046050 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe63320c740 10 client.17874 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d770d53150 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.485+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe618022d10 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fe610005070 con 0x55d7708fb1b0 2024-09-21T03:31:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 10 client.17874 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 10 client.17874 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 20 client.17874 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708b7da0 0x55d770d558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 4 client.17874 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d770c3f8a0 con 0x55d7708b7da0 2024-09-21T03:31:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe62659c640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708b7da0 0x55d770d558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:55.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe60dffb640 10 client.17874 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:55.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.486+0000 7fe62659c640 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708b7da0 0x55d770d558e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9bad5880b49c123 server_cookie=b83951f015824988 in_seq=0 out_seq=0 2024-09-21T03:31:55.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.487+0000 7fe60dffb640 1 -- 172.21.15.83:0/4054916142 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d770c3f8a0 con 0x55d7708b7da0 2024-09-21T03:31:55.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.487+0000 7fe60dffb640 10 client.17874 handle_command_reply: tid=0 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.488+0000 7fe63320c740 1 client.17874 shutdown 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 msgr2=0x55d770cdf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe610006010 tx=0x7fe610009530 comp rx=0 tx=0).stop 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 shutdown_connections 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x55d7708b7da0 0x55d770d558e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d770924070 0x55d770cdacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d770903390 0x55d770cda780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 --2- 172.21.15.83:0/4054916142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7708fb1b0 0x55d770cdf2d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 >> 172.21.15.83:0/4054916142 conn(0x55d770918460 msgr2=0x55d770cd85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe60dffb640 0 client.17874 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 shutdown_connections 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 1 -- 172.21.15.83:0/4054916142 wait complete. 2024-09-21T03:31:55.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.489+0000 7fe63320c740 20 client.17874 trim_cache size 0 max 16384 2024-09-21T03:31:55.494 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 msgr2=0x7fe6140b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 0x7fe6140b8c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe6200cabe0 tx=0x7fe6200ce800 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 msgr2=0x7fe62819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 0x7fe62819ecb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe620009ca0 tx=0x7fe6200316f0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 shutdown_connections 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fe6140b6780 0x7fe6140b8c90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe628108cf0 0x7fe62819ecb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6281083a0 0x7fe628199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 --2- 172.21.15.83:0/924988572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe628102a60 0x7fe628199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 >> 172.21.15.83:0/924988572 conn(0x7fe628079730 msgr2=0x7fe628105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 shutdown_connections 2024-09-21T03:31:55.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:55.490+0000 7fe62cc98640 1 -- 172.21.15.83:0/924988572 wait complete. 2024-09-21T03:31:55.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 875.0 s, 1177.3 tps, lat 85.126 ms stddev 92.945 2024-09-21T03:31:56.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:31:56.616 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:56.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:56 smithi142 ceph-mon[34512]: pgmap v801: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-09-21T03:31:56.934 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:56 smithi083 ceph-mon[31219]: pgmap v801: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-09-21T03:31:57.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:56 smithi193 ceph-mon[35224]: pgmap v801: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-09-21T03:31:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/2176631952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08109e00 msgr2=0x7f3d08110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/2176631952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08109e00 0x7f3d08110630 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009ca0 tx=0x7f3d0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/2176631952 shutdown_connections 2024-09-21T03:31:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/2176631952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08109e00 0x7f3d08110630 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/2176631952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/2176631952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08103bc0 0x7f3d08103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/2176631952 >> 172.21.15.83:0/2176631952 conn(0x7f3d0807b910 msgr2=0x7f3d0807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/2176631952 shutdown_connections 2024-09-21T03:31:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.235+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/2176631952 wait complete. 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 Processor -- start 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 -- start start 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d0819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08109e00 0x7f3d0819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08113ac0 con 0x7f3d08103bc0 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08113940 con 0x7f3d08109e00 2024-09-21T03:31:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d0dfc3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08113c40 con 0x7f3d081094b0 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52024/0 (socket says 172.21.15.83:52024) 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d0819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d077fe640 1 -- 172.21.15.83:0/218456736 learned_addr learned my addr 172.21.15.83:0/218456736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.236+0000 7f3d07fff640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08109e00 0x7f3d0819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 -- 172.21.15.83:0/218456736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 msgr2=0x7f3d0819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d0819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 -- 172.21.15.83:0/218456736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08109e00 msgr2=0x7f3d0819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08109e00 0x7f3d0819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 -- 172.21.15.83:0/218456736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081a0460 con 0x7f3d08103bc0 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3cfffff640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d0819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d077fe640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d950 tx=0x7f3cf800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d057fa640 1 -- 172.21.15.83:0/218456736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8017070 con 0x7f3d08103bc0 2024-09-21T03:31:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d081a0750 con 0x7f3d08103bc0 2024-09-21T03:31:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.237+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a6eb0 con 0x7f3d08103bc0 2024-09-21T03:31:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.238+0000 7f3d057fa640 1 -- 172.21.15.83:0/218456736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf8009c30 con 0x7f3d08103bc0 2024-09-21T03:31:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.238+0000 7f3d057fa640 1 -- 172.21.15.83:0/218456736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8020060 con 0x7f3d08103bc0 2024-09-21T03:31:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.241+0000 7f3d057fa640 1 -- 172.21.15.83:0/218456736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3cf800f620 con 0x7f3d08103bc0 2024-09-21T03:31:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.241+0000 7f3d14537740 1 Processor -- start 2024-09-21T03:31:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.241+0000 7f3d14537740 1 -- start start 2024-09-21T03:31:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581a39c2ea0 con 0x5581a36a5070 2024-09-21T03:31:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581a39b22d0 con 0x5581a367c1b0 2024-09-21T03:31:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d14537740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581a39b60e0 con 0x5581a3684390 2024-09-21T03:31:57.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d057fa640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 0x7f3cf40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52040/0 (socket says 172.21.15.83:52040) 2024-09-21T03:31:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 -- 172.21.15.83:0/456845952 learned_addr learned my addr 172.21.15.83:0/456845952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 0x7f3cf40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3cfffff640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d077fe640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d057fa640 1 -- 172.21.15.83:0/218456736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f3cf80e6fa0 con 0x7f3d08103bc0 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 -- 172.21.15.83:0/456845952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 msgr2=0x5581a3a4cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a4cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 -- 172.21.15.83:0/456845952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 msgr2=0x5581a3a4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a4c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d07fff640 1 -- 172.21.15.83:0/456845952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a3710650 con 0x5581a36a5070 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.242+0000 7f3d077fe640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.243+0000 7f3d07fff640 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3d00031550 tx=0x7f3d00031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e4f1297b975133 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.243+0000 7f3d07fff640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 0x7f3cf40b8c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d000315f0 tx=0x7f3d0003b000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.243+0000 7f3cfdffb640 1 -- 172.21.15.83:0/456845952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0002f370 con 0x5581a36a5070 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.243+0000 7f3cfdffb640 1 -- 172.21.15.83:0/456845952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00036040 con 0x5581a36a5070 2024-09-21T03:31:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.243+0000 7f3cfdffb640 1 -- 172.21.15.83:0/456845952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0002f8c0 con 0x5581a36a5070 2024-09-21T03:31:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 -- 172.21.15.83:0/456845952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 msgr2=0x5581a3a4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3d00031550 tx=0x7f3d00031580 comp rx=0 tx=0).stop 2024-09-21T03:31:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 -- 172.21.15.83:0/456845952 shutdown_connections 2024-09-21T03:31:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 --2- 172.21.15.83:0/456845952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a4c2c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 -- 172.21.15.83:0/456845952 >> 172.21.15.83:0/456845952 conn(0x5581a3699460 msgr2=0x5581a3a4a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 -- 172.21.15.83:0/456845952 shutdown_connections 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.244+0000 7f3d14537740 1 -- 172.21.15.83:0/456845952 wait complete. 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.245+0000 7f3d14537740 1 -- start start 2024-09-21T03:31:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.245+0000 7f3d14537740 1 Processor -- start 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.245+0000 7f3d14537740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 0x5581a3a57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a57f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a36a5070 0x5581a3a63d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581a3813700 con 0x5581a367c1b0 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5581a394d160 con 0x5581a36a5070 2024-09-21T03:31:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581a3a5aa40 con 0x5581a3684390 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d14537740 10 client.0 _ll_register_callbacks cb 0x5581a39669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a57f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 0x5581a3a57a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a57f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:40752/0 (socket says 172.21.15.83:40752) 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3cfffff640 1 -- 172.21.15.83:0/474217556 learned_addr learned my addr 172.21.15.83:0/474217556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d07fff640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a36a5070 0x5581a3a63d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 -- 172.21.15.83:0/474217556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 msgr2=0x5581a3a57f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a57f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 -- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a36a5070 msgr2=0x5581a3a63d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a36a5070 0x5581a3a63d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 -- 172.21.15.83:0/474217556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a397ac90 con 0x5581a367c1b0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3d077fe640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 0x5581a3a57a20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80e4740 tx=0x7f3cf80acdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4824834b8b91e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.246+0000 7f3ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80af070 con 0x5581a367c1b0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f3cf80a8840 con 0x5581a367c1b0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d00036040 con 0x5581a367c1b0 2024-09-21T03:31:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3cf80aca00 con 0x5581a367c1b0 2024-09-21T03:31:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3d14537740 10 client.17886 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5581a3a66750 con 0x5581a367c1b0 2024-09-21T03:31:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.247+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00036040 con 0x5581a367c1b0 2024-09-21T03:31:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f3cf80a8a40 con 0x5581a367c1b0 2024-09-21T03:31:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 10 client.17886 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 10 client.17886 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 20 client.17886 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a36639f0 0x5581a3a69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 4 client.17886 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581a39c4df0 con 0x5581a36639f0 2024-09-21T03:31:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3cfffff640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a36639f0 0x5581a3a69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3ce6ffd640 10 client.17886 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.248+0000 7f3cfffff640 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a36639f0 0x5581a3a69d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7093725f7e27a9d3 server_cookie=e929ae7c3a59a525 in_seq=0 out_seq=0 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/474217556 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581a39c4df0 con 0x5581a36639f0 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3ce6ffd640 10 client.17886 handle_command_reply: tid=0 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 client.17886 shutdown 2024-09-21T03:31:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 msgr2=0x5581a3a57a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 0x5581a3a57a20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80e4740 tx=0x7f3cf80acdf0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 shutdown_connections 2024-09-21T03:31:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a36639f0 0x5581a3a69d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a36a5070 0x5581a3a63d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a3684390 0x5581a3a57f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 --2- 172.21.15.83:0/474217556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a367c1b0 0x5581a3a57a20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3ce6ffd640 0 client.17886 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.249+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 >> 172.21.15.83:0/474217556 conn(0x5581a3699460 msgr2=0x5581a3a4f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.250+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 shutdown_connections 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.250+0000 7f3d14537740 1 -- 172.21.15.83:0/474217556 wait complete. 2024-09-21T03:31:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.250+0000 7f3d14537740 20 client.17886 trim_cache size 0 max 16384 2024-09-21T03:31:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.264+0000 7f3d14537740 1 Processor -- start 2024-09-21T03:31:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.264+0000 7f3d14537740 1 -- start start 2024-09-21T03:31:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a367c1b0 0x5581a3a57ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 0x5581a3a63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce800e630 con 0x5581a36a5070 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce80101d0 con 0x5581a3684390 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d14537740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce800ed00 con 0x5581a367c1b0 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 0x5581a3a63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52062/0 (socket says 172.21.15.83:52062) 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a367c1b0 0x5581a3a57ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3d07fff640 1 -- 172.21.15.83:0/3464850418 learned_addr learned my addr 172.21.15.83:0/3464850418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 -- 172.21.15.83:0/3464850418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a367c1b0 msgr2=0x5581a3a57ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a367c1b0 0x5581a3a57ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 -- 172.21.15.83:0/3464850418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 msgr2=0x5581a3a641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.265+0000 7f3cfffff640 1 -- 172.21.15.83:0/3464850418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a3a58ce0 con 0x5581a3684390 2024-09-21T03:31:57.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.266+0000 7f3cfffff640 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 0x5581a3a63c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5581a3a6a2e0 tx=0x7f3cf0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6770a77a8dfd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.266+0000 7f3cfd7fa640 1 -- 172.21.15.83:0/3464850418 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0005020 con 0x5581a3684390 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.266+0000 7f3cfd7fa640 1 -- 172.21.15.83:0/3464850418 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00047a0 con 0x5581a3684390 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.266+0000 7f3cfd7fa640 1 -- 172.21.15.83:0/3464850418 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000f9f0 con 0x5581a3684390 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- 172.21.15.83:0/3464850418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 msgr2=0x5581a3a63c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 0x5581a3a63c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5581a3a6a2e0 tx=0x7f3cf0005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- 172.21.15.83:0/3464850418 shutdown_connections 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a36a5070 0x5581a3a641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a3684390 0x5581a3a63c90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 --2- 172.21.15.83:0/3464850418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a367c1b0 0x5581a3a57ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- 172.21.15.83:0/3464850418 >> 172.21.15.83:0/3464850418 conn(0x5581a3699460 msgr2=0x5581a3a577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- 172.21.15.83:0/3464850418 shutdown_connections 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- 172.21.15.83:0/3464850418 wait complete. 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 -- start start 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.267+0000 7f3d14537740 1 Processor -- start 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a36a5070 0x5581a3ad3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce800fb00 con 0x5581a3684390 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce800f430 con 0x5581a367c1b0 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce800c630 con 0x5581a36a5070 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d14537740 10 client.0 _ll_register_callbacks cb 0x7f3cf80e4bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52072/0 (socket says 172.21.15.83:52072) 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55886/0 (socket says 172.21.15.83:55886) 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3cfffff640 1 -- 172.21.15.83:0/298128748 learned_addr learned my addr 172.21.15.83:0/298128748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d07fff640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a36a5070 0x5581a3ad3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 -- 172.21.15.83:0/298128748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a36a5070 msgr2=0x5581a3ad3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a36a5070 0x5581a3ad3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 -- 172.21.15.83:0/298128748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 msgr2=0x5581a3a5c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 -- 172.21.15.83:0/298128748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a3a5cdb0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3cfffff640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3d077fe640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80aa160 tx=0x7f3cf80d42f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b227436f9a4fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.268+0000 7f3ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80af050 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf00047a0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f3cf80b2ab0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3cf80a8d90 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 10 client.27425 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5581a3ad4cd0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf00047a0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f3cf80d00b0 con 0x5581a367c1b0 2024-09-21T03:31:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 10 client.27425 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 10 client.27425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 20 client.27425 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a3638da0 0x5581a3ae84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 4 client.27425 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.269+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581a39c08a0 con 0x5581a3638da0 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.270+0000 7f3cfffff640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a3638da0 0x5581a3ae84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.270+0000 7f3cfffff640 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a3638da0 0x5581a3ae84a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d24129dc1f4515c0 server_cookie=245445199fb8ae6f in_seq=0 out_seq=0 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.270+0000 7f3ce6ffd640 10 client.27425 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3ce6ffd640 1 -- 172.21.15.83:0/298128748 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581a39c08a0 con 0x5581a3638da0 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3ce6ffd640 10 client.27425 handle_command_reply: tid=0 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 client.27425 shutdown 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 msgr2=0x5581a3a5bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80aa160 tx=0x7f3cf80d42f0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 shutdown_connections 2024-09-21T03:31:57.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x5581a3638da0 0x5581a3ae84a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581a36a5070 0x5581a3ad3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581a3684390 0x5581a3a5c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 --2- 172.21.15.83:0/298128748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581a367c1b0 0x5581a3a5bfd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3ce6ffd640 0 client.27425 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 >> 172.21.15.83:0/298128748 conn(0x5581a3699460 msgr2=0x5581a3a49970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 shutdown_connections 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.271+0000 7f3d14537740 1 -- 172.21.15.83:0/298128748 wait complete. 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d14537740 20 client.27425 trim_cache size 0 max 16384 2024-09-21T03:31:57.285 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 msgr2=0x7f3cf40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 0x7f3cf40b8c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d000315f0 tx=0x7f3d0003b000 comp rx=0 tx=0).stop 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 msgr2=0x7f3d0819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d950 tx=0x7f3cf800de10 comp rx=0 tx=0).stop 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 shutdown_connections 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f3cf40b6700 0x7f3cf40b8c10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d08109e00 0x7f3d0819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d081094b0 0x7f3d0819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 --2- 172.21.15.83:0/218456736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08103bc0 0x7f3d0819a4f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 >> 172.21.15.83:0/218456736 conn(0x7f3d0807b910 msgr2=0x7f3d08107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 shutdown_connections 2024-09-21T03:31:57.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:57.272+0000 7f3d0dfc3640 1 -- 172.21.15.83:0/218456736 wait complete. 2024-09-21T03:31:58.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:31:58.330 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:31:58 smithi083 ceph-mon[31219]: pgmap v802: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 779 op/s 2024-09-21T03:31:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/95082284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 msgr2=0x7f2a7410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/95082284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7410f530 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009ca0 tx=0x7f2a6c02a8b0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/95082284 shutdown_connections 2024-09-21T03:31:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/95082284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7410f530 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/95082284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a741087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/95082284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a74103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/95082284 >> 172.21.15.83:0/95082284 conn(0x7f2a74077c60 msgr2=0x7f2a740780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/95082284 shutdown_connections 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/95082284 wait complete. 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 Processor -- start 2024-09-21T03:31:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.911+0000 7f2a7b3e3640 1 -- start start 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a7419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74113ae0 con 0x7f2a74108d00 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74113960 con 0x7f2a741083b0 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7b3e3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74113c60 con 0x7f2a74103140 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a79980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7917f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a7419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55904/0 (socket says 172.21.15.83:55904) 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 -- 172.21.15.83:0/2273826645 learned_addr learned my addr 172.21.15.83:0/2273826645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 -- 172.21.15.83:0/2273826645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 msgr2=0x7f2a7419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a7419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 -- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 msgr2=0x7f2a7419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7897e640 1 -- 172.21.15.83:0/2273826645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a741a03a0 con 0x7f2a741083b0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.912+0000 7f2a7917f640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a7419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a7897e640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400e9f0 tx=0x7f2a6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a6a7fc640 1 -- 172.21.15.83:0/2273826645 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400ccc0 con 0x7f2a741083b0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a741a0690 con 0x7f2a741083b0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a6a7fc640 1 -- 172.21.15.83:0/2273826645 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a640108e0 con 0x7f2a741083b0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a6a7fc640 1 -- 172.21.15.83:0/2273826645 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64018860 con 0x7f2a741083b0 2024-09-21T03:31:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.913+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a741a6df0 con 0x7f2a741083b0 2024-09-21T03:31:58.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 Processor -- start 2024-09-21T03:31:58.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 -- start start 2024-09-21T03:31:58.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a6a7fc640 1 -- 172.21.15.83:0/2273826645 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2a64010430 con 0x7f2a741083b0 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 0x55b830a796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a79be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8309efea0 con 0x55b8306d2070 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b8309df2d0 con 0x55b8306b1390 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a81957740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8309e30e0 con 0x55b8306a91b0 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a79980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 0x55b830a796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:55914/0 (socket says 172.21.15.83:55914) 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.917+0000 7f2a7897e640 1 -- 172.21.15.83:0/1815583349 learned_addr learned my addr 172.21.15.83:0/1815583349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a7917f640 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a79be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 -- 172.21.15.83:0/1815583349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 msgr2=0x55b830a79be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a79be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 -- 172.21.15.83:0/1815583349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 msgr2=0x55b830a7a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 -- 172.21.15.83:0/1815583349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83073d650 con 0x55b8306d2070 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a79980640 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 0x55b830a796a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005e30 tx=0x7f2a6c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba0d278083412ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a5f7fe640 1 -- 172.21.15.83:0/1815583349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0305a0 con 0x55b8306d2070 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a5f7fe640 1 -- 172.21.15.83:0/1815583349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6c036030 con 0x55b8306d2070 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a5f7fe640 1 -- 172.21.15.83:0/1815583349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0042a0 con 0x55b8306d2070 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.918+0000 7f2a6a7fc640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 0x7f2a600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a7917f640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 0x7f2a600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a6a7fc640 1 -- 172.21.15.83:0/2273826645 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f2a640e7950 con 0x7f2a741083b0 2024-09-21T03:31:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 -- 172.21.15.83:0/1815583349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 msgr2=0x55b830a796a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a7917f640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 0x7f2a600b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55b830a78320 tx=0x7f2a70005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 0x55b830a796a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005e30 tx=0x7f2a6c009800 comp rx=0 tx=0).stop 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 -- 172.21.15.83:0/1815583349 shutdown_connections 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 --2- 172.21.15.83:0/1815583349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306d2070 0x55b830a796a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.919+0000 7f2a81957740 1 -- 172.21.15.83:0/1815583349 >> 172.21.15.83:0/1815583349 conn(0x55b8306c6460 msgr2=0x55b830a77590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.920+0000 7f2a81957740 1 -- 172.21.15.83:0/1815583349 shutdown_connections 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.920+0000 7f2a81957740 1 -- 172.21.15.83:0/1815583349 wait complete. 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.920+0000 7f2a81957740 1 -- start start 2024-09-21T03:31:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.922+0000 7f2a81957740 1 Processor -- start 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.922+0000 7f2a81957740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a89970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.922+0000 7f2a81957740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.922+0000 7f2a81957740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306d2070 0x55b830a8ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a81957740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b830840700 con 0x55b8306b1390 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a81957740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b83097a160 con 0x55b8306d2070 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a81957740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b830a87e20 con 0x55b8306a91b0 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a79980640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306d2070 0x55b830a8ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a81957740 10 client.0 _ll_register_callbacks cb 0x55b8309939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52102/0 (socket says 172.21.15.83:52102) 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 -- 172.21.15.83:0/1845655949 learned_addr learned my addr 172.21.15.83:0/1845655949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7917f640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a89970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 -- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 msgr2=0x55b830a89970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a89970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 -- 172.21.15.83:0/1845655949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306d2070 msgr2=0x55b830a8ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306d2070 0x55b830a8ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 -- 172.21.15.83:0/1845655949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8309a7c90 con 0x55b8306b1390 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7917f640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a89970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c6a0 tx=0x7f2a6400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4364b2040234cce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.923+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640a9ef0 con 0x55b8306b1390 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c036030 con 0x55b8306b1390 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f2a640a9400 con 0x55b8306b1390 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2a640d2070 con 0x55b8306b1390 2024-09-21T03:31:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 10 client.17904 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b830a97f10 con 0x55b8306b1390 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6c036030 con 0x55b8306b1390 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f2a640b3460 con 0x55b8306b1390 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 10 client.17904 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 10 client.17904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 20 client.17904 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306909f0 0x55b830a9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 4 client.17904 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.924+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8309f1df0 con 0x55b8306909f0 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.925+0000 7f2a7917f640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306909f0 0x55b830a9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.925+0000 7f2a5cff9640 10 client.17904 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.925+0000 7f2a7917f640 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306909f0 0x55b830a9b350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d98fb2c56c0c060 server_cookie=c74c7aa83ee2c97b in_seq=0 out_seq=0 2024-09-21T03:31:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a5cff9640 1 -- 172.21.15.83:0/1845655949 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8309f1df0 con 0x55b8306909f0 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a5cff9640 10 client.17904 handle_command_reply: tid=0 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 client.17904 shutdown 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 msgr2=0x55b830a89f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c6a0 tx=0x7f2a6400ce60 comp rx=0 tx=0).stop 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 shutdown_connections 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306909f0 0x55b830a9b350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306d2070 0x55b830a8ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a5cff9640 0 client.17904 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a89f50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 --2- 172.21.15.83:0/1845655949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a89970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 >> 172.21.15.83:0/1845655949 conn(0x55b8306c6460 msgr2=0x55b830a7c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 shutdown_connections 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 1 -- 172.21.15.83:0/1845655949 wait complete. 2024-09-21T03:31:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.926+0000 7f2a81957740 20 client.17904 trim_cache size 0 max 16384 2024-09-21T03:31:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 Processor -- start 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 -- start start 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 0x55b830a848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741ad360 con 0x55b8306909f0 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741aef00 con 0x55b8306b1390 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a81957740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741ada30 con 0x55b8306a91b0 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a7917f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 0x55b830a848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a7897e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a7917f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 0x55b830a848c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52116/0 (socket says 172.21.15.83:52116) 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a7917f640 1 -- 172.21.15.83:0/1061102105 learned_addr learned my addr 172.21.15.83:0/1061102105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.943+0000 7f2a79980640 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 -- 172.21.15.83:0/1061102105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 msgr2=0x55b830a7a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 -- 172.21.15.83:0/1061102105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 msgr2=0x55b830a848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 0x55b830a848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 -- 172.21.15.83:0/1061102105 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b830a7d7b0 con 0x55b8306a91b0 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a7897e640 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a8f5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b39a0 tx=0x7f2a6400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594916f6b5e26a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a5effd640 1 -- 172.21.15.83:0/1061102105 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640b2020 con 0x55b8306a91b0 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a5effd640 1 -- 172.21.15.83:0/1061102105 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a64010720 con 0x55b8306a91b0 2024-09-21T03:31:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.944+0000 7f2a5effd640 1 -- 172.21.15.83:0/1061102105 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640ad010 con 0x55b8306a91b0 2024-09-21T03:31:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- 172.21.15.83:0/1061102105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 msgr2=0x55b830a8f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a8f5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b39a0 tx=0x7f2a6400cef0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- 172.21.15.83:0/1061102105 shutdown_connections 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306b1390 0x55b830a7a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306a91b0 0x55b830a8f5f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 --2- 172.21.15.83:0/1061102105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306909f0 0x55b830a848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- 172.21.15.83:0/1061102105 >> 172.21.15.83:0/1061102105 conn(0x55b8306c6460 msgr2=0x55b830a8eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- 172.21.15.83:0/1061102105 shutdown_connections 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- 172.21.15.83:0/1061102105 wait complete. 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 -- start start 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.945+0000 7f2a81957740 1 Processor -- start 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306909f0 0x55b830a8ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 0x55b830a904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741ae830 con 0x55b8306b1390 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741ae160 con 0x55b8306a91b0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741ab360 con 0x55b8306909f0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a79980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a79980640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52130/0 (socket says 172.21.15.83:52130) 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a79980640 1 -- 172.21.15.83:0/4072668578 learned_addr learned my addr 172.21.15.83:0/4072668578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a7897e640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 0x55b830a904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a7917f640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306909f0 0x55b830a8ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.946+0000 7f2a81957740 10 client.0 _ll_register_callbacks cb 0x7f2a6400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 -- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306909f0 msgr2=0x55b830a8ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306909f0 0x55b830a8ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 -- 172.21.15.83:0/4072668578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 msgr2=0x55b830a90a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 -- 172.21.15.83:0/4072668578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b830a94ff0 con 0x55b8306a91b0 2024-09-21T03:31:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a79980640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a7897e640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 0x55b830a904e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55b830a7c080 tx=0x7f2a640b3670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580cc194fac094fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640ad1f0 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f2a640b6820 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b8309ed8a0 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.947+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2a640ad670 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.948+0000 7f2a81957740 10 client.27443 fetch_fsmap learned FSMap version 54 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.948+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b830b05ea0 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.948+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a640ad3b0 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.948+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f2a64018a00 con 0x55b8306a91b0 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 10 client.27443 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 10 client.27443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 20 client.27443 populate_metadata read hostname 'smithi083' 2024-09-21T03:31:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306d2070 0x55b830b08330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 4 client.27443 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8309ed8a0 con 0x55b8306d2070 2024-09-21T03:31:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a7917f640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306d2070 0x55b830b08330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.949+0000 7f2a5cff9640 10 client.27443 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a7917f640 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306d2070 0x55b830b08330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b43bc0caacf70ca server_cookie=fa8291183a014dae in_seq=0 out_seq=0 2024-09-21T03:31:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a5cff9640 1 -- 172.21.15.83:0/4072668578 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8309ed8a0 con 0x55b8306d2070 2024-09-21T03:31:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a5cff9640 10 client.27443 handle_command_reply: tid=0 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 client.27443 shutdown 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 msgr2=0x55b830a904e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 0x55b830a904e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55b830a7c080 tx=0x7f2a640b3670 comp rx=0 tx=0).stop 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 shutdown_connections 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55b8306d2070 0x55b830b08330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8306b1390 0x55b830a90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8306a91b0 0x55b830a904e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 --2- 172.21.15.83:0/4072668578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8306909f0 0x55b830a8ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 >> 172.21.15.83:0/4072668578 conn(0x55b8306c6460 msgr2=0x55b830a7c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a5cff9640 0 client.27443 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.950+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 shutdown_connections 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.951+0000 7f2a81957740 1 -- 172.21.15.83:0/4072668578 wait complete. 2024-09-21T03:31:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.951+0000 7f2a81957740 20 client.27443 trim_cache size 0 max 16384 2024-09-21T03:31:58.957 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:31:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.952+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 msgr2=0x7f2a600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.952+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 0x7f2a600b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55b830a78320 tx=0x7f2a70005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.952+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 msgr2=0x7f2a7419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.952+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400e9f0 tx=0x7f2a6400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 shutdown_connections 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f2a600b6680 0x7f2a600b8b90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a74108d00 0x7f2a7419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a741083b0 0x7f2a7419a9d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 --2- 172.21.15.83:0/2273826645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74103140 0x7f2a7419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 >> 172.21.15.83:0/2273826645 conn(0x7f2a74077c60 msgr2=0x7f2a741063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 shutdown_connections 2024-09-21T03:31:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:31:58.953+0000 7f2a7b3e3640 1 -- 172.21.15.83:0/2273826645 wait complete. 2024-09-21T03:31:59.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:31:58 smithi193 ceph-mon[35224]: pgmap v802: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 779 op/s 2024-09-21T03:31:59.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:31:58 smithi142 ceph-mon[34512]: pgmap v802: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 779 op/s 2024-09-21T03:31:59.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:32:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:00.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 -- 172.21.15.83:0/2693579332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 msgr2=0x7fb46c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 --2- 172.21.15.83:0/2693579332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c1087b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009900 tx=0x7fb45c02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:00.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 -- 172.21.15.83:0/2693579332 shutdown_connections 2024-09-21T03:32:00.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 --2- 172.21.15.83:0/2693579332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 0x7fb46c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 --2- 172.21.15.83:0/2693579332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c1087b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 --2- 172.21.15.83:0/2693579332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 -- 172.21.15.83:0/2693579332 >> 172.21.15.83:0/2693579332 conn(0x7fb46c079730 msgr2=0x7fb46c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 -- 172.21.15.83:0/2693579332 shutdown_connections 2024-09-21T03:32:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.670+0000 7fb473af1640 1 -- 172.21.15.83:0/2693579332 wait complete. 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 Processor -- start 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 -- start start 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 0x7fb46c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c112960 con 0x7fb46c1083a0 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1127e0 con 0x7fb46c108cf0 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb473af1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c112ae0 con 0x7fb46c102a60 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb47108c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb47188d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb47108c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41054/0 (socket says 172.21.15.83:41054) 2024-09-21T03:32:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb47108c640 1 -- 172.21.15.83:0/1079178685 learned_addr learned my addr 172.21.15.83:0/1079178685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.671+0000 7fb47208e640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 0x7fb46c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 -- 172.21.15.83:0/1079178685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 msgr2=0x7fb46c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 0x7fb46c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 -- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 msgr2=0x7fb46c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 -- 172.21.15.83:0/1079178685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a03d0 con 0x7fb46c102a60 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47108c640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb47188d640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c19a460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb468005fd0 tx=0x7fb46800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb462ffd640 1 -- 172.21.15.83:0/1079178685 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468009c30 con 0x7fb46c102a60 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb462ffd640 1 -- 172.21.15.83:0/1079178685 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb468009dd0 con 0x7fb46c102a60 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a06c0 con 0x7fb46c102a60 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb462ffd640 1 -- 172.21.15.83:0/1079178685 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468003e20 con 0x7fb46c102a60 2024-09-21T03:32:00.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.672+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a6e20 con 0x7fb46c102a60 2024-09-21T03:32:00.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 Processor -- start 2024-09-21T03:32:00.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 -- start start 2024-09-21T03:32:00.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb462ffd640 1 -- 172.21.15.83:0/1079178685 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb468003fc0 con 0x7fb46c102a60 2024-09-21T03:32:00.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x5625080482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562507fbeea0 con 0x562507c781b0 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562507fae2d0 con 0x562507ca1070 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47a065740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562507fb20e0 con 0x562507c80390 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47188d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.675+0000 7fb47208e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x5625080482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47108c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47108c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57146/0 (socket says 172.21.15.83:57146) 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47208e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x5625080482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45324/0 (socket says 172.21.15.83:45324) 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41062/0 (socket says 172.21.15.83:41062) 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47108c640 1 -- 172.21.15.83:0/684648332 learned_addr learned my addr 172.21.15.83:0/684648332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 -- 172.21.15.83:0/684648332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 msgr2=0x562508048d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 -- 172.21.15.83:0/684648332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 msgr2=0x5625080482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x5625080482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 -- 172.21.15.83:0/684648332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562507d0c650 con 0x562507c781b0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47108c640 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb462ffd640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 0x7fb4580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47188d640 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb4680e4a30 tx=0x7fb4680ef990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb334c613d03c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb47208e640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 0x7fb4580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.676+0000 7fb462ffd640 1 -- 172.21.15.83:0/1079178685 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fb4680e7290 con 0x7fb46c102a60 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb457fff640 1 -- 172.21.15.83:0/684648332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680a9830 con 0x562507c781b0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb457fff640 1 -- 172.21.15.83:0/684648332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680accf0 con 0x562507c781b0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb457fff640 1 -- 172.21.15.83:0/684648332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680ac950 con 0x562507c781b0 2024-09-21T03:32:00.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb47a065740 1 -- 172.21.15.83:0/684648332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 msgr2=0x562508048800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb47208e640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 0x7fb4580b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb464005fd0 tx=0x7fb464009500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.677+0000 7fb47a065740 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb4680e4a30 tx=0x7fb4680ef990 comp rx=0 tx=0).stop 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 -- 172.21.15.83:0/684648332 shutdown_connections 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c80390 0x562508048d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c781b0 0x562508048800 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 --2- 172.21.15.83:0/684648332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x5625080482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 -- 172.21.15.83:0/684648332 >> 172.21.15.83:0/684648332 conn(0x562507c95460 msgr2=0x5625080461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 -- 172.21.15.83:0/684648332 shutdown_connections 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 -- 172.21.15.83:0/684648332 wait complete. 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 -- start start 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.678+0000 7fb47a065740 1 Processor -- start 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c781b0 0x562508057df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507ca1070 0x562508064100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562507e0f700 con 0x562507c80390 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562507f49160 con 0x562507c781b0 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56250805add0 con 0x562507ca1070 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47a065740 10 client.0 _ll_register_callbacks cb 0x562507f629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41066/0 (socket says 172.21.15.83:41066) 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 -- 172.21.15.83:0/1531425281 learned_addr learned my addr 172.21.15.83:0/1531425281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47188d640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c781b0 0x562508057df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47208e640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507ca1070 0x562508064100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 -- 172.21.15.83:0/1531425281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507ca1070 msgr2=0x562508064100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507ca1070 0x562508064100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 -- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c781b0 msgr2=0x562508057df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c781b0 0x562508057df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.679+0000 7fb47108c640 1 -- 172.21.15.83:0/1531425281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562507f76c90 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47108c640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031440 tx=0x7fb45c031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df9adb76e117e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c035040 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4680accf0 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fb45c004250 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb45c004450 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 10 client.17922 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562508066bd0 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680accf0 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fb45c0035f0 con 0x562507c80390 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 10 client.17922 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.680+0000 7fb47a065740 20 client.17922 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:00.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c5f9f0 0x56250806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb47a065740 4 client.17922 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562507fc0df0 con 0x562507c5f9f0 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb47188d640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c5f9f0 0x56250806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb4557fa640 10 client.17922 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.681+0000 7fb47188d640 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c5f9f0 0x56250806a4a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554dc6e47543c9a6 server_cookie=3a2b8dc79a2512c8 in_seq=0 out_seq=0 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.682+0000 7fb4557fa640 1 -- 172.21.15.83:0/1531425281 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562507fc0df0 con 0x562507c5f9f0 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.682+0000 7fb4557fa640 10 client.17922 handle_command_reply: tid=0 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.682+0000 7fb47a065740 1 client.17922 shutdown 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 msgr2=0x562508058330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031440 tx=0x7fb45c031810 comp rx=0 tx=0).stop 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 shutdown_connections 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c5f9f0 0x56250806a4a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507ca1070 0x562508064100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x562508058330 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 --2- 172.21.15.83:0/1531425281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c781b0 0x562508057df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 >> 172.21.15.83:0/1531425281 conn(0x562507c95460 msgr2=0x562508044750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb4557fa640 0 client.17922 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 shutdown_connections 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 1 -- 172.21.15.83:0/1531425281 wait complete. 2024-09-21T03:32:00.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.683+0000 7fb47a065740 20 client.17922 trim_cache size 0 max 16384 2024-09-21T03:32:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 Processor -- start 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 -- start start 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c80390 0x562508064000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ad370 con 0x562507ca1070 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1aef10 con 0x562507c80390 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.697+0000 7fb47a065740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ada40 con 0x562507c781b0 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47208e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47208e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41080/0 (socket says 172.21.15.83:41080) 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57166/0 (socket says 172.21.15.83:57166) 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47208e640 1 -- 172.21.15.83:0/1290226070 learned_addr learned my addr 172.21.15.83:0/1290226070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47108c640 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c80390 0x562508064000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 -- 172.21.15.83:0/1290226070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c80390 msgr2=0x562508064000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c80390 0x562508064000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 -- 172.21.15.83:0/1290226070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 msgr2=0x562508054fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 -- 172.21.15.83:0/1290226070 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562508058e90 con 0x562507c781b0 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47208e640 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb47188d640 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb4680eae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923842050e0e05f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb4577fe640 1 -- 172.21.15.83:0/1290226070 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468020040 con 0x562507c781b0 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb4577fe640 1 -- 172.21.15.83:0/1290226070 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680e6040 con 0x562507c781b0 2024-09-21T03:32:00.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.698+0000 7fb4577fe640 1 -- 172.21.15.83:0/1290226070 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680acb30 con 0x562507c781b0 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 -- 172.21.15.83:0/1290226070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 msgr2=0x562508046200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb4680eae90 comp rx=0 tx=0).stop 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 -- 172.21.15.83:0/1290226070 shutdown_connections 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507ca1070 0x562508054fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507c80390 0x562508064000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 --2- 172.21.15.83:0/1290226070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508046200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 -- 172.21.15.83:0/1290226070 >> 172.21.15.83:0/1290226070 conn(0x562507c95460 msgr2=0x562508045af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 -- 172.21.15.83:0/1290226070 shutdown_connections 2024-09-21T03:32:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.699+0000 7fb47a065740 1 -- 172.21.15.83:0/1290226070 wait complete. 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.700+0000 7fb47a065740 1 -- start start 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.700+0000 7fb47a065740 1 Processor -- start 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x56250805be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x56250805c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ae840 con 0x562507c80390 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ae170 con 0x562507ca1070 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ab370 con 0x562507c781b0 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47108c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x56250805be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57172/0 (socket says 172.21.15.83:57172) 2024-09-21T03:32:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 -- 172.21.15.83:0/1620183310 learned_addr learned my addr 172.21.15.83:0/1620183310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47a065740 10 client.0 _ll_register_callbacks cb 0x7fb45c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 -- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 msgr2=0x56250805c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x56250805c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 -- 172.21.15.83:0/1620183310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 msgr2=0x56250805be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x56250805be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47188d640 1 -- 172.21.15.83:0/1620183310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250805cba0 con 0x562507c781b0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.701+0000 7fb47108c640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x56250805be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47188d640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb468005d80 tx=0x7fb4680acd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e01c15a9274dc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468020070 con 0x562507c781b0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7fb4680eabb0 con 0x562507c781b0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4680e6040 con 0x562507c781b0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb4680acfb0 con 0x562507c781b0 2024-09-21T03:32:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 10 client.27418 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5625080d4e90 con 0x562507c781b0 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680e6040 con 0x562507c781b0 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7fb4680e4cb0 con 0x562507c781b0 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 10 client.27418 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 10 client.27418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.702+0000 7fb47a065740 20 client.27418 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c34da0 0x5625080d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47a065740 4 client.27418 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625080539a0 con 0x562507c34da0 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47108c640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c34da0 0x5625080d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb4557fa640 10 client.27418 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47108c640 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c34da0 0x5625080d7170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73cb3442f47d2363 server_cookie=bb0eb0d596bf6ced in_seq=0 out_seq=0 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb4557fa640 1 -- 172.21.15.83:0/1620183310 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625080539a0 con 0x562507c34da0 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb4557fa640 10 client.27418 handle_command_reply: tid=0 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.703+0000 7fb47a065740 1 client.27418 shutdown 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 msgr2=0x562508060910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb468005d80 tx=0x7fb4680acd80 comp rx=0 tx=0).stop 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 shutdown_connections 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x562507c34da0 0x5625080d7170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562507ca1070 0x56250805c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb4557fa640 0 client.27418 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562507c80390 0x56250805be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 --2- 172.21.15.83:0/1620183310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562507c781b0 0x562508060910 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 >> 172.21.15.83:0/1620183310 conn(0x562507c95460 msgr2=0x56250804d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 shutdown_connections 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 1 -- 172.21.15.83:0/1620183310 wait complete. 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.704+0000 7fb47a065740 20 client.27418 trim_cache size 0 max 16384 2024-09-21T03:32:00.712 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 msgr2=0x7fb4580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 0x7fb4580b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb464005fd0 tx=0x7fb464009500 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 msgr2=0x7fb46c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c19a460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb468005fd0 tx=0x7fb46800dde0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 shutdown_connections 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb4580b6680 0x7fb4580b8b90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb46c108cf0 0x7fb46c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46c1083a0 0x7fb46c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 --2- 172.21.15.83:0/1079178685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c102a60 0x7fb46c19a460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 >> 172.21.15.83:0/1079178685 conn(0x7fb46c079730 msgr2=0x7fb46c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 shutdown_connections 2024-09-21T03:32:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:00.705+0000 7fb473af1640 1 -- 172.21.15.83:0/1079178685 wait complete. 2024-09-21T03:32:00.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:00 smithi083 ceph-mon[31219]: pgmap v803: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-09-21T03:32:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 880.0 s, 1492.2 tps, lat 67.155 ms stddev 71.410 2024-09-21T03:32:01.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:00 smithi193 ceph-mon[35224]: pgmap v803: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-09-21T03:32:01.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:00 smithi142 ceph-mon[34512]: pgmap v803: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-09-21T03:32:02.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:32:02.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:32:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:03.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:02 smithi193 ceph-mon[35224]: pgmap v804: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-09-21T03:32:03.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:02 smithi083 ceph-mon[31219]: pgmap v804: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-09-21T03:32:03.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:02 smithi142 ceph-mon[34512]: pgmap v804: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-09-21T03:32:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.670+0000 7f5818308640 1 -- 172.21.15.83:0/4116675470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5810111f40 msgr2=0x7f581010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.670+0000 7f5818308640 1 --2- 172.21.15.83:0/4116675470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5810111f40 0x7f581010a3c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5808009be0 tx=0x7f580802a710 comp rx=0 tx=0).stop 2024-09-21T03:32:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.671+0000 7f58158a3640 1 --2- 172.21.15.83:0/4116675470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581010a9e0 0x7f581010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:03.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.673+0000 7f5818308640 1 -- 172.21.15.83:0/4116675470 shutdown_connections 2024-09-21T03:32:03.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.673+0000 7f5818308640 1 --2- 172.21.15.83:0/4116675470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f581011cc30 0x7f581011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.673+0000 7f5818308640 1 --2- 172.21.15.83:0/4116675470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581010a9e0 0x7f581010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.673+0000 7f5818308640 1 --2- 172.21.15.83:0/4116675470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5810111f40 0x7f581010a3c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.673+0000 7f5818308640 1 -- 172.21.15.83:0/4116675470 >> 172.21.15.83:0/4116675470 conn(0x7f5810070ec0 msgr2=0x7f58100712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.677+0000 7f5818308640 1 -- 172.21.15.83:0/4116675470 shutdown_connections 2024-09-21T03:32:03.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.677+0000 7f5818308640 1 -- 172.21.15.83:0/4116675470 wait complete. 2024-09-21T03:32:03.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 Processor -- start 2024-09-21T03:32:03.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 -- start start 2024-09-21T03:32:03.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f581010a9e0 0x7f58101a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581011cc30 0x7f58101aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5810122910 con 0x7f581011cc30 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5810122790 con 0x7f581010a9e0 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.678+0000 7f5818308640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5810122a90 con 0x7f5810111f40 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.679+0000 7f58158a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.679+0000 7f58158a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57194/0 (socket says 172.21.15.83:57194) 2024-09-21T03:32:03.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.679+0000 7f58158a3640 1 -- 172.21.15.83:0/1272389625 learned_addr learned my addr 172.21.15.83:0/1272389625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:03.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.680+0000 7f58158a3640 1 -- 172.21.15.83:0/1272389625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f581010a9e0 msgr2=0x7f58101a9360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:32:03.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.680+0000 7f58158a3640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f581010a9e0 0x7f58101a9360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.680+0000 7f58158a3640 1 -- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581011cc30 msgr2=0x7f58101aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.680+0000 7f58158a3640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581011cc30 0x7f58101aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.680+0000 7f58158a3640 1 -- 172.21.15.83:0/1272389625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58101af2d0 con 0x7f5810111f40 2024-09-21T03:32:03.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.681+0000 7f58158a3640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f580c0058c0 tx=0x7f580c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.682+0000 7f58077fe640 1 -- 172.21.15.83:0/1272389625 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c017020 con 0x7f5810111f40 2024-09-21T03:32:03.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.683+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101af5c0 con 0x7f5810111f40 2024-09-21T03:32:03.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.683+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101b5d20 con 0x7f5810111f40 2024-09-21T03:32:03.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.683+0000 7f58077fe640 1 -- 172.21.15.83:0/1272389625 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c009c30 con 0x7f5810111f40 2024-09-21T03:32:03.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.683+0000 7f58077fe640 1 -- 172.21.15.83:0/1272389625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c00f620 con 0x7f5810111f40 2024-09-21T03:32:03.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.687+0000 7f581e87c740 1 Processor -- start 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 -- start start 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7132430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d6a670 0x55f5d7132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5d70a8e60 con 0x55f5d6d8b350 2024-09-21T03:32:03.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f5d7098290 con 0x55f5d6d62490 2024-09-21T03:32:03.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.688+0000 7f581e87c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f5d709c0a0 con 0x55f5d6d6a670 2024-09-21T03:32:03.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.691+0000 7f58077fe640 1 -- 172.21.15.83:0/1272389625 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f580c01a020 con 0x7f5810111f40 2024-09-21T03:32:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.691+0000 7f58160a4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.691+0000 7f58160a4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45388/0 (socket says 172.21.15.83:45388) 2024-09-21T03:32:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.691+0000 7f58160a4640 1 -- 172.21.15.83:0/857565564 learned_addr learned my addr 172.21.15.83:0/857565564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.692+0000 7f58077fe640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 0x7f57fc0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.692+0000 7f58168a5640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 0x7f57fc0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.692+0000 7f58077fe640 1 -- 172.21.15.83:0/1272389625 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8441+0+0 (secure 0 0 0) 0x7f580c0a9560 con 0x7f5810111f40 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.693+0000 7f58160a4640 1 -- 172.21.15.83:0/857565564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d6a670 msgr2=0x55f5d7132eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.693+0000 7f58160a4640 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d6a670 0x55f5d7132eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.693+0000 7f58160a4640 1 -- 172.21.15.83:0/857565564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 msgr2=0x55f5d7132430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.693+0000 7f58160a4640 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7132430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.693+0000 7f58160a4640 1 -- 172.21.15.83:0/857565564 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d6df6930 con 0x55f5d6d62490 2024-09-21T03:32:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.694+0000 7f58160a4640 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5808009d00 tx=0x7f5808030410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fbf40fb4b8a3fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.700+0000 7f5804d9a640 1 -- 172.21.15.83:0/857565564 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808040020 con 0x55f5d6d62490 2024-09-21T03:32:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.700+0000 7f5804d9a640 1 -- 172.21.15.83:0/857565564 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5808003e20 con 0x55f5d6d62490 2024-09-21T03:32:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.701+0000 7f581e87c740 1 -- 172.21.15.83:0/857565564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 msgr2=0x55f5d7132970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.701+0000 7f581e87c740 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5808009d00 tx=0x7f5808030410 comp rx=0 tx=0).stop 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.702+0000 7f58168a5640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 0x7f57fc0b8ce0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5800009460 tx=0x7f58000093d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.703+0000 7f581e87c740 1 -- 172.21.15.83:0/857565564 shutdown_connections 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.703+0000 7f581e87c740 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d6a670 0x55f5d7132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.703+0000 7f581e87c740 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7132970 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.703+0000 7f581e87c740 1 --2- 172.21.15.83:0/857565564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7132430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.703+0000 7f581e87c740 1 -- 172.21.15.83:0/857565564 >> 172.21.15.83:0/857565564 conn(0x55f5d6d7f740 msgr2=0x55f5d7130320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.706+0000 7f581e87c740 1 -- 172.21.15.83:0/857565564 shutdown_connections 2024-09-21T03:32:03.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.706+0000 7f581e87c740 1 -- 172.21.15.83:0/857565564 wait complete. 2024-09-21T03:32:03.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.706+0000 7f581e87c740 1 -- start start 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 Processor -- start 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 0x55f5d713dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d713e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5d6ef9890 con 0x55f5d6d8b350 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f5d7032d40 con 0x55f5d6d6a670 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f5d7140c00 con 0x55f5d6d62490 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f581e87c740 10 client.0 _ll_register_callbacks cb 0x55f5d7096c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f58168a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.707+0000 7f58168a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41140/0 (socket says 172.21.15.83:41140) 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58168a5640 1 -- 172.21.15.83:0/3375090935 learned_addr learned my addr 172.21.15.83:0/3375090935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58158a3640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d713e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 0x55f5d713dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 -- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 msgr2=0x55f5d713e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d713e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 -- 172.21.15.83:0/3375090935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 msgr2=0x55f5d7149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.708+0000 7f58160a4640 1 -- 172.21.15.83:0/3375090935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d714a510 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.709+0000 7f58160a4640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 0x55f5d713dfb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5808009be0 tx=0x7f58080092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59306bd28f2d446e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.709+0000 7f57f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.710+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580804e060 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.710+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8485+0+0 (secure 0 0 0) 0x7f58080047c0 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.710+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5808003e20 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.711+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5808040070 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.711+0000 7f581e87c740 10 client.27427 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.711+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f5d714cda0 con 0x55f5d6d62490 2024-09-21T03:32:03.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.711+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5808003e20 con 0x55f5d6d62490 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f580803c7e0 con 0x55f5d6d62490 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 10 client.27427 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 20 client.27427 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d49cd0 0x55f5d71502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 4 client.27427 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:03.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5d70aadb0 con 0x55f5d6d49cd0 2024-09-21T03:32:03.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.713+0000 7f58158a3640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d49cd0 0x55f5d71502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.717+0000 7f58158a3640 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d49cd0 0x55f5d71502a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963b78c4dc001d97 server_cookie=a3441a700b39a909 in_seq=0 out_seq=0 2024-09-21T03:32:03.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.717+0000 7f57f9ffb640 10 client.27427 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:03.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.718+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3375090935 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5d70aadb0 con 0x55f5d6d49cd0 2024-09-21T03:32:03.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.718+0000 7f57f9ffb640 10 client.27427 handle_command_reply: tid=0 2024-09-21T03:32:03.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.718+0000 7f581e87c740 1 client.27427 shutdown 2024-09-21T03:32:03.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.720+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 msgr2=0x55f5d713dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.720+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 0x55f5d713dfb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5808009be0 tx=0x7f58080092f0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 shutdown_connections 2024-09-21T03:32:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d49cd0 0x55f5d71502a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d8b350 0x55f5d7149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d713e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 --2- 172.21.15.83:0/3375090935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d62490 0x55f5d713dfb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.722+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 >> 172.21.15.83:0/3375090935 conn(0x55f5d6d7f740 msgr2=0x55f5d7135610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.723+0000 7f57f9ffb640 0 client.27427 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.723+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 shutdown_connections 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.723+0000 7f581e87c740 1 -- 172.21.15.83:0/3375090935 wait complete. 2024-09-21T03:32:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.723+0000 7f581e87c740 20 client.27427 trim_cache size 0 max 16384 2024-09-21T03:32:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.757+0000 7f581e87c740 1 Processor -- start 2024-09-21T03:32:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 -- start start 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7130210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7138550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400e6e0 con 0x55f5d6d6a670 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4010280 con 0x55f5d6d62490 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.758+0000 7f581e87c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400edb0 con 0x55f5d6d8b350 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.760+0000 7f58158a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.760+0000 7f58158a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41144/0 (socket says 172.21.15.83:41144) 2024-09-21T03:32:03.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.760+0000 7f58158a3640 1 -- 172.21.15.83:0/2561233525 learned_addr learned my addr 172.21.15.83:0/2561233525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:03.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.760+0000 7f58168a5640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.760+0000 7f58160a4640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7130210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.764+0000 7f58168a5640 1 -- 172.21.15.83:0/2561233525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 msgr2=0x55f5d7130210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.764+0000 7f58168a5640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7130210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.764+0000 7f58168a5640 1 -- 172.21.15.83:0/2561233525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 msgr2=0x55f5d7149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.764+0000 7f58168a5640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.764+0000 7f58168a5640 1 -- 172.21.15.83:0/2561233525 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d713f070 con 0x55f5d6d8b350 2024-09-21T03:32:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.765+0000 7f58158a3640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.768+0000 7f58168a5640 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7138550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5800009810 tx=0x7f5800013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ebc29dc8bdd127 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.771+0000 7f57fbfff640 1 -- 172.21.15.83:0/2561233525 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800003d40 con 0x55f5d6d8b350 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.771+0000 7f57fbfff640 1 -- 172.21.15.83:0/2561233525 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58000044d0 con 0x55f5d6d8b350 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.771+0000 7f57fbfff640 1 -- 172.21.15.83:0/2561233525 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800011440 con 0x55f5d6d8b350 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.772+0000 7f581e87c740 1 -- 172.21.15.83:0/2561233525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 msgr2=0x55f5d7138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.772+0000 7f581e87c740 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7138550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5800009810 tx=0x7f5800013000 comp rx=0 tx=0).stop 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.773+0000 7f581e87c740 1 -- 172.21.15.83:0/2561233525 shutdown_connections 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.773+0000 7f581e87c740 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7138550 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.773+0000 7f581e87c740 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d6a670 0x55f5d7149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.773+0000 7f581e87c740 1 --2- 172.21.15.83:0/2561233525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d62490 0x55f5d7130210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.773+0000 7f581e87c740 1 -- 172.21.15.83:0/2561233525 >> 172.21.15.83:0/2561233525 conn(0x55f5d6d7f740 msgr2=0x55f5d713c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.775+0000 7f581e87c740 1 -- 172.21.15.83:0/2561233525 shutdown_connections 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.776+0000 7f581e87c740 1 -- 172.21.15.83:0/2561233525 wait complete. 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.776+0000 7f581e87c740 1 -- start start 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.776+0000 7f581e87c740 1 Processor -- start 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 0x55f5d7141f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d71424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7142a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400fbb0 con 0x55f5d6d62490 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400f4e0 con 0x55f5d6d6a670 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57f400c770 con 0x55f5d6d8b350 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f581e87c740 10 client.0 _ll_register_callbacks cb 0x7f5800005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f58160a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 0x55f5d7141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f58160a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 0x55f5d7141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41156/0 (socket says 172.21.15.83:41156) 2024-09-21T03:32:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.777+0000 7f58160a4640 1 -- 172.21.15.83:0/3355845094 learned_addr learned my addr 172.21.15.83:0/3355845094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.778+0000 7f58168a5640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.778+0000 7f58158a3640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d71424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 -- 172.21.15.83:0/3355845094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 msgr2=0x55f5d7142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 -- 172.21.15.83:0/3355845094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 msgr2=0x55f5d7141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 0x55f5d7141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 -- 172.21.15.83:0/3355845094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d71b9d20 con 0x55f5d6d6a670 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58158a3640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d71424d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f580c0aa190 tx=0x7f580c0b6810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b123a3d272fe7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f58168a5640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f57f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.782+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0a9ae0 con 0x55f5d6d6a670 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.783+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58000044d0 con 0x55f5d6d6a670 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.783+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8485+0+0 (secure 0 0 0) 0x7f580c0ac760 con 0x55f5d6d6a670 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.783+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f580c0a9c80 con 0x55f5d6d6a670 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.784+0000 7f581e87c740 10 client.27473 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.784+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f5d71dd040 con 0x55f5d6d6a670 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.786+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c014040 con 0x55f5d6d6a670 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.786+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f580c0b2f60 con 0x55f5d6d6a670 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 10 client.27473 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 20 client.27473 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d1f080 0x55f5d71df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 4 client.27473 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:32:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.787+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f58000044d0 con 0x55f5d6d1f080 2024-09-21T03:32:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.788+0000 7f58160a4640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d1f080 0x55f5d71df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.791+0000 7f57f9ffb640 10 client.27473 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.791+0000 7f58160a4640 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d1f080 0x55f5d71df760 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c229914b6560b854 server_cookie=4ac9b7ff577841ab in_seq=0 out_seq=0 2024-09-21T03:32:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.796+0000 7f57f9ffb640 1 -- 172.21.15.83:0/3355845094 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f58000044d0 con 0x55f5d6d1f080 2024-09-21T03:32:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.796+0000 7f57f9ffb640 10 client.27473 handle_command_reply: tid=0 2024-09-21T03:32:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.798+0000 7f581e87c740 1 client.27473 shutdown 2024-09-21T03:32:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.799+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 msgr2=0x55f5d71424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.799+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d71424d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f580c0aa190 tx=0x7f580c0b6810 comp rx=0 tx=0).stop 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 shutdown_connections 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55f5d6d1f080 0x55f5d71df760 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5d6d8b350 0x55f5d7142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f5d6d6a670 0x55f5d71424d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 --2- 172.21.15.83:0/3355845094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5d6d62490 0x55f5d7141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f57f9ffb640 0 client.27473 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 >> 172.21.15.83:0/3355845094 conn(0x55f5d6d7f740 msgr2=0x55f5d713f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 shutdown_connections 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 1 -- 172.21.15.83:0/3355845094 wait complete. 2024-09-21T03:32:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.803+0000 7f581e87c740 20 client.27473 trim_cache size 0 max 16384 2024-09-21T03:32:03.806 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-09-21T03:32:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28", 2024-09-21T03:32:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-09-21T03:32:03.806 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:03.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 msgr2=0x7f57fc0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 0x7f57fc0b8ce0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5800009460 tx=0x7f58000093d0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 msgr2=0x7f58101a9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f580c0058c0 tx=0x7f580c00dde0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 shutdown_connections 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f57fc0b67d0 0x7f57fc0b8ce0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f581011cc30 0x7f58101aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5810111f40 0x7f58101a9900 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 --2- 172.21.15.83:0/1272389625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f581010a9e0 0x7f58101a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 >> 172.21.15.83:0/1272389625 conn(0x7f5810070ec0 msgr2=0x7f581011d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 shutdown_connections 2024-09-21T03:32:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:03.807+0000 7f5818308640 1 -- 172.21.15.83:0/1272389625 wait complete. 2024-09-21T03:32:04.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: pgmap v805: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub queued for path: / 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: active paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:04.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:04 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: pgmap v805: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-21T03:32:05.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:32:05.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub queued for path: / 2024-09-21T03:32:05.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: active paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:04 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.089 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: pgmap v805: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub queued for path: / 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: active paths [/] 2024-09-21T03:32:05.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:04 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:05.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 885.0 s, 427.2 tps, lat 234.591 ms stddev 416.263 2024-09-21T03:32:06.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:05 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:06.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:05 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:06.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:05 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 -- 172.21.15.83:0/2687236313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a10095740 msgr2=0x7f8a10095b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/2687236313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a10095740 0x7f8a10095b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009860 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-21T03:32:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 -- 172.21.15.83:0/2687236313 shutdown_connections 2024-09-21T03:32:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/2687236313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009b2c0 0x7f8a100a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/2687236313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009a970 0x7f8a1009ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/2687236313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a10095740 0x7f8a10095b50 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009860 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-21T03:32:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.335+0000 7f8a1f41c640 1 -- 172.21.15.83:0/2687236313 >> 172.21.15.83:0/2687236313 conn(0x7f8a1000c500 msgr2=0x7f8a1000c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.337+0000 7f8a1f41c640 1 -- 172.21.15.83:0/2687236313 shutdown_connections 2024-09-21T03:32:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.337+0000 7f8a1f41c640 1 -- 172.21.15.83:0/2687236313 wait complete. 2024-09-21T03:32:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 Processor -- start 2024-09-21T03:32:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 -- start start 2024-09-21T03:32:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1009a970 0x7f8a1009d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009dc80 0x7f8a10136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10095050 con 0x7f8a1009b2c0 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10094ed0 con 0x7f8a1009dc80 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.338+0000 7f8a1f41c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100951d0 con 0x7f8a1009a970 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1ec1b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009dc80 0x7f8a10136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41176/0 (socket says 172.21.15.83:41176) 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 -- 172.21.15.83:0/3055871177 learned_addr learned my addr 172.21.15.83:0/3055871177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1009a970 0x7f8a1009d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 -- 172.21.15.83:0/3055871177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1009a970 msgr2=0x7f8a1009d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1009a970 0x7f8a1009d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 -- 172.21.15.83:0/3055871177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009dc80 msgr2=0x7f8a10136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009dc80 0x7f8a10136640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 -- 172.21.15.83:0/3055871177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a10136e60 con 0x7f8a1009b2c0 2024-09-21T03:32:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.339+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800e9c0 tx=0x7f8a0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.341+0000 7f8a0f7fe640 1 -- 172.21.15.83:0/3055871177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800ccc0 con 0x7f8a1009b2c0 2024-09-21T03:32:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.341+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a10137150 con 0x7f8a1009b2c0 2024-09-21T03:32:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.341+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101376b0 con 0x7f8a1009b2c0 2024-09-21T03:32:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.341+0000 7f8a0f7fe640 1 -- 172.21.15.83:0/3055871177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080108e0 con 0x7f8a1009b2c0 2024-09-21T03:32:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.342+0000 7f8a0f7fe640 1 -- 172.21.15.83:0/3055871177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08026060 con 0x7f8a1009b2c0 2024-09-21T03:32:06.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.345+0000 7f8a26bf2740 1 Processor -- start 2024-09-21T03:32:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.345+0000 7f8a26bf2740 1 -- start start 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.345+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906df6070 0x55990719d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.345+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dd5390 0x55990719de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559907113e50 con 0x559906dcd1b0 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559907103280 con 0x559906df6070 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559907107090 con 0x559906dd5390 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41184/0 (socket says 172.21.15.83:41184) 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 -- 172.21.15.83:0/3054071112 learned_addr learned my addr 172.21.15.83:0/3054071112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906df6070 0x55990719d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 -- 172.21.15.83:0/3054071112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dd5390 msgr2=0x55990719de90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dd5390 0x55990719de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 -- 172.21.15.83:0/3054071112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906df6070 msgr2=0x55990719d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906df6070 0x55990719d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 -- 172.21.15.83:0/3054071112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559906e61650 con 0x559906dcd1b0 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.346+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009830 tx=0x7f8a14009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f3af2d53165eee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.347+0000 7f8a0cff9640 1 -- 172.21.15.83:0/3054071112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14063030 con 0x559906dcd1b0 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.347+0000 7f8a0cff9640 1 -- 172.21.15.83:0/3054071112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a14038040 con 0x559906dcd1b0 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.347+0000 7f8a0cff9640 1 -- 172.21.15.83:0/3054071112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14037070 con 0x559906dcd1b0 2024-09-21T03:32:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.347+0000 7f8a0f7fe640 1 -- 172.21.15.83:0/3055871177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8a08018680 con 0x7f8a1009b2c0 2024-09-21T03:32:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- 172.21.15.83:0/3054071112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 msgr2=0x55990719d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009830 tx=0x7f8a14009510 comp rx=0 tx=0).stop 2024-09-21T03:32:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- 172.21.15.83:0/3054071112 shutdown_connections 2024-09-21T03:32:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dd5390 0x55990719de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x55990719d950 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/3054071112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906df6070 0x55990719d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- 172.21.15.83:0/3054071112 >> 172.21.15.83:0/3054071112 conn(0x559906dea460 msgr2=0x55990719b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- 172.21.15.83:0/3054071112 shutdown_connections 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- 172.21.15.83:0/3054071112 wait complete. 2024-09-21T03:32:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a0f7fe640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 0x7f8a040b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a0f7fe640 1 -- 172.21.15.83:0/3055871177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8485+0+0 (secure 0 0 0) 0x7f8a080e7940 con 0x7f8a1009b2c0 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a26bf2740 1 -- start start 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.348+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 0x7f8a040b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 Processor -- start 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x5599071a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x5599071a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559906f64700 con 0x559906dcd1b0 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55990709e110 con 0x559906dd5390 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599071abb90 con 0x559906df6070 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a26bf2740 10 client.0 _ll_register_callbacks cb 0x5599070b7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a1ec1b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a1ec1b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:57238/0 (socket says 172.21.15.83:57238) 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/2218887886 learned_addr learned my addr 172.21.15.83:0/2218887886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.349+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x5599071a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 msgr2=0x5599071a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x5599071a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/2218887886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 msgr2=0x5599071a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x5599071a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/2218887886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599070cbc40 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a18051d10 tx=0x7f8a1806bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc028b5b460f3b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 0x7f8a040b8b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005940 tx=0x7f8a1402a340 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18080960 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a14038040 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8485+0+0 (secure 0 0 0) 0x7f8a1807f5d0 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8a18080b00 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a26bf2740 10 client.27439 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.350+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5599071b7830 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a14038040 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f8a18087070 con 0x559906df6070 2024-09-21T03:32:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 10 client.27439 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 10 client.27439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 20 client.27439 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906db49f0 0x5599071bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 4 client.27439 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559907115da0 con 0x559906db49f0 2024-09-21T03:32:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.351+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906db49f0 0x5599071bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.352+0000 7f8a01ffb640 10 client.27439 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.352+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906db49f0 0x5599071bb190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e446b6182ba2a73a server_cookie=7e3316a61f5d8953 in_seq=0 out_seq=0 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a01ffb640 1 -- 172.21.15.83:0/2218887886 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559907115da0 con 0x559906db49f0 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a01ffb640 10 client.27439 handle_command_reply: tid=0 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a26bf2740 1 client.27439 shutdown 2024-09-21T03:32:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 msgr2=0x5599071a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a18051d10 tx=0x7f8a1806bca0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 shutdown_connections 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.353+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906db49f0 0x5599071bb190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071a9470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x5599071a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/2218887886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dcd1b0 0x5599071a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 >> 172.21.15.83:0/2218887886 conn(0x559906dea460 msgr2=0x5599071a05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a01ffb640 0 client.27439 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 shutdown_connections 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 1 -- 172.21.15.83:0/2218887886 wait complete. 2024-09-21T03:32:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.354+0000 7f8a26bf2740 20 client.27439 trim_cache size 0 max 16384 2024-09-21T03:32:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 Processor -- start 2024-09-21T03:32:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 -- start start 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dcd1b0 0x55990719eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x55990719a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1013ff50 con 0x559906df6070 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.378+0000 7f8a1e41a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dcd1b0 0x55990719eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10141af0 con 0x559906dd5390 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a1dc19640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x55990719a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a1ec1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a1ec1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41214/0 (socket says 172.21.15.83:41214) 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a1dc19640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x55990719a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45464/0 (socket says 172.21.15.83:45464) 2024-09-21T03:32:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/1035203569 learned_addr learned my addr 172.21.15.83:0/1035203569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.379+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10140620 con 0x559906dcd1b0 2024-09-21T03:32:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.380+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/1035203569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dcd1b0 msgr2=0x55990719eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.380+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dcd1b0 0x55990719eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.380+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/1035203569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 msgr2=0x55990719a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.380+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x55990719a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.380+0000 7f8a1ec1b640 1 -- 172.21.15.83:0/1035203569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599071a7950 con 0x559906df6070 2024-09-21T03:32:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.381+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8a18052910 tx=0x7f8a18051ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b07bfb44757250e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.381+0000 7f8a03fff640 1 -- 172.21.15.83:0/1035203569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1808d070 con 0x559906df6070 2024-09-21T03:32:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.381+0000 7f8a03fff640 1 -- 172.21.15.83:0/1035203569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1806b3b0 con 0x559906df6070 2024-09-21T03:32:06.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.382+0000 7f8a03fff640 1 -- 172.21.15.83:0/1035203569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1807fa40 con 0x559906df6070 2024-09-21T03:32:06.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 msgr2=0x55990719ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8a18052910 tx=0x7f8a18051ed0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 shutdown_connections 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906df6070 0x55990719ae70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dd5390 0x55990719a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/1035203569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906dcd1b0 0x55990719eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 >> 172.21.15.83:0/1035203569 conn(0x559906dea460 msgr2=0x55990719e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 shutdown_connections 2024-09-21T03:32:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.383+0000 7f8a26bf2740 1 -- 172.21.15.83:0/1035203569 wait complete. 2024-09-21T03:32:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.384+0000 7f8a26bf2740 1 -- start start 2024-09-21T03:32:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.384+0000 7f8a26bf2740 1 Processor -- start 2024-09-21T03:32:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 0x5599071b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 0x5599071aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10141420 con 0x559906dd5390 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10140d50 con 0x559906dcd1b0 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1013df50 con 0x559906df6070 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a26bf2740 10 client.0 _ll_register_callbacks cb 0x7f8a18069a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a1e41a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 0x5599071b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a1e41a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 0x5599071b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.83:45468/0 (socket says 172.21.15.83:45468) 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a1e41a640 1 -- 172.21.15.83:0/899246927 learned_addr learned my addr 172.21.15.83:0/899246927 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a1ec1b640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.385+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 0x5599071aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 -- 172.21.15.83:0/899246927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 msgr2=0x5599071acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 -- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 msgr2=0x5599071b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 0x5599071b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 -- 172.21.15.83:0/899246927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599071ad700 con 0x559906dd5390 2024-09-21T03:32:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a1dc19640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 0x5599071aca80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8a080e50b0 tx=0x7f8a08010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ac5ad62a4f7672 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.386+0000 7f8a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.390+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080b71d0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.390+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8485+0+0 (secure 0 0 0) 0x7f8a080aeae0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.390+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1806b3b0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.396+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8a080aece0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.400+0000 7f8a26bf2740 10 client.17955 fetch_fsmap learned FSMap version 54 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.400+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559907225ab0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.400+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080ac530 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8370+0+0 (secure 0 0 0) 0x7f8a080b34b0 con 0x559906dd5390 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 10 client.17955 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 20 client.17955 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906d89da0 0x559907228260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 4 client.17955 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:32:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.401+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8a1806b3b0 con 0x559906d89da0 2024-09-21T03:32:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.407+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906d89da0 0x559907228260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.417+0000 7f8a01ffb640 10 client.17955 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:06.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.417+0000 7f8a1e41a640 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906d89da0 0x559907228260 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf00175ac1fd226a server_cookie=feb229adcad7edb0 in_seq=0 out_seq=0 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a01ffb640 1 -- 172.21.15.83:0/899246927 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x7f8a1806b3b0 con 0x559906d89da0 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a01ffb640 10 client.17955 handle_command_reply: tid=0 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 client.17955 shutdown 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 msgr2=0x5599071aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 0x5599071aca80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8a080e50b0 tx=0x7f8a08010f70 comp rx=0 tx=0).stop 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 shutdown_connections 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x559906d89da0 0x559907228260 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a01ffb640 0 client.17955 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559906df6070 0x5599071acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559906dd5390 0x5599071aca80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 --2- 172.21.15.83:0/899246927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559906dcd1b0 0x5599071b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.423+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 >> 172.21.15.83:0/899246927 conn(0x559906dea460 msgr2=0x5599071a9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.424+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 shutdown_connections 2024-09-21T03:32:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.424+0000 7f8a26bf2740 1 -- 172.21.15.83:0/899246927 wait complete. 2024-09-21T03:32:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.424+0000 7f8a26bf2740 20 client.17955 trim_cache size 0 max 16384 2024-09-21T03:32:06.428 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:32:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-09-21T03:32:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28": { 2024-09-21T03:32:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28", 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28": { 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:32:03.791935+0000", 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:32:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "20ac5392-b139-40a5-bf76-f9084f445257": { 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:31:15.839749+0000", 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:32:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 msgr2=0x7f8a040b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 0x7f8a040b8b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005940 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 msgr2=0x7f8a1009d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800e9c0 tx=0x7f8a0800ee80 comp rx=0 tx=0).stop 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 shutdown_connections 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8a040b6610 0x7f8a040b8b20 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005940 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a1009dc80 0x7f8a10136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1009b2c0 0x7f8a1009d6a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 --2- 172.21.15.83:0/3055871177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1009a970 0x7f8a1009d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:06.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.426+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 >> 172.21.15.83:0/3055871177 conn(0x7f8a1000c500 msgr2=0x7f8a10091240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:06.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.429+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 shutdown_connections 2024-09-21T03:32:06.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:06.429+0000 7f8a1f41c640 1 -- 172.21.15.83:0/3055871177 wait complete. 2024-09-21T03:32:07.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:06 smithi193 ceph-mon[35224]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:07.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:06 smithi193 ceph-mon[35224]: pgmap v807: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 873 op/s 2024-09-21T03:32:07.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:06 smithi083 ceph-mon[31219]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:07.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:06 smithi083 ceph-mon[31219]: pgmap v807: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 873 op/s 2024-09-21T03:32:07.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:06 smithi142 ceph-mon[34512]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:07.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:06 smithi142 ceph-mon[34512]: pgmap v807: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 873 op/s 2024-09-21T03:32:07.196 INFO:tasks.cephfs.filesystem:scrub status for tag:dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28 - {'path': '/', 'tag': 'dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28', 'options': 'recursive,force'} 2024-09-21T03:32:07.991 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:07 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:08.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:07 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:08.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:07 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:08 smithi193 ceph-mon[35224]: pgmap v808: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-09-21T03:32:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:09.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:08 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:32:09.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:08 smithi083 ceph-mon[31219]: pgmap v808: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-09-21T03:32:09.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:09.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:09.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:08 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:32:09.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:08 smithi142 ceph-mon[34512]: pgmap v808: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-09-21T03:32:09.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:09.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:09.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:08 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:32:10.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:09 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:10.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:09 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:10.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:09 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 890.0 s, 1707.0 tps, lat 57.994 ms stddev 68.290 2024-09-21T03:32:11.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:10 smithi193 ceph-mon[35224]: pgmap v809: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-21T03:32:11.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:10 smithi083 ceph-mon[31219]: pgmap v809: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-21T03:32:11.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:10 smithi142 ceph-mon[34512]: pgmap v809: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-21T03:32:13.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:12 smithi193 ceph-mon[35224]: pgmap v810: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.77k op/s 2024-09-21T03:32:13.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:12 smithi083 ceph-mon[31219]: pgmap v810: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.77k op/s 2024-09-21T03:32:13.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:12 smithi142 ceph-mon[34512]: pgmap v810: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 1.77k op/s 2024-09-21T03:32:14.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:13 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:32:14.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:13 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:32:14.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:13 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:32:14.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:13 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:32:14.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:13 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:32:14.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:13 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:32:14.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:13 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:32:14.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:13 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:32:14.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:13 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:32:15.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:32:15.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:14 smithi193 ceph-mon[35224]: scrub summary: idle 2024-09-21T03:32:15.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:14 smithi193 ceph-mon[35224]: pgmap v811: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-09-21T03:32:15.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:14 smithi193 ceph-mon[35224]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:32:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:32:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:14 smithi083 ceph-mon[31219]: scrub summary: idle 2024-09-21T03:32:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:14 smithi083 ceph-mon[31219]: pgmap v811: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-09-21T03:32:15.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:14 smithi083 ceph-mon[31219]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:32:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:32:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:14 smithi142 ceph-mon[34512]: scrub summary: idle 2024-09-21T03:32:15.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:14 smithi142 ceph-mon[34512]: pgmap v811: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-09-21T03:32:15.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:14 smithi142 ceph-mon[34512]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:32:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 895.0 s, 1044.8 tps, lat 87.907 ms stddev 103.995 2024-09-21T03:32:17.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:16 smithi193 ceph-mon[35224]: pgmap v813: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-09-21T03:32:17.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:16 smithi083 ceph-mon[31219]: pgmap v813: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-09-21T03:32:17.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:16 smithi142 ceph-mon[34512]: pgmap v813: 97 pgs: 97 active+clean; 4.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-09-21T03:32:19.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:18 smithi193 ceph-mon[35224]: pgmap v814: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-09-21T03:32:19.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:18 smithi083 ceph-mon[31219]: pgmap v814: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-09-21T03:32:19.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:18 smithi142 ceph-mon[34512]: pgmap v814: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-09-21T03:32:20.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:19 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:20.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:19 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:20.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:19 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 900.0 s, 817.4 tps, lat 133.175 ms stddev 469.611 2024-09-21T03:32:21.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:20 smithi193 ceph-mon[35224]: pgmap v815: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-21T03:32:21.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:20 smithi083 ceph-mon[31219]: pgmap v815: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:transaction type: 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:scaling factor: 32 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:query mode: simple 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:number of clients: 100 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:number of threads: 4 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:duration: 900 s 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:number of transactions actually processed: 1946580 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:latency average = 46.232 ms 2024-09-21T03:32:21.163 INFO:teuthology.orchestra.run.smithi083.stdout:latency stddev = 109.763 ms 2024-09-21T03:32:21.164 INFO:teuthology.orchestra.run.smithi083.stdout:tps = 2162.322701 (including connections establishing) 2024-09-21T03:32:21.164 INFO:teuthology.orchestra.run.smithi083.stdout:tps = 2162.328607 (excluding connections establishing) 2024-09-21T03:32:21.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-21T03:32:21.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:20 smithi142 ceph-mon[34512]: pgmap v815: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-21T03:32:23.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:22 smithi083 ceph-mon[31219]: pgmap v816: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 1.07k op/s 2024-09-21T03:32:23.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:22 smithi142 ceph-mon[34512]: pgmap v816: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 1.07k op/s 2024-09-21T03:32:23.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:22 smithi193 ceph-mon[35224]: pgmap v816: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 1.07k op/s 2024-09-21T03:32:25.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:24 smithi083 ceph-mon[31219]: pgmap v817: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:32:25.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:24 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:25.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:24 smithi142 ceph-mon[34512]: pgmap v817: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:32:25.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:24 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:24 smithi193 ceph-mon[35224]: pgmap v817: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-21T03:32:25.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:24 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:27.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:26 smithi083 ceph-mon[31219]: pgmap v818: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-09-21T03:32:27.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:26 smithi142 ceph-mon[34512]: pgmap v818: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-09-21T03:32:27.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:26 smithi193 ceph-mon[35224]: pgmap v818: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-09-21T03:32:29.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:28 smithi083 ceph-mon[31219]: pgmap v819: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-09-21T03:32:29.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:28 smithi142 ceph-mon[34512]: pgmap v819: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-09-21T03:32:29.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:28 smithi193 ceph-mon[35224]: pgmap v819: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-09-21T03:32:30.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 data 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:total 36 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 . 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 .. 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 base 2024-09-21T03:32:30.708 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 21 03:16 current_logfiles 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 03:16 global 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 log 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_commit_ts 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_dynshmem 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 03:16 pg_hba.conf 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 03:16 pg_ident.conf 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 pg_logical 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 pg_multixact 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_notify 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_replslot 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_serial 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_snapshots 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 pg_stat 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:32 pg_stat_tmp 2024-09-21T03:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:32 pg_subtrans 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_tblspc 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 pg_twophase 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 03:32 pg_wal 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:23 pg_xact 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 03:16 postgresql.auto.conf 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 03:16 postgresql.conf 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 21 03:16 postmaster.opts 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T03:32:30.710 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 . 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 1 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 13493 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 03:20 13494 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:17 pgsql_tmp 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 . 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 .. 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 1247 2024-09-21T03:32:30.711 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1247_vm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:16 1249 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1249_fsm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1249_vm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:16 1259 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1259_vm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:32:30.712 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:32:30.713 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2579 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:32:30.714 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:32:30.715 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606_vm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:16 2608 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2608_vm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2610 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:32:30.716 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2610_vm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:32:30.717 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 03:16 2619 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2619_fsm 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2619_vm 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:32:30.718 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:16 2658 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 2659 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2662 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2663 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2664 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2665 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2666 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2667 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:32:30.719 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:16 2673 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:16 2674 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2678 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2679 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:32:30.720 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2696 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2703 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2704 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:32:30.721 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:32:30.722 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:32:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:32:30.724 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3455 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:32:30.725 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:32:30.726 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:32:30.727 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:32:30.728 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:32:30.729 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:32:30.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:32:30.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:32:30.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:32:30.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:32:30.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:32:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 03:16 . 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 .. 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 1247 2024-09-21T03:32:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1247_vm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:16 1249 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1249_fsm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1249_vm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:16 1259 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1259_vm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:32:30.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:32:30.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2579 2024-09-21T03:32:30.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:32:30.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2606_vm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:16 2608 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2608_vm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:32:30.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2610 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2610_vm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:32:30.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 03:16 2619 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2619_fsm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2619_vm 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:32:30.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:16 2658 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:16 2659 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2662 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2663 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2664 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2665 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2666 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2667 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:32:30.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:16 2673 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:16 2674 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2678 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2679 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:32:30.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2696 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2703 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2704 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:32:30.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:32:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:32:30.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:32:30.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:32:30.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3455 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:32:30.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:32:30.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:32:30.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:32:30.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:32:30.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:32:30.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:32:30.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:32:30.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:32:30.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:total 617358 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 03:20 . 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 .. 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 112 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 113 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:17 1247 2024-09-21T03:32:30.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1247_fsm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 1247_vm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 03:17 1249 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:17 1249_fsm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 1249_vm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 03:16 1255 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1255_fsm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1255_vm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 03:32 1259 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1259_fsm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 1259_vm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 03:16 13344 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13344_fsm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13344_vm 2024-09-21T03:32:30.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13346 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13348 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13349_fsm 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13349_vm 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13351 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13353 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13354_fsm 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13354_vm 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13356 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13358 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 13359_fsm 2024-09-21T03:32:30.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13359_vm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 13361 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 13363 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1417 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 1418 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 445120512 Sep 21 03:32 16396 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 21 03:32 16396_fsm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:20 16396_vm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 507904 Sep 21 03:32 16397 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:32 16397_fsm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:32 16397_vm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1818624 Sep 21 03:32 16399 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:32 16399_fsm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:32 16399_vm 2024-09-21T03:32:30.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:32 16400 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:32 16402 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 21 03:32 16404 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 104341504 Sep 21 03:32 16406 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:32 16406_fsm 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:28 16406_vm 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 174 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 175 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2187 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2224 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2328 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2336 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2337 2024-09-21T03:32:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2579 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2600 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2600_fsm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2600_vm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2601_fsm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2601_vm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2602 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2602_fsm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2602_vm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2603 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2603_fsm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2603_vm 2024-09-21T03:32:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2604 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2605 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2605_fsm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2605_vm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 2606 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2606_fsm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 2606_vm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2607 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2607_fsm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2607_vm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 03:17 2608 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2608_fsm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 2608_vm 2024-09-21T03:32:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 03:16 2609 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2609_fsm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2609_vm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:17 2610 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2610_fsm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 2610_vm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2611 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2612_fsm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2612_vm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2613 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2615_fsm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2615_vm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2616 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2616_fsm 2024-09-21T03:32:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2616_vm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 03:16 2617 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2617_fsm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2617_vm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 03:16 2618 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2618_fsm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2618_vm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 21 03:32 2619 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:19 2619_fsm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:17 2619_vm 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2620 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2650 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2651 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2652 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2653 2024-09-21T03:32:30.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2654 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2655 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2656 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2657 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 03:17 2658 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 03:17 2659 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2660 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2661 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:17 2662 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:17 2663 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2664 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2665 2024-09-21T03:32:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2666 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2667 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2668 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2669 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2670 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 03:17 2673 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 03:17 2674 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 03:16 2675 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2678 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:17 2679 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2680 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2681 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2682 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2683 2024-09-21T03:32:30.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2684 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2685 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2686 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2687 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 2688 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 03:16 2689 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 03:16 2690 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 03:16 2691 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2692 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2693 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:21 2696 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2699 2024-09-21T03:32:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2701 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2702 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:17 2703 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:17 2704 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2753 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2753_fsm 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2753_vm 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2754 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2755 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2756 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 2757 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2830 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2831 2024-09-21T03:32:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2832 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2833 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2834 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2835 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2836 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2837 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 03:16 2838 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2838_fsm 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2838_vm 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2839 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2840_fsm 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2840_vm 2024-09-21T03:32:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2841 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2995 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2996 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3079_fsm 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3079_vm 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3080 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3081 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:16 3085 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3118 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3119 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 03:16 3164 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3256 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3257 2024-09-21T03:32:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3258 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3350 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3351 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3379 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3380 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3381 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3394 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3394_fsm 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3394_vm 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3395 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3429 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3430 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3431 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3433 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3439 2024-09-21T03:32:30.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3440 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 03:17 3455 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 03:16 3456 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3456_fsm 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3456_vm 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3466 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3467 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3468 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3501 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3502 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3503 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3534 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541 2024-09-21T03:32:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3541_fsm 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3541_vm 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3542 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3574 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3575 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3576 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3596 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3597 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3598 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3599 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3600_fsm 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3600_vm 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601 2024-09-21T03:32:30.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3601_fsm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3601_vm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3602_fsm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3602_vm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3603_fsm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3603_vm 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3604 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3605 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3606 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3607 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3608 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 03:16 3609 2024-09-21T03:32:30.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3712 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 3764_fsm 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3764_vm 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3766 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 3767 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3997 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4143 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4144 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4145 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4146 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4147 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4148 2024-09-21T03:32:30.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4149 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4150 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4151 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4152 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4153 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4154 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4155 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4156 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4157 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4158 2024-09-21T03:32:30.806 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4159 2024-09-21T03:32:30.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4160 2024-09-21T03:32:30.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4161 2024-09-21T03:32:30.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4162 2024-09-21T03:32:30.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4163 2024-09-21T03:32:30.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4164 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4165 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4166 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4167 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4168 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4169 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4170 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4171 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4172 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4173 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4174 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 5002 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 548 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 549 2024-09-21T03:32:30.808 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6102 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6104 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6106 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6110 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6111 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6112 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6113 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6117 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 826 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 827 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 828 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 21 03:16 pg_internal.init 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:16 PG_VERSION 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-21T03:32:30.809 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:17 . 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 03:17 .. 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:total 560 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 03:16 . 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1213 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1213_fsm 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1213_vm 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1214 2024-09-21T03:32:30.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1214_fsm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1214_vm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 1232 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 1233 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1260 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1260_fsm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1260_vm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1261 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1261_fsm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1261_vm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1262 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 1262_fsm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 1262_vm 2024-09-21T03:32:30.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2396 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 03:16 2396_fsm 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2396_vm 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2397 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2671 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2672 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2676 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2677 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2694 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2695 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2697 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 03:16 2698 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2846 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2847 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2964 2024-09-21T03:32:30.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2965 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 2966 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 2967 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 3592 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 3593 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4060 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4061 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4175 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4176 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4177 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4178 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4181 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4182 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4183 2024-09-21T03:32:30.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4184 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 4185 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 4186 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6000 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6001 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6002 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 6100 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6114 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 6115 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:32 pg_control 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 03:16 pg_filenode.map 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 21 03:16 pg_internal.init 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T03:32:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:total 2 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1362 Sep 21 03:32 postgresql-Sat.log 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.815 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:total 1 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 . 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 mappings 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 03:32 replorigin_checkpoint 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 snapshots 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 .. 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.816 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 .. 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 . 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 members 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 offsets 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 .. 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:16 0000 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T03:32:30.817 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:16 . 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:16 .. 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 03:20 0000 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.818 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:total 15 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 03:32 . 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 21 03:32 db_0.stat 2024-09-21T03:32:30.819 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 21 03:32 db_13494.stat 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 21 03:32 global.stat 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:32 . 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:total 184 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 03:32 . 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Sep 21 03:32 001D 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.820 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:total 1048576 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 03:32 . 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:32 0000000100000000000000A5 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000A6 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000A7 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000A8 2024-09-21T03:32:30.821 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000A9 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000AA 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000AB 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000AC 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000AD 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000AE 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000AF 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B0 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B1 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000B2 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B3 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000B4 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B5 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B6 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000B7 2024-09-21T03:32:30.822 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000B8 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:26 0000000100000000000000B9 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:24 0000000100000000000000BA 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:26 0000000100000000000000BB 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:26 0000000100000000000000BC 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:25 0000000100000000000000BD 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:23 0000000100000000000000BE 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000BF 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000C0 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000C1 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:30 0000000100000000000000C2 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000C3 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000C4 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:30 0000000100000000000000C5 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000C6 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000C7 2024-09-21T03:32:30.823 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000C8 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000C9 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:32 0000000100000000000000CA 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000CB 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000CC 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000CD 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000CE 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:31 0000000100000000000000CF 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000D0 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000D1 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000D2 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000D3 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000D4 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000D5 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000D6 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:31 0000000100000000000000D7 2024-09-21T03:32:30.824 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000D8 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:27 0000000100000000000000D9 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000DA 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000DB 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000DC 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000DD 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000DE 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:29 0000000100000000000000DF 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:31 0000000100000000000000E0 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000E1 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000E2 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000E3 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 03:28 0000000100000000000000E4 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 archive_status 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.825 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 03:16 . 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 03:32 .. 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:total 480 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 03:23 . 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 03:32 .. 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 21 03:23 0000 2024-09-21T03:32:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 229376 Sep 21 03:32 0001 2024-09-21T03:32:30.827 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T03:32:30.843 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T03:32:30.843 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:32:30.844 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:32:30.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T03:32:30.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:30 smithi083 ceph-mon[31219]: pgmap v820: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-21T03:32:30.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T03:32:30.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:32:31.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:30 smithi142 ceph-mon[34512]: pgmap v820: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 -- 172.21.15.83:0/2824244017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 msgr2=0x7fa9e81105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 --2- 172.21.15.83:0/2824244017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e81105b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009ca0 tx=0x7fa9e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 -- 172.21.15.83:0/2824244017 shutdown_connections 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 --2- 172.21.15.83:0/2824244017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e81105b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 --2- 172.21.15.83:0/2824244017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 0x7fa9e8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 --2- 172.21.15.83:0/2824244017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 0x7fa9e8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 -- 172.21.15.83:0/2824244017 >> 172.21.15.83:0/2824244017 conn(0x7fa9e80ff500 msgr2=0x7fa9e8101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 -- 172.21.15.83:0/2824244017 shutdown_connections 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.176+0000 7fa9eeade640 1 -- 172.21.15.83:0/2824244017 wait complete. 2024-09-21T03:32:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 Processor -- start 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 -- start start 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 0x7fa9e819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 0x7fa9e819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8113a40 con 0x7fa9e8109d80 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81138c0 con 0x7fa9e8103b00 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9eeade640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8113bc0 con 0x7fa9e8109430 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53798/0 (socket says 172.21.15.83:53798) 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 -- 172.21.15.83:0/334715248 learned_addr learned my addr 172.21.15.83:0/334715248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9dffff640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 0x7fa9e819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ec87a640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 0x7fa9e819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 -- 172.21.15.83:0/334715248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 msgr2=0x7fa9e819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 0x7fa9e819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 -- 172.21.15.83:0/334715248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 msgr2=0x7fa9e819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 0x7fa9e819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.177+0000 7fa9ed07b640 1 -- 172.21.15.83:0/334715248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e81a0350 con 0x7fa9e8109d80 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9ed07b640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005cc0 tx=0x7fa9e0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0003b70 con 0x7fa9e8109d80 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e00318f0 con 0x7fa9e8109d80 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e81a05e0 con 0x7fa9e8109d80 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e002f950 con 0x7fa9e8109d80 2024-09-21T03:32:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.178+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81a6dd0 con 0x7fa9e8109d80 2024-09-21T03:32:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.179+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8103f10 con 0x7fa9e8109d80 2024-09-21T03:32:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.180+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa9e002faf0 con 0x7fa9e8109d80 2024-09-21T03:32:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.181+0000 7fa9ddffb640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 0x7fa9d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.181+0000 7fa9ec87a640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 0x7fa9d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.181+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7fa9e0108be0 con 0x7fa9e8109d80 2024-09-21T03:32:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.182+0000 7fa9ec87a640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 0x7fa9d00b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009b60 tx=0x7fa9e4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.184+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9e00cf180 con 0x7fa9e8109d80 2024-09-21T03:32:31.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:30 smithi193 ceph-mon[35224]: pgmap v820: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-21T03:32:31.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.388+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa9e810bbd0 con 0x7fa9d00b6680 2024-09-21T03:32:31.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.397+0000 7fa9ddffb640 1 -- 172.21.15.83:0/334715248 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fa9e810bbd0 con 0x7fa9d00b6680 2024-09-21T03:32:31.399 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:16:48", "first": "2024-09-21T03:17:00", "last": "2024-09-21T03:32:00", "last_pruned": "2024-09-21T03:32:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-09-21T03:32:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 msgr2=0x7fa9d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 0x7fa9d00b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009b60 tx=0x7fa9e4005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 msgr2=0x7fa9e819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005cc0 tx=0x7fa9e0031700 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 shutdown_connections 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fa9d00b6680 0x7fa9d00b8b90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e8109d80 0x7fa9e819fc50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9e8109430 0x7fa9e819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 --2- 172.21.15.83:0/334715248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9e8103b00 0x7fa9e819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 >> 172.21.15.83:0/334715248 conn(0x7fa9e80ff500 msgr2=0x7fa9e8108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 shutdown_connections 2024-09-21T03:32:31.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.399+0000 7fa9eeade640 1 -- 172.21.15.83:0/334715248 wait complete. 2024-09-21T03:32:31.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 -- 172.21.15.83:0/275381455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07981052b0 msgr2=0x7f079810d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 --2- 172.21.15.83:0/275381455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07981052b0 0x7f079810d7f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0788009ca0 tx=0x7f078802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 -- 172.21.15.83:0/275381455 shutdown_connections 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 --2- 172.21.15.83:0/275381455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f0798110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 --2- 172.21.15.83:0/275381455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07981052b0 0x7f079810d7f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 --2- 172.21.15.83:0/275381455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0798104960 0x7f0798104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 -- 172.21.15.83:0/275381455 >> 172.21.15.83:0/275381455 conn(0x7f079807a740 msgr2=0x7f079807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 -- 172.21.15.83:0/275381455 shutdown_connections 2024-09-21T03:32:31.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 -- 172.21.15.83:0/275381455 wait complete. 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.523+0000 7f079ef32640 1 Processor -- start 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 -- start start 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07981052b0 0x7f079819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f079819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0798113990 con 0x7f0798104960 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0798113810 con 0x7f079810dd80 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ef32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0798113b10 con 0x7f07981052b0 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53816/0 (socket says 172.21.15.83:53816) 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 -- 172.21.15.83:0/1757264761 learned_addr learned my addr 172.21.15.83:0/1757264761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079d4cf640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f079819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f0797fff640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07981052b0 0x7f079819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 -- 172.21.15.83:0/1757264761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07981052b0 msgr2=0x7f079819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07981052b0 0x7f079819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 -- 172.21.15.83:0/1757264761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 msgr2=0x7f079819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f079819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 -- 172.21.15.83:0/1757264761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07981a0330 con 0x7f0798104960 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079d4cf640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f079819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f079ccce640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f079000d950 tx=0x7f079000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.524+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790017030 con 0x7f0798104960 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.525+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0790004020 con 0x7f0798104960 2024-09-21T03:32:31.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.525+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07981a0620 con 0x7f0798104960 2024-09-21T03:32:31.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.525+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07900047a0 con 0x7f0798104960 2024-09-21T03:32:31.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.525+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981a6e40 con 0x7f0798104960 2024-09-21T03:32:31.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.526+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0798104d70 con 0x7f0798104960 2024-09-21T03:32:31.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.527+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0790009a30 con 0x7f0798104960 2024-09-21T03:32:31.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.528+0000 7f0795ffb640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 0x7f07840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:31.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.528+0000 7f0797fff640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 0x7f07840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:31.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.528+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f07900e78c0 con 0x7f0798104960 2024-09-21T03:32:31.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.528+0000 7f0797fff640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 0x7f07840b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0788000f80 tx=0x7f0788009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:31.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.531+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07900ade60 con 0x7f0798104960 2024-09-21T03:32:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.731+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 --> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0798108f10 con 0x7f07840b6680 2024-09-21T03:32:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.739+0000 7f0795ffb640 1 -- 172.21.15.83:0/1757264761 <== mgr.14217 v2:172.21.15.83:6800/1298689131 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0798108f10 con 0x7f07840b6680 2024-09-21T03:32:31.741 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 msgr2=0x7f07840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 0x7f07840b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0788000f80 tx=0x7f0788009360 comp rx=0 tx=0).stop 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 msgr2=0x7f079819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f079000d950 tx=0x7f079000de10 comp rx=0 tx=0).stop 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 shutdown_connections 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f07840b6680 0x7f07840b8b90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079810dd80 0x7f079819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07981052b0 0x7f079819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 --2- 172.21.15.83:0/1757264761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0798104960 0x7f079819a490 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 >> 172.21.15.83:0/1757264761 conn(0x7f079807a740 msgr2=0x7f07980ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 shutdown_connections 2024-09-21T03:32:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:31.741+0000 7f079ef32640 1 -- 172.21.15.83:0/1757264761 wait complete. 2024-09-21T03:32:31.754 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T03:32:31.764 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T03:32:32.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:31 smithi083 ceph-mon[31219]: from='client.17961 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:32.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:31 smithi142 ceph-mon[34512]: from='client.17961 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:32.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:31 smithi193 ceph-mon[35224]: from='client.17961 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:33.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:32 smithi083 ceph-mon[31219]: pgmap v821: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.1 MiB/s wr, 881 op/s 2024-09-21T03:32:33.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:32 smithi083 ceph-mon[31219]: from='client.17967 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:33.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:32 smithi142 ceph-mon[34512]: pgmap v821: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.1 MiB/s wr, 881 op/s 2024-09-21T03:32:33.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:32 smithi142 ceph-mon[34512]: from='client.17967 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:32 smithi193 ceph-mon[35224]: pgmap v821: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.1 MiB/s wr, 881 op/s 2024-09-21T03:32:33.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:32 smithi193 ceph-mon[35224]: from='client.17967 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:35.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:35 smithi083 ceph-mon[31219]: pgmap v822: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-09-21T03:32:35.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:35 smithi142 ceph-mon[34512]: pgmap v822: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-09-21T03:32:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:35 smithi193 ceph-mon[35224]: pgmap v822: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-09-21T03:32:37.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:32:37.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:37 smithi083 ceph-mon[31219]: pgmap v823: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 8.3 MiB/s wr, 905 op/s 2024-09-21T03:32:37.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:37 smithi142 ceph-mon[34512]: pgmap v823: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 8.3 MiB/s wr, 905 op/s 2024-09-21T03:32:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:37.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:37 smithi193 ceph-mon[35224]: pgmap v823: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 8.3 MiB/s wr, 905 op/s 2024-09-21T03:32:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 -- 172.21.15.83:0/198433230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c108cf0 msgr2=0x7f391c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 --2- 172.21.15.83:0/198433230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c108cf0 0x7f391c10f520 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3914009ca0 tx=0x7f391402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 -- 172.21.15.83:0/198433230 shutdown_connections 2024-09-21T03:32:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 --2- 172.21.15.83:0/198433230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c108cf0 0x7f391c10f520 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 --2- 172.21.15.83:0/198433230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c1083a0 0x7f391c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 --2- 172.21.15.83:0/198433230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 0x7f391c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 -- 172.21.15.83:0/198433230 >> 172.21.15.83:0/198433230 conn(0x7f391c079730 msgr2=0x7f391c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 -- 172.21.15.83:0/198433230 shutdown_connections 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.881+0000 7f3921f51640 1 -- 172.21.15.83:0/198433230 wait complete. 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 Processor -- start 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 -- start start 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 0x7f391c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c108cf0 0x7f391c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f391c112960 con 0x7f391c1083a0 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1127e0 con 0x7f391c102a60 2024-09-21T03:32:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f3921f51640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f391c112ae0 con 0x7f391c108cf0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.882+0000 7f391affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53838/0 (socket says 172.21.15.83:53838) 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 -- 172.21.15.83:0/284107612 learned_addr learned my addr 172.21.15.83:0/284107612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391bfff640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c108cf0 0x7f391c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391b7fe640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 0x7f391c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 -- 172.21.15.83:0/284107612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c108cf0 msgr2=0x7f391c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c108cf0 0x7f391c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 -- 172.21.15.83:0/284107612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 msgr2=0x7f391c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 0x7f391c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 -- 172.21.15.83:0/284107612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1a6490 con 0x7f391c1083a0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f391affd640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f390400d740 tx=0x7f390400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f3918ff9640 1 -- 172.21.15.83:0/284107612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904017070 con 0x7f391c1083a0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f3918ff9640 1 -- 172.21.15.83:0/284107612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3904009820 con 0x7f391c1083a0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1a6780 con 0x7f391c1083a0 2024-09-21T03:32:38.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f3918ff9640 1 -- 172.21.15.83:0/284107612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904004ce0 con 0x7f391c1083a0 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.883+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a6c70 con 0x7f391c1083a0 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 Processor -- start 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f3918ff9640 1 -- 172.21.15.83:0/284107612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3904010460 con 0x7f391c1083a0 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 -- start start 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e26070 0x55d7a11cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0e05390 0x55d7a11ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7a1143e50 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7a1133280 con 0x55d7a0e05390 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f39284c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7a1137090 con 0x55d7a0e26070 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.886+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53846/0 (socket says 172.21.15.83:53846) 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 -- 172.21.15.83:0/1429668845 learned_addr learned my addr 172.21.15.83:0/1429668845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391bfff640 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e26070 0x55d7a11cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391affd640 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0e05390 0x55d7a11ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 -- 172.21.15.83:0/1429668845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e26070 msgr2=0x55d7a11cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e26070 0x55d7a11cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 -- 172.21.15.83:0/1429668845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0e05390 msgr2=0x55d7a11ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0e05390 0x55d7a11ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 -- 172.21.15.83:0/1429668845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0e91650 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f3918ff9640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 0x7f39080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391bfff640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 0x7f39080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f391b7fe640 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f390c00ac80 tx=0x7f390c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990fb069c5d1a530 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f3918ff9640 1 -- 172.21.15.83:0/284107612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f39040e7030 con 0x7f391c1083a0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f3911ffb640 1 -- 172.21.15.83:0/1429668845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c01d070 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f3911ffb640 1 -- 172.21.15.83:0/1429668845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c018af0 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.887+0000 7f3911ffb640 1 -- 172.21.15.83:0/1429668845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c010a60 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 -- 172.21.15.83:0/1429668845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 msgr2=0x55d7a11cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f391bfff640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 0x7f39080b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3914009a60 tx=0x7f391402f450 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f390c00ac80 tx=0x7f390c00f790 comp rx=0 tx=0).stop 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 -- 172.21.15.83:0/1429668845 shutdown_connections 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0e05390 0x55d7a11ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11cdb90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 --2- 172.21.15.83:0/1429668845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e26070 0x55d7a11cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 -- 172.21.15.83:0/1429668845 >> 172.21.15.83:0/1429668845 conn(0x55d7a0e1a460 msgr2=0x55d7a11cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 -- 172.21.15.83:0/1429668845 shutdown_connections 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.888+0000 7f39284c5740 1 -- 172.21.15.83:0/1429668845 wait complete. 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 -- start start 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 Processor -- start 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0f94700 con 0x55d7a0e26070 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7a10ce110 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7a11dbe30 con 0x55d7a0e05390 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.889+0000 7f39284c5740 10 client.0 _ll_register_callbacks cb 0x55d7a10e7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53862/0 (socket says 172.21.15.83:53862) 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 -- 172.21.15.83:0/4237818899 learned_addr learned my addr 172.21.15.83:0/4237818899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391affd640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391b7fe640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 -- 172.21.15.83:0/4237818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 msgr2=0x55d7a11d9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 -- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 msgr2=0x55d7a11d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 -- 172.21.15.83:0/4237818899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a10fbc40 con 0x55d7a0e26070 2024-09-21T03:32:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391b7fe640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f391bfff640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3914035000 tx=0x7f391402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600f6d87298368cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f38faffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914004040 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f39140041e0 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.890+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c018af0 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f39140043e0 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 10 client.17979 fetch_fsmap learned FSMap version 57 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d7a11e7b20 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c018af0 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3914051070 con 0x55d7a0e26070 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 10 client.17979 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 20 client.17979 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:38.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0de49f0 0x55d7a11eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 4 client.17979 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a1145da0 con 0x55d7a0de49f0 2024-09-21T03:32:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.891+0000 7f391b7fe640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0de49f0 0x55d7a11eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.892+0000 7f38faffd640 10 client.17979 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:38.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.892+0000 7f391b7fe640 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0de49f0 0x55d7a11eb0c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac406402bde4442 server_cookie=78cd8d131a9d44a6 in_seq=0 out_seq=0 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f38faffd640 1 -- 172.21.15.83:0/4237818899 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7a1145da0 con 0x55d7a0de49f0 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f38faffd640 10 client.17979 handle_command_reply: tid=0 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 client.17979 shutdown 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 msgr2=0x55d7a11e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3914035000 tx=0x7f391402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 shutdown_connections 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0de49f0 0x55d7a11eb0c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f38faffd640 0 client.17979 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e26070 0x55d7a11e5120 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 --2- 172.21.15.83:0/4237818899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 >> 172.21.15.83:0/4237818899 conn(0x55d7a0e1a460 msgr2=0x55d7a11d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 shutdown_connections 2024-09-21T03:32:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 1 -- 172.21.15.83:0/4237818899 wait complete. 2024-09-21T03:32:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.893+0000 7f39284c5740 20 client.17979 trim_cache size 0 max 16384 2024-09-21T03:32:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.911+0000 7f39284c5740 1 Processor -- start 2024-09-21T03:32:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.911+0000 7f39284c5740 1 -- start start 2024-09-21T03:32:38.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 0x55d7a11e5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1af510 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b1140 con 0x55d7a0de49f0 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f39284c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1afbe0 con 0x55d7a0e05390 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53872/0 (socket says 172.21.15.83:53872) 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 0x55d7a11e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 -- 172.21.15.83:0/3207075315 learned_addr learned my addr 172.21.15.83:0/3207075315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391bfff640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 -- 172.21.15.83:0/3207075315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 msgr2=0x55d7a11d6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 -- 172.21.15.83:0/3207075315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 msgr2=0x55d7a11e5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 0x55d7a11e5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391affd640 1 -- 172.21.15.83:0/3207075315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a11cc300 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391bfff640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.912+0000 7f391b7fe640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 0x55d7a11e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.913+0000 7f391affd640 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f39040e4dd0 tx=0x7f39040ea740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ddebf44be7cf7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.913+0000 7f39117fa640 1 -- 172.21.15.83:0/3207075315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904020040 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.913+0000 7f39117fa640 1 -- 172.21.15.83:0/3207075315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39040ac9e0 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.913+0000 7f39117fa640 1 -- 172.21.15.83:0/3207075315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040a9140 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- 172.21.15.83:0/3207075315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 msgr2=0x55d7a11d5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f39040e4dd0 tx=0x7f39040ea740 comp rx=0 tx=0).stop 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- 172.21.15.83:0/3207075315 shutdown_connections 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0e05390 0x55d7a11d6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11d5ee0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 --2- 172.21.15.83:0/3207075315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0de49f0 0x55d7a11e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- 172.21.15.83:0/3207075315 >> 172.21.15.83:0/3207075315 conn(0x55d7a0e1a460 msgr2=0x55d7a11d7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- 172.21.15.83:0/3207075315 shutdown_connections 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- 172.21.15.83:0/3207075315 wait complete. 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 -- start start 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.914+0000 7f39284c5740 1 Processor -- start 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0de49f0 0x55d7a11dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b09e0 con 0x55d7a0e05390 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b0310 con 0x55d7a0dfd1b0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ad510 con 0x55d7a0de49f0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f39284c5740 10 client.0 _ll_register_callbacks cb 0x7f391402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53874/0 (socket says 172.21.15.83:53874) 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 -- 172.21.15.83:0/918042212 learned_addr learned my addr 172.21.15.83:0/918042212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391b7fe640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0de49f0 0x55d7a11dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391affd640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 -- 172.21.15.83:0/918042212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0de49f0 msgr2=0x55d7a11dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0de49f0 0x55d7a11dd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 -- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 msgr2=0x55d7a11dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 -- 172.21.15.83:0/918042212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a11de100 con 0x55d7a0e05390 2024-09-21T03:32:38.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f391bfff640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3914005e30 tx=0x7f3914004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b9872bd9cef8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.915+0000 7f38faffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914060020 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f3914004790 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39040ac9e0 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f391406b050 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 10 client.17985 fetch_fsmap learned FSMap version 57 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d7a1256010 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39040ac9e0 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3914051040 con 0x55d7a0e05390 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 10 client.17985 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 20 client.17985 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0e26070 0x55d7a12588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 4 client.17985 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:32:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.916+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7a1141850 con 0x55d7a0e26070 2024-09-21T03:32:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.917+0000 7f391b7fe640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0e26070 0x55d7a12588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.917+0000 7f391b7fe640 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0e26070 0x55d7a12588b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f19e4aafd805e74 server_cookie=86ad0c067b2f2311 in_seq=0 out_seq=0 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.917+0000 7f38faffd640 10 client.17985 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.917+0000 7f38faffd640 1 -- 172.21.15.83:0/918042212 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x55d7a1141850 con 0x55d7a0e26070 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.917+0000 7f38faffd640 10 client.17985 handle_command_reply: tid=0 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 client.17985 shutdown 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 msgr2=0x55d7a1254a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3914005e30 tx=0x7f3914004210 comp rx=0 tx=0).stop 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 shutdown_connections 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x55d7a0e26070 0x55d7a12588b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7a0e05390 0x55d7a1254a80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7a0dfd1b0 0x55d7a11dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f38faffd640 0 client.17985 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 --2- 172.21.15.83:0/918042212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0de49f0 0x55d7a11dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 >> 172.21.15.83:0/918042212 conn(0x55d7a0e1a460 msgr2=0x55d7a11d7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 shutdown_connections 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 1 -- 172.21.15.83:0/918042212 wait complete. 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.918+0000 7f39284c5740 20 client.17985 trim_cache size 0 max 16384 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {}, 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_stats": { 2024-09-21T03:32:38.924 INFO:teuthology.orchestra.run.smithi083.stdout: "dbb9f927-2c33-4cf2-9f1e-eee09b3c7e28": { 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:32:03.791935+0000", 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "fec9277f-ee8e-4d8f-a1da-951b855d3d0e": { 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:30:19.636701+0000", 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.925 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "d107bf75-3d29-4357-a930-5487411ac4d2": { 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:29:29.969546+0000", 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "1d7afd16-69f4-4d76-9a37-cc9fcfa820ea": { 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:28:37.175446+0000", 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.926 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "18fa4bc2-31be-4b57-8aa4-e8eb4c6e47a4": { 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:27:48.125283+0000", 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "1d819a88-2c9a-4459-a4f4-ef203dd15338": { 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:16:48.507182+0000", 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.927 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "0a46e274-636d-4f02-882e-a6f7cd613a3d": { 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:18:27.652327+0000", 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "d0cbca43-4c1e-4314-b9b4-5725bc86d92f": { 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:17:38.112253+0000", 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.928 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "90072a91-addd-4dbf-a10c-84c6ce031faf": { 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:24:25.126757+0000", 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "81bdb2f6-a018-40f8-b3a8-29c75c89e9fc": { 2024-09-21T03:32:38.929 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:23:32.399702+0000", 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "90c69917-6ace-4c38-8d33-f0da1deff7a8": { 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:20:11.656399+0000", 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "b80ab80a-2e5b-4614-b429-33354044a838": { 2024-09-21T03:32:38.930 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:02.894452+0000", 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "172a5c6b-cfa1-4e83-87f7-40e01466199c": { 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:21:53.250522+0000", 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "0d6230b2-cfc0-4376-939d-c59a2997a3f3": { 2024-09-21T03:32:38.931 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:22:41.759627+0000", 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "2338c509-a119-4e4c-8946-5b1a3b649fe8": { 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:58.574130+0000", 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "20ac5392-b139-40a5-bf76-f9084f445257": { 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:31:15.839749+0000", 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "1070de3e-e59d-4ef4-b62c-b7a5dbb1cb09": { 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:19:19.205371+0000", 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "e1d42f2c-77b3-4186-9c20-c7446db13fce": { 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:25:16.870862+0000", 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "e5cdcd58-df20-47cb-bced-27da9f50e8be": { 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "start_time": "2024-09-21T03:26:06.285024+0000", 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_passed": 0, 2024-09-21T03:32:38.934 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_failed": 0, 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_skipped": 0 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 msgr2=0x7f39080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 0x7f39080b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3914009a60 tx=0x7f391402f450 comp rx=0 tx=0).stop 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 msgr2=0x7f391c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:38.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f390400d740 tx=0x7f390400db40 comp rx=0 tx=0).stop 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 shutdown_connections 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f39080b6680 0x7f39080b8b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f391c108cf0 0x7f391c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f391c1083a0 0x7f391c19b750 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 --2- 172.21.15.83:0/284107612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f391c102a60 0x7f391c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 >> 172.21.15.83:0/284107612 conn(0x7f391c079730 msgr2=0x7f391c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 shutdown_connections 2024-09-21T03:32:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:38.919+0000 7f3921f51640 1 -- 172.21.15.83:0/284107612 wait complete. 2024-09-21T03:32:39.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:39 smithi083 ceph-mon[31219]: pgmap v824: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 7.5 MiB/s wr, 823 op/s 2024-09-21T03:32:39.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:39 smithi142 ceph-mon[34512]: pgmap v824: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 7.5 MiB/s wr, 823 op/s 2024-09-21T03:32:39.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:39 smithi193 ceph-mon[35224]: pgmap v824: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 7.5 MiB/s wr, 823 op/s 2024-09-21T03:32:39.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:32:39.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:32:39.942 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 -- 172.21.15.83:0/2789419886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc102a80 msgr2=0x7fdcdc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/2789419886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc102a80 0x7fdcdc102e90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009ca0 tx=0x7fdccc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 -- 172.21.15.83:0/2789419886 shutdown_connections 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/2789419886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc108d10 0x7fdcdc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/2789419886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 0x7fdcdc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/2789419886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc102a80 0x7fdcdc102e90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 -- 172.21.15.83:0/2789419886 >> 172.21.15.83:0/2789419886 conn(0x7fdcdc079740 msgr2=0x7fdcdc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 -- 172.21.15.83:0/2789419886 shutdown_connections 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.433+0000 7fdce3ef9640 1 -- 172.21.15.83:0/2789419886 wait complete. 2024-09-21T03:32:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 Processor -- start 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 -- start start 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc102a80 0x7fdcdc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 0x7fdcdc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1129e0 con 0x7fdcdc108d10 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc112860 con 0x7fdcdc1083c0 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce3ef9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc112b60 con 0x7fdcdc102a80 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce2496640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce2496640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48006/0 (socket says 172.21.15.83:48006) 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce2496640 1 -- 172.21.15.83:0/3502543659 learned_addr learned my addr 172.21.15.83:0/3502543659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce1494640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 0x7fdcdc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce1c95640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc102a80 0x7fdcdc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.434+0000 7fdce2496640 1 -- 172.21.15.83:0/3502543659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc102a80 msgr2=0x7fdcdc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce2496640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc102a80 0x7fdcdc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce2496640 1 -- 172.21.15.83:0/3502543659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 msgr2=0x7fdcdc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce2496640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 0x7fdcdc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce2496640 1 -- 172.21.15.83:0/3502543659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a0450 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce2496640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4011830 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4016070 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a0740 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4010de0 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.435+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a6ea0 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.436+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc102e90 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.437+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdcd40119d0 con 0x7fdcdc108d10 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.438+0000 7fdcdaffd640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 0x7fdcc40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:40.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.438+0000 7fdce1c95640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 0x7fdcc40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.438+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7fdcd40e72c0 con 0x7fdcdc108d10 2024-09-21T03:32:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.439+0000 7fdce1c95640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 0x7fdcc40b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdccc000f80 tx=0x7fdccc009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:40.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.441+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcd40ad860 con 0x7fdcdc108d10 2024-09-21T03:32:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.656+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcdc19b610 con 0x7fdcdc108d10 2024-09-21T03:32:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.657+0000 7fdcdaffd640 1 -- 172.21.15.83:0/3502543659 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11990 (secure 0 0 0) 0x7fdcdc19b610 con 0x7fdcdc108d10 2024-09-21T03:32:40.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:40.659 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":57,"btime":"2024-09-21T03:32:23:812652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:32:23.267214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24647,24628,24620,24613,24601]},"id":1}]} 2024-09-21T03:32:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 57 2024-09-21T03:32:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 msgr2=0x7fdcc40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 0x7fdcc40b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdccc000f80 tx=0x7fdccc009360 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 msgr2=0x7fdcdc19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 shutdown_connections 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fdcc40b6700 0x7fdcc40b8c10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcdc108d10 0x7fdcdc19fd20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcdc1083c0 0x7fdcdc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 --2- 172.21.15.83:0/3502543659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcdc102a80 0x7fdcdc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.659+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 >> 172.21.15.83:0/3502543659 conn(0x7fdcdc079740 msgr2=0x7fdcdc106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.660+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 shutdown_connections 2024-09-21T03:32:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:40.660+0000 7fdce3ef9640 1 -- 172.21.15.83:0/3502543659 wait complete. 2024-09-21T03:32:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:41 smithi083 ceph-mon[31219]: pgmap v825: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-21T03:32:41.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:41 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3502543659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:41 smithi142 ceph-mon[34512]: pgmap v825: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-21T03:32:41.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:41 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3502543659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:41.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:32:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:41 smithi193 ceph-mon[35224]: pgmap v825: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-21T03:32:41.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:41 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3502543659' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 -- 172.21.15.83:0/1574437874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c1052f0 msgr2=0x7f996c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 --2- 172.21.15.83:0/1574437874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c1052f0 0x7f996c10d930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f996002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 -- 172.21.15.83:0/1574437874 shutdown_connections 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 --2- 172.21.15.83:0/1574437874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c10de70 0x7f996c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 --2- 172.21.15.83:0/1574437874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c1052f0 0x7f996c10d930 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 --2- 172.21.15.83:0/1574437874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1048c0 0x7f996c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 -- 172.21.15.83:0/1574437874 >> 172.21.15.83:0/1574437874 conn(0x7f996c079720 msgr2=0x7f996c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 -- 172.21.15.83:0/1574437874 shutdown_connections 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.413+0000 7f9971556640 1 -- 172.21.15.83:0/1574437874 wait complete. 2024-09-21T03:32:42.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 Processor -- start 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 -- start start 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c1048c0 0x7f996c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 0x7f996c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113ac0 con 0x7f996c10de70 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113940 con 0x7f996c1052f0 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.414+0000 7f9971556640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113c40 con 0x7f996c1048c0 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48026/0 (socket says 172.21.15.83:48026) 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 -- 172.21.15.83:0/1165083961 learned_addr learned my addr 172.21.15.83:0/1165083961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996affd640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c1048c0 0x7f996c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996a7fc640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 0x7f996c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 -- 172.21.15.83:0/1165083961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c1048c0 msgr2=0x7f996c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c1048c0 0x7f996c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 -- 172.21.15.83:0/1165083961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 msgr2=0x7f996c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 0x7f996c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 -- 172.21.15.83:0/1165083961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1a0440 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996a7fc640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 0x7f996c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f996b7fe640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f996400cce0 tx=0x7f996400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f9953fff640 1 -- 172.21.15.83:0/1165083961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996400f450 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a0730 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f9953fff640 1 -- 172.21.15.83:0/1165083961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996400faa0 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f9953fff640 1 -- 172.21.15.83:0/1165083961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964018b70 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.415+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1a6f50 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 Processor -- start 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9953fff640 1 -- 172.21.15.83:0/1165083961 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9964018d10 con 0x7f996c10de70 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 -- start start 2024-09-21T03:32:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f6c626ea0 con 0x557f6c2e8390 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557f6c6162d0 con 0x557f6c309070 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f9977aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f6c61a0e0 con 0x557f6c2e01b0 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f996a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f996a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48030/0 (socket says 172.21.15.83:48030) 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f996a7fc640 1 -- 172.21.15.83:0/3403739921 learned_addr learned my addr 172.21.15.83:0/3403739921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.418+0000 7f996affd640 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 -- 172.21.15.83:0/3403739921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 msgr2=0x557f6c6b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 -- 172.21.15.83:0/3403739921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 msgr2=0x557f6c6b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 -- 172.21.15.83:0/3403739921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6c374650 con 0x557f6c2e8390 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f9960031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36d80e3869f0536 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f9953fff640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 0x7f99540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f996affd640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 0x7f99540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f99517fa640 1 -- 172.21.15.83:0/3403739921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960041020 con 0x557f6c2e8390 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f9953fff640 1 -- 172.21.15.83:0/1165083961 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f99640e7bd0 con 0x7f996c10de70 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f99517fa640 1 -- 172.21.15.83:0/3403739921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99600317a0 con 0x557f6c2e8390 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.419+0000 7f99517fa640 1 -- 172.21.15.83:0/3403739921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960035070 con 0x557f6c2e8390 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- 172.21.15.83:0/3403739921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 msgr2=0x557f6c6b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f996affd640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 0x7f99540b8bc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x557f6c6aef40 tx=0x7f995c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f9960031400 comp rx=0 tx=0).stop 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- 172.21.15.83:0/3403739921 shutdown_connections 2024-09-21T03:32:42.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6b0d40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 --2- 172.21.15.83:0/3403739921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- 172.21.15.83:0/3403739921 >> 172.21.15.83:0/3403739921 conn(0x557f6c2fd460 msgr2=0x557f6c6ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- 172.21.15.83:0/3403739921 shutdown_connections 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- 172.21.15.83:0/3403739921 wait complete. 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.420+0000 7f9977aca740 1 -- start start 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 Processor -- start 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c2e8390 0x557f6c6bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f6c477700 con 0x557f6c309070 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557f6c5b1160 con 0x557f6c2e8390 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f6c6bea40 con 0x557f6c2e01b0 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f9977aca740 10 client.0 _ll_register_callbacks cb 0x557f6c5ca9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48038/0 (socket says 172.21.15.83:48038) 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 -- 172.21.15.83:0/3549534179 learned_addr learned my addr 172.21.15.83:0/3549534179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c2e8390 0x557f6c6bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996affd640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 -- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 msgr2=0x557f6c6bb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6bb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 -- 172.21.15.83:0/3549534179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c2e8390 msgr2=0x557f6c6bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c2e8390 0x557f6c6bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 -- 172.21.15.83:0/3549534179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6c5dec90 con 0x557f6c309070 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f996b7fe640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f99640ada00 tx=0x7f99640ad1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee4b714f7cd0c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.421+0000 7f994eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640b2020 con 0x557f6c309070 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f99640a9ce0 con 0x557f6c309070 2024-09-21T03:32:42.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99600317a0 con 0x557f6c309070 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f99640d5020 con 0x557f6c309070 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 10 client.18003 fetch_fsmap learned FSMap version 58 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557f6c6ca830 con 0x557f6c309070 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99600317a0 con 0x557f6c309070 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8386+0+0 (secure 0 0 0) 0x7f99640a93c0 con 0x557f6c309070 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 10 client.18003 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 10 client.18003 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 20 client.18003 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c2c79f0 0x557f6c6cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 4 client.18003 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.422+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f6c628df0 con 0x557f6c2c79f0 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.423+0000 7f996affd640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c2c79f0 0x557f6c6cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.423+0000 7f994eff5640 10 client.18003 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.423+0000 7f996affd640 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c2c79f0 0x557f6c6cdda0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42213a0d1e50b930 server_cookie=20fa56472283eaa6 in_seq=0 out_seq=0 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f994eff5640 1 -- 172.21.15.83:0/3549534179 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f6c628df0 con 0x557f6c2c79f0 2024-09-21T03:32:42.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f994eff5640 10 client.18003 handle_command_reply: tid=0 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 client.18003 shutdown 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 msgr2=0x557f6c6c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f99640ada00 tx=0x7f99640ad1e0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 shutdown_connections 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c2c79f0 0x557f6c6cdda0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c309070 0x557f6c6c7d60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c2e8390 0x557f6c6bbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 --2- 172.21.15.83:0/3549534179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f994eff5640 0 client.18003 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 >> 172.21.15.83:0/3549534179 conn(0x557f6c2fd460 msgr2=0x557f6c6b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 shutdown_connections 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 1 -- 172.21.15.83:0/3549534179 wait complete. 2024-09-21T03:32:42.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.424+0000 7f9977aca740 20 client.18003 trim_cache size 0 max 16384 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 Processor -- start 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 -- start start 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 0x557f6c6c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1af4b0 con 0x557f6c2e01b0 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b1110 con 0x557f6c309070 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f9977aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1afbb0 con 0x557f6c2e8390 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f996affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f996affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48048/0 (socket says 172.21.15.83:48048) 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.437+0000 7f996affd640 1 -- 172.21.15.83:0/3648063144 learned_addr learned my addr 172.21.15.83:0/3648063144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 -- 172.21.15.83:0/3648063144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 msgr2=0x557f6c6c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996b7fe640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 0x557f6c6c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 0x557f6c6c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 -- 172.21.15.83:0/3648063144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 msgr2=0x557f6c6b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 -- 172.21.15.83:0/3648063144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6c6a95d0 con 0x557f6c2e01b0 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996a7fc640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 0x557f6c6c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996b7fe640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:42.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f996affd640 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x557f6c6ce4e0 tx=0x7f995c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443af0ffe25db4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f9950ff9640 1 -- 172.21.15.83:0/3648063144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c02f070 con 0x557f6c2e01b0 2024-09-21T03:32:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f9950ff9640 1 -- 172.21.15.83:0/3648063144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c01d040 con 0x557f6c2e01b0 2024-09-21T03:32:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.438+0000 7f9950ff9640 1 -- 172.21.15.83:0/3648063144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c01c040 con 0x557f6c2e01b0 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- 172.21.15.83:0/3648063144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 msgr2=0x557f6c6b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x557f6c6ce4e0 tx=0x7f995c013000 comp rx=0 tx=0).stop 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- 172.21.15.83:0/3648063144 shutdown_connections 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e8390 0x557f6c6c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 --2- 172.21.15.83:0/3648063144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e01b0 0x557f6c6b1a00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- 172.21.15.83:0/3648063144 >> 172.21.15.83:0/3648063144 conn(0x557f6c2fd460 msgr2=0x557f6c6b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- 172.21.15.83:0/3648063144 shutdown_connections 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- 172.21.15.83:0/3648063144 wait complete. 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 -- start start 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.439+0000 7f9977aca740 1 Processor -- start 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b09b0 con 0x557f6c2e8390 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b02e0 con 0x557f6c309070 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1ad4b0 con 0x557f6c2e01b0 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f9977aca740 10 client.0 _ll_register_callbacks cb 0x7f9964018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48052/0 (socket says 172.21.15.83:48052) 2024-09-21T03:32:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 -- 172.21.15.83:0/2191955125 learned_addr learned my addr 172.21.15.83:0/2191955125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996affd640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996b7fe640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 -- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 msgr2=0x557f6c6c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 -- 172.21.15.83:0/2191955125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 msgr2=0x557f6c6bfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6bfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 -- 172.21.15.83:0/2191955125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6c6c0780 con 0x557f6c2e8390 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f996a7fc640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9960005490 tx=0x7f9960005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5c8fa47081d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f994eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960041030 con 0x557f6c2e8390 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.440+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f9960031bb0 con 0x557f6c2e8390 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c01d040 con 0x557f6c2e8390 2024-09-21T03:32:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9960057050 con 0x557f6c2e8390 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 10 client.18009 fetch_fsmap learned FSMap version 58 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557f6c7385e0 con 0x557f6c2e8390 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c01d040 con 0x557f6c2e8390 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8386+0+0 (secure 0 0 0) 0x7f996003a050 con 0x557f6c2e8390 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 10 client.18009 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 10 client.18009 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 20 client.18009 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c29cda0 0x557f6c73a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 4 client.18009 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 --> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f6c6248a0 con 0x557f6c29cda0 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.441+0000 7f996affd640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c29cda0 0x557f6c73a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f994eff5640 10 client.18009 ms_handle_connect on v2:172.21.15.193:6834/1130551949 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f996affd640 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c29cda0 0x557f6c73a8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9d5db0986eb30f server_cookie=ec8e96265810df6a in_seq=0 out_seq=0 2024-09-21T03:32:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f994eff5640 1 -- 172.21.15.83:0/2191955125 <== mds.4 v2:172.21.15.193:6834/1130551949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f6c6248a0 con 0x557f6c29cda0 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f994eff5640 10 client.18009 handle_command_reply: tid=0 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 client.18009 shutdown 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 msgr2=0x557f6c6bfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9960005490 tx=0x7f9960005fd0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 shutdown_connections 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:6834/1130551949,v1:172.21.15.193:6835/1130551949] conn(0x557f6c29cda0 0x557f6c73a8d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f6c309070 0x557f6c6bfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6c2e8390 0x557f6c6bfa30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 --2- 172.21.15.83:0/2191955125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f6c2e01b0 0x557f6c6c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f994eff5640 0 client.18009 ms_handle_reset on v2:172.21.15.193:6834/1130551949 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 >> 172.21.15.83:0/2191955125 conn(0x557f6c2fd460 msgr2=0x557f6c6b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 shutdown_connections 2024-09-21T03:32:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 1 -- 172.21.15.83:0/2191955125 wait complete. 2024-09-21T03:32:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.442+0000 7f9977aca740 20 client.18009 trim_cache size 0 max 16384 2024-09-21T03:32:42.450 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 msgr2=0x7f99540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 0x7f99540b8bc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x557f6c6aef40 tx=0x7f995c008040 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 msgr2=0x7f996c19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f996400cce0 tx=0x7f996400a620 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 shutdown_connections 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f99540b66b0 0x7f99540b8bc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996c10de70 0x7f996c19fdd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f996c1052f0 0x7f996c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 --2- 172.21.15.83:0/1165083961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f996c1048c0 0x7f996c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 >> 172.21.15.83:0/1165083961 conn(0x7f996c079720 msgr2=0x7f996c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 shutdown_connections 2024-09-21T03:32:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:42.443+0000 7f9971556640 1 -- 172.21.15.83:0/1165083961 wait complete. 2024-09-21T03:32:43.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:32:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:43 smithi083 ceph-mon[31219]: pgmap v826: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-21T03:32:43.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:43 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:43.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:43 smithi142 ceph-mon[34512]: pgmap v826: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-21T03:32:43.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:43 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:43 smithi193 ceph-mon[35224]: pgmap v826: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-21T03:32:43.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:43 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.932+0000 7f4de2665640 1 -- 172.21.15.83:0/401925768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc102ae0 msgr2=0x7f4ddc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.932+0000 7f4de2665640 1 --2- 172.21.15.83:0/401925768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc102ef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009900 tx=0x7f4dcc02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 -- 172.21.15.83:0/401925768 shutdown_connections 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 --2- 172.21.15.83:0/401925768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108d70 0x7f4ddc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 --2- 172.21.15.83:0/401925768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc108420 0x7f4ddc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 --2- 172.21.15.83:0/401925768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc102ef0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 -- 172.21.15.83:0/401925768 >> 172.21.15.83:0/401925768 conn(0x7f4ddc079770 msgr2=0x7f4ddc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 -- 172.21.15.83:0/401925768 shutdown_connections 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 -- 172.21.15.83:0/401925768 wait complete. 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 Processor -- start 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.933+0000 7f4de2665640 1 -- start start 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108420 0x7f4ddc19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc112a30 con 0x7f4ddc108d70 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1128b0 con 0x7f4ddc108420 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de2665640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc112bb0 con 0x7f4ddc102ae0 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48066/0 (socket says 172.21.15.83:48066) 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 -- 172.21.15.83:0/4068395420 learned_addr learned my addr 172.21.15.83:0/4068395420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108420 0x7f4ddc19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 -- 172.21.15.83:0/4068395420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc102ae0 msgr2=0x7f4ddc19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 -- 172.21.15.83:0/4068395420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108420 msgr2=0x7f4ddc19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108420 0x7f4ddc19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 -- 172.21.15.83:0/4068395420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc1a66a0 con 0x7f4ddc108d70 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.934+0000 7f4de0c02640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400cad0 tx=0x7f4dd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.935+0000 7f4dd97fa640 1 -- 172.21.15.83:0/4068395420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400ae20 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.935+0000 7f4dd97fa640 1 -- 172.21.15.83:0/4068395420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd4005020 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.935+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc1a6990 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.935+0000 7f4dd97fa640 1 -- 172.21.15.83:0/4068395420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400f6b0 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.935+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1a6ed0 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.937+0000 7f4de8bd9740 1 Processor -- start 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.937+0000 7f4dd97fa640 1 -- 172.21.15.83:0/4068395420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4dd40051d0 con 0x7f4ddc108d70 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.937+0000 7f4de8bd9740 1 -- start start 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c65396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6171390 0x5572c653a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5572c64afea0 con 0x5572c61691b0 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5572c649f2d0 con 0x5572c6171390 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572c64a30e0 con 0x5572c6192070 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48074/0 (socket says 172.21.15.83:48074) 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1846088737 learned_addr learned my addr 172.21.15.83:0/1846088737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6171390 0x5572c653a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c65396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4dd97fa640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 0x7f4dc40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1846088737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 msgr2=0x5572c65396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c65396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1846088737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6171390 msgr2=0x5572c653a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6171390 0x5572c653a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1846088737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c61fd650 con 0x5572c61691b0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4de0c02640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 0x7f4dc40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4dd97fa640 1 -- 172.21.15.83:0/4068395420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f4dd40e75b0 con 0x7f4ddc108d70 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.938+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009900 tx=0x7f4dcc032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350048d3775a014d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.939+0000 7f4dc27fc640 1 -- 172.21.15.83:0/1846088737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc040070 con 0x5572c61691b0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.939+0000 7f4dc27fc640 1 -- 172.21.15.83:0/1846088737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dcc005db0 con 0x5572c61691b0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.939+0000 7f4dc27fc640 1 -- 172.21.15.83:0/1846088737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc035040 con 0x5572c61691b0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.939+0000 7f4de0c02640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 0x7f4dc40b8c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40aa770 tx=0x7f4dd407a7e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- 172.21.15.83:0/1846088737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 msgr2=0x5572c6539be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009900 tx=0x7f4dcc032df0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- 172.21.15.83:0/1846088737 shutdown_connections 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6171390 0x5572c653a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c6539be0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/1846088737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c65396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- 172.21.15.83:0/1846088737 >> 172.21.15.83:0/1846088737 conn(0x5572c6186460 msgr2=0x5572c6537590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- 172.21.15.83:0/1846088737 shutdown_connections 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- 172.21.15.83:0/1846088737 wait complete. 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 -- start start 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.940+0000 7f4de8bd9740 1 Processor -- start 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c6544d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c6551170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5572c6300700 con 0x5572c6171390 2024-09-21T03:32:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5572c643a160 con 0x5572c61691b0 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572c6547e20 con 0x5572c6192070 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4de8bd9740 10 client.0 _ll_register_callbacks cb 0x5572c64539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48084/0 (socket says 172.21.15.83:48084) 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 -- 172.21.15.83:0/3408973802 learned_addr learned my addr 172.21.15.83:0/3408973802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c6544d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c6551170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 -- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 msgr2=0x5572c6551170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c6551170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 -- 172.21.15.83:0/3408973802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 msgr2=0x5572c6544d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c6544d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 -- 172.21.15.83:0/3408973802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c6467c90 con 0x5572c6171390 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006010 tx=0x7f4dcc0051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d3c18801413f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4dbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.941+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc04f030 con 0x5572c6171390 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dcc005db0 con 0x5572c6171390 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f4dcc031730 con 0x5572c6171390 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4dcc059050 con 0x5572c6171390 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 10 client.18021 fetch_fsmap learned FSMap version 58 2024-09-21T03:32:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5572c6553b50 con 0x5572c6171390 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dcc005db0 con 0x5572c6171390 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8386+0+0 (secure 0 0 0) 0x7f4dcc004c20 con 0x5572c6171390 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 10 client.18021 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 10 client.18021 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 20 client.18021 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c61509f0 0x5572c6556eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 4 client.18021 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.942+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c64b1df0 con 0x5572c61509f0 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.943+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c61509f0 0x5572c6556eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.943+0000 7f4dbb7fe640 10 client.18021 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:32:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.943+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c61509f0 0x5572c6556eb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=817d5f5a960b3007 server_cookie=c45c0043d6bdb65 in_seq=0 out_seq=0 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/3408973802 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572c64b1df0 con 0x5572c61509f0 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4dbb7fe640 10 client.18021 handle_command_reply: tid=0 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 client.18021 shutdown 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 msgr2=0x5572c65452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006010 tx=0x7f4dcc0051c0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 shutdown_connections 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c61509f0 0x5572c6556eb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6192070 0x5572c6551170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6171390 0x5572c65452b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3408973802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c6544d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 >> 172.21.15.83:0/3408973802 conn(0x5572c6186460 msgr2=0x5572c653c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4dbb7fe640 0 client.18021 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 shutdown_connections 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3408973802 wait complete. 2024-09-21T03:32:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.944+0000 7f4de8bd9740 20 client.18021 trim_cache size 0 max 16384 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.957+0000 7f4de8bd9740 1 Processor -- start 2024-09-21T03:32:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.957+0000 7f4de8bd9740 1 -- start start 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.957+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c653b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.957+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1ad3b0 con 0x5572c6192070 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1af010 con 0x5572c61691b0 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1adab0 con 0x5572c6171390 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48100/0 (socket says 172.21.15.83:48100) 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/3380841020 learned_addr learned my addr 172.21.15.83:0/3380841020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c653b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/3380841020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 msgr2=0x5572c65469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/3380841020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 msgr2=0x5572c653b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c653b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/3380841020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c6545e10 con 0x5572c6192070 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x5572c643d810 tx=0x7f4dd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cd4a35d054e2da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.958+0000 7f4dc1ffb640 1 -- 172.21.15.83:0/3380841020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0018030 con 0x5572c6192070 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4dc1ffb640 1 -- 172.21.15.83:0/3380841020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd00054d0 con 0x5572c6192070 2024-09-21T03:32:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4dc1ffb640 1 -- 172.21.15.83:0/3380841020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd001b040 con 0x5572c6192070 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3380841020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 msgr2=0x5572c6536bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x5572c643d810 tx=0x7f4dd0004730 comp rx=0 tx=0).stop 2024-09-21T03:32:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3380841020 shutdown_connections 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c6192070 0x5572c6536bc0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/3380841020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c61691b0 0x5572c653b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3380841020 >> 172.21.15.83:0/3380841020 conn(0x5572c6186460 msgr2=0x5572c6546290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3380841020 shutdown_connections 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.959+0000 7f4de8bd9740 1 -- 172.21.15.83:0/3380841020 wait complete. 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 -- start start 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 Processor -- start 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6192070 0x5572c6557e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1ae8b0 con 0x5572c61691b0 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1ae1e0 con 0x5572c6192070 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc1ab390 con 0x5572c6171390 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48102/0 (socket says 172.21.15.83:48102) 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/252543273 learned_addr learned my addr 172.21.15.83:0/252543273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.960+0000 7f4de8bd9740 10 client.0 _ll_register_callbacks cb 0x7f4dcc032d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6192070 0x5572c6557e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 msgr2=0x5572c65578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/252543273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6192070 msgr2=0x5572c6557e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6192070 0x5572c6557e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 -- 172.21.15.83:0/252543273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c6558360 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4ddb7fe640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00057b0 tx=0x7f4dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4697adc285c410bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4dbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0018070 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f4dd0012050 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd00054d0 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4dd002f040 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4de8bd9740 10 client.18027 fetch_fsmap learned FSMap version 58 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5572c65c1270 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.961+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd00054d0 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8386+0+0 (secure 0 0 0) 0x7f4dd0035020 con 0x5572c61691b0 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 10 client.18027 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T03:32:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 10 client.18027 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 20 client.18027 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c6125da0 0x5572c65c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 4 client.18027 mds_command: new command op to 24613 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 --> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572c64ad8a0 con 0x5572c6125da0 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c6125da0 0x5572c65c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c6125da0 0x5572c65c38d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73cfc888cdafd1a4 server_cookie=c0a03618aa0182a8 in_seq=0 out_seq=0 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.962+0000 7f4dbb7fe640 10 client.18027 ms_handle_connect on v2:172.21.15.193:6836/3456789844 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4dbb7fe640 1 -- 172.21.15.83:0/252543273 <== mds.3 v2:172.21.15.193:6836/3456789844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572c64ad8a0 con 0x5572c6125da0 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4dbb7fe640 10 client.18027 handle_command_reply: tid=0 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 client.18027 shutdown 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 msgr2=0x5572c655a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00057b0 tx=0x7f4dd00046c0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 shutdown_connections 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572c6192070 0x5572c6557e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:6836/3456789844,v1:172.21.15.193:6837/3456789844] conn(0x5572c6125da0 0x5572c65c38d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572c6171390 0x5572c65578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4dbb7fe640 0 client.18027 ms_handle_reset on v2:172.21.15.193:6836/3456789844 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 --2- 172.21.15.83:0/252543273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572c61691b0 0x5572c655a3c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 >> 172.21.15.83:0/252543273 conn(0x5572c6186460 msgr2=0x5572c653ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 shutdown_connections 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 1 -- 172.21.15.83:0/252543273 wait complete. 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.963+0000 7f4de8bd9740 20 client.18027 trim_cache size 0 max 16384 2024-09-21T03:32:43.970 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 msgr2=0x7f4dc40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 0x7f4dc40b8c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40aa770 tx=0x7f4dd407a7e0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 msgr2=0x7f4ddc19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400cad0 tx=0x7f4dd400cf90 comp rx=0 tx=0).stop 2024-09-21T03:32:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 shutdown_connections 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f4dc40b6700 0x7f4dc40b8c10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108d70 0x7f4ddc19be60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ddc108420 0x7f4ddc19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 --2- 172.21.15.83:0/4068395420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ddc102ae0 0x7f4ddc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 >> 172.21.15.83:0/4068395420 conn(0x7f4ddc079770 msgr2=0x7f4ddc106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 shutdown_connections 2024-09-21T03:32:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:43.964+0000 7f4de2665640 1 -- 172.21.15.83:0/4068395420 wait complete. 2024-09-21T03:32:44.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:45 smithi083 ceph-mon[31219]: pgmap v827: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-21T03:32:45.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:45 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:45 smithi142 ceph-mon[34512]: pgmap v827: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-21T03:32:45.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:45 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:45 smithi193 ceph-mon[35224]: pgmap v827: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-21T03:32:45.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:45 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 -- 172.21.15.83:0/100706891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109310 msgr2=0x7f7e70109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 --2- 172.21.15.83:0/100706891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109310 0x7f7e70109720 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 -- 172.21.15.83:0/100706891 shutdown_connections 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 --2- 172.21.15.83:0/100706891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e70109c60 0x7f7e70110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 --2- 172.21.15.83:0/100706891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109310 0x7f7e70109720 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 --2- 172.21.15.83:0/100706891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e701039d0 0x7f7e70103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 -- 172.21.15.83:0/100706891 >> 172.21.15.83:0/100706891 conn(0x7f7e700ff440 msgr2=0x7f7e70101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 -- 172.21.15.83:0/100706891 shutdown_connections 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 -- 172.21.15.83:0/100706891 wait complete. 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.543+0000 7f7e767a0640 1 Processor -- start 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 -- start start 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e701039d0 0x7f7e7019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e70109310 0x7f7e7019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70113920 con 0x7f7e70109c60 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701137a0 con 0x7f7e701039d0 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e767a0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70113aa0 con 0x7f7e70109310 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48124/0 (socket says 172.21.15.83:48124) 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 -- 172.21.15.83:0/44811960 learned_addr learned my addr 172.21.15.83:0/44811960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e7579e640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e701039d0 0x7f7e7019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e70109310 0x7f7e7019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 -- 172.21.15.83:0/44811960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e70109310 msgr2=0x7f7e7019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e70109310 0x7f7e7019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 -- 172.21.15.83:0/44811960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e701039d0 msgr2=0x7f7e7019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e701039d0 0x7f7e7019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 -- 172.21.15.83:0/44811960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a02a0 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.544+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.545+0000 7f7e667fc640 1 -- 172.21.15.83:0/44811960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68017070 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.545+0000 7f7e667fc640 1 -- 172.21.15.83:0/44811960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e68003e20 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.545+0000 7f7e667fc640 1 -- 172.21.15.83:0/44811960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68010460 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.545+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a0590 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.545+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a6cf0 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.547+0000 7f7e7df76740 1 Processor -- start 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e667fc640 1 -- 172.21.15.83:0/44811960 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7e68003970 con 0x7f7e70109c60 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 -- start start 2024-09-21T03:32:45.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ec94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d44ec0aea0 con 0x55d44e8cc390 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d44ebfa2d0 con 0x55d44e8ed070 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7df76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d44ebfe0e0 con 0x55d44e8c41b0 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e74f9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38846/0 (socket says 172.21.15.83:38846) 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e74f9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48140/0 (socket says 172.21.15.83:48140) 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e7579e640 1 -- 172.21.15.83:0/207307247 learned_addr learned my addr 172.21.15.83:0/207307247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.548+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ec94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 -- 172.21.15.83:0/207307247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 msgr2=0x55d44ec94460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ec94460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 -- 172.21.15.83:0/207307247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 msgr2=0x55d44ec94ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 -- 172.21.15.83:0/207307247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44e958650 con 0x55d44e8c41b0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e667fc640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 0x7f7e5c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 0x7f7e5c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e7579e640 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b760 tx=0x7f7e6c00bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b9b23cffe2c7a27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e667fc640 1 -- 172.21.15.83:0/44811960 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f7e680e73d0 con 0x7f7e70109c60 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e5b7fe640 1 -- 172.21.15.83:0/207307247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003dd0 con 0x55d44e8c41b0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e5b7fe640 1 -- 172.21.15.83:0/207307247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c015070 con 0x55d44e8c41b0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.549+0000 7f7e5b7fe640 1 -- 172.21.15.83:0/207307247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c004d20 con 0x55d44e8c41b0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- 172.21.15.83:0/207307247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 msgr2=0x55d44ec949a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b760 tx=0x7f7e6c00bc20 comp rx=0 tx=0).stop 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 0x7f7e5c0b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009230 tx=0x7f7e60040040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- 172.21.15.83:0/207307247 shutdown_connections 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8cc390 0x55d44ec94ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8c41b0 0x55d44ec949a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 --2- 172.21.15.83:0/207307247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ec94460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- 172.21.15.83:0/207307247 >> 172.21.15.83:0/207307247 conn(0x55d44e8e1460 msgr2=0x55d44ec92350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- 172.21.15.83:0/207307247 shutdown_connections 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- 172.21.15.83:0/207307247 wait complete. 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.550+0000 7f7e7df76740 1 -- start start 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 Processor -- start 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44eca0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ecabf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d44ea5b700 con 0x55d44e8c41b0 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d44eb95160 con 0x55d44e8ed070 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d44eca2c40 con 0x55d44e8cc390 2024-09-21T03:32:45.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7df76740 10 client.0 _ll_register_callbacks cb 0x55d44ebae9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48144/0 (socket says 172.21.15.83:48144) 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e7579e640 1 -- 172.21.15.83:0/3498612509 learned_addr learned my addr 172.21.15.83:0/3498612509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ecabf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.551+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44eca0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 -- 172.21.15.83:0/3498612509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 msgr2=0x55d44eca0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44eca0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 -- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 msgr2=0x55d44ecabf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ecabf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 -- 172.21.15.83:0/3498612509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44ebc2c90 con 0x55d44e8c41b0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44eca0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ecabf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7579e640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003e00 tx=0x7f7e6c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ca31ae41e22737 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:45.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c028060 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f7e6c011850 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c015070 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c00be70 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7df76740 10 client.18039 fetch_fsmap learned FSMap version 59 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d44ecae940 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.552+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c015070 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55d44e8c41b0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 10 client.18039 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 10 client.18039 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 20 client.18039 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e8ab9f0 0x55d44ecb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 4 client.18039 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d44ec0cdf0 con 0x55d44e8ab9f0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e8ab9f0 0x55d44ecb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e58ff9640 10 client.18039 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.553+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e8ab9f0 0x55d44ecb1d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a688a3be0f66f26 server_cookie=479e6a00c40a35b4 in_seq=0 out_seq=0 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e58ff9640 1 -- 172.21.15.83:0/3498612509 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d44ec0cdf0 con 0x55d44e8ab9f0 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e58ff9640 10 client.18039 handle_command_reply: tid=0 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 client.18039 shutdown 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 msgr2=0x55d44ec9fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003e00 tx=0x7f7e6c003e30 comp rx=0 tx=0).stop 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 shutdown_connections 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e8ab9f0 0x55d44ecb1d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8ed070 0x55d44ecabf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44eca0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e58ff9640 0 client.18039 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3498612509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ec9fb10 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.554+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 >> 172.21.15.83:0/3498612509 conn(0x55d44e8e1460 msgr2=0x55d44ec975f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.555+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 shutdown_connections 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.555+0000 7f7e7df76740 1 -- 172.21.15.83:0/3498612509 wait complete. 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.555+0000 7f7e7df76740 20 client.18039 trim_cache size 0 max 16384 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 Processor -- start 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 -- start start 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8c41b0 0x55d44ec95ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44ec91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701af430 con 0x55d44e8ed070 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701b10c0 con 0x55d44e8c41b0 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7df76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701afb30 con 0x55d44e8cc390 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48152/0 (socket says 172.21.15.83:48152) 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 -- 172.21.15.83:0/3885402079 learned_addr learned my addr 172.21.15.83:0/3885402079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e7579e640 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8c41b0 0x55d44ec95ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44ec91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 -- 172.21.15.83:0/3885402079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 msgr2=0x55d44ec91980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44ec91980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 -- 172.21.15.83:0/3885402079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8c41b0 msgr2=0x55d44ec95ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8c41b0 0x55d44ec95ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 -- 172.21.15.83:0/3885402079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44eca0dd0 con 0x55d44e8ed070 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.568+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e680e5170 tx=0x7f7e680accb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d2a4964f1df49c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e5affd640 1 -- 172.21.15.83:0/3885402079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68020070 con 0x55d44e8ed070 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e5affd640 1 -- 172.21.15.83:0/3885402079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e680acd80 con 0x55d44e8ed070 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e5affd640 1 -- 172.21.15.83:0/3885402079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680abf40 con 0x55d44e8ed070 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 -- 172.21.15.83:0/3885402079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 msgr2=0x55d44ec91ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e680e5170 tx=0x7f7e680accb0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 -- 172.21.15.83:0/3885402079 shutdown_connections 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8ed070 0x55d44ec91ec0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8cc390 0x55d44ec91980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 --2- 172.21.15.83:0/3885402079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8c41b0 0x55d44ec95ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.569+0000 7f7e7df76740 1 -- 172.21.15.83:0/3885402079 >> 172.21.15.83:0/3885402079 conn(0x55d44e8e1460 msgr2=0x55d44ec95490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- 172.21.15.83:0/3885402079 shutdown_connections 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- 172.21.15.83:0/3885402079 wait complete. 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- start start 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 Processor -- start 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 0x55d44ecb2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8ed070 0x55d44ecb2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701b0930 con 0x55d44e8c41b0 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701b0260 con 0x55d44e8cc390 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ad430 con 0x55d44e8ed070 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7df76740 10 client.0 _ll_register_callbacks cb 0x7f7e60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.570+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48160/0 (socket says 172.21.15.83:48160) 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 -- 172.21.15.83:0/1535889685 learned_addr learned my addr 172.21.15.83:0/1535889685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e75f9f640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8ed070 0x55d44ecb2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 0x55d44ecb2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 -- 172.21.15.83:0/1535889685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8ed070 msgr2=0x55d44ecb2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8ed070 0x55d44ecb2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 -- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 msgr2=0x55d44ecb2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 0x55d44ecb2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 -- 172.21.15.83:0/1535889685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44ecb32b0 con 0x55d44e8c41b0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 0x55d44ecb2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7579e640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c010 tx=0x7f7e6c003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7761736763a715 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c028040 con 0x55d44e8c41b0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f7e6c011650 con 0x55d44e8c41b0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e680acd80 con 0x55d44e8c41b0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c011870 con 0x55d44e8c41b0 2024-09-21T03:32:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7df76740 10 client.18045 fetch_fsmap learned FSMap version 59 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d44ed1c1e0 con 0x55d44e8c41b0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.571+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e680acd80 con 0x55d44e8c41b0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55d44e8c41b0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 10 client.18045 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 10 client.18045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 20 client.18045 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e880da0 0x55d44ed1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 4 client.18045 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 --> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d44ec088a0 con 0x55d44e880da0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.572+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e880da0 0x55d44ed1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e58ff9640 10 client.18045 ms_handle_connect on v2:172.21.15.142:6836/1691832808 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e74f9d640 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e880da0 0x55d44ed1ea00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b516762bd6f973a9 server_cookie=92689ecddb83b345 in_seq=0 out_seq=0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e58ff9640 1 -- 172.21.15.83:0/1535889685 <== mds.2 v2:172.21.15.142:6836/1691832808 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d44ec088a0 con 0x55d44e880da0 2024-09-21T03:32:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e58ff9640 10 client.18045 handle_command_reply: tid=0 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 client.18045 shutdown 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 msgr2=0x55d44ecb5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c010 tx=0x7f7e6c003ea0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 shutdown_connections 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:6836/1691832808,v1:172.21.15.142:6837/1691832808] conn(0x55d44e880da0 0x55d44ed1ea00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d44e8ed070 0x55d44ecb2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d44e8cc390 0x55d44ecb2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 --2- 172.21.15.83:0/1535889685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d44e8c41b0 0x55d44ecb5330 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 >> 172.21.15.83:0/1535889685 conn(0x55d44e8e1460 msgr2=0x55d44eca0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e58ff9640 0 client.18045 ms_handle_reset on v2:172.21.15.142:6836/1691832808 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 shutdown_connections 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 1 -- 172.21.15.83:0/1535889685 wait complete. 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.573+0000 7f7e7df76740 20 client.18045 trim_cache size 0 max 16384 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 msgr2=0x7f7e5c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 0x7f7e5c0b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009230 tx=0x7f7e60040040 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 msgr2=0x7f7e7019fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e6800b260 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 shutdown_connections 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f7e5c0b6700 0x7f7e5c0b8c10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e70109c60 0x7f7e7019fb70 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e70109310 0x7f7e7019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 --2- 172.21.15.83:0/44811960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e701039d0 0x7f7e7019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 >> 172.21.15.83:0/44811960 conn(0x7f7e700ff440 msgr2=0x7f7e70101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 shutdown_connections 2024-09-21T03:32:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:45.574+0000 7f7e767a0640 1 -- 172.21.15.83:0/44811960 wait complete. 2024-09-21T03:32:46.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:32:46.587 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 -- 172.21.15.83:0/3523811085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 msgr2=0x7f26b810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 --2- 172.21.15.83:0/3523811085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b810f540 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009ca0 tx=0x7f26b002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 -- 172.21.15.83:0/3523811085 shutdown_connections 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 --2- 172.21.15.83:0/3523811085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b810f540 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 --2- 172.21.15.83:0/3523811085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 0x7f26b81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 --2- 172.21.15.83:0/3523811085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 -- 172.21.15.83:0/3523811085 >> 172.21.15.83:0/3523811085 conn(0x7f26b8079740 msgr2=0x7f26b8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 -- 172.21.15.83:0/3523811085 shutdown_connections 2024-09-21T03:32:47.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.080+0000 7f26be66f640 1 -- 172.21.15.83:0/3523811085 wait complete. 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 Processor -- start 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 -- start start 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 0x7f26b8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8112980 con 0x7f26b8108d10 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8112800 con 0x7f26b81083c0 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26be66f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8112b00 con 0x7f26b8102a80 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48170/0 (socket says 172.21.15.83:48170) 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 -- 172.21.15.83:0/2080994663 learned_addr learned my addr 172.21.15.83:0/2080994663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26affff640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 0x7f26b8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 -- 172.21.15.83:0/2080994663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 msgr2=0x7f26b8199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 -- 172.21.15.83:0/2080994663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 msgr2=0x7f26b8199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 0x7f26b8199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26bcc0c640 1 -- 172.21.15.83:0/2080994663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b81a87b0 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.081+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f26b0005cc0 tx=0x7f26b002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26b5ffb640 1 -- 172.21.15.83:0/2080994663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003bf0 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b81a8a40 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26b5ffb640 1 -- 172.21.15.83:0/2080994663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b0003d90 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26b5ffb640 1 -- 172.21.15.83:0/2080994663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b002fbc0 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.082+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b81a8f80 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.084+0000 7f26c4be3740 1 Processor -- start 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26b5ffb640 1 -- 172.21.15.83:0/2080994663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f26b002fd60 con 0x7f26b8108d10 2024-09-21T03:32:47.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 -- start start 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd489712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd48971800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd488e7ea0 con 0x55cd485a9390 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd488d72d0 con 0x55cd485a11b0 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26c4be3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd488db0e0 con 0x55cd485ca070 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48176/0 (socket says 172.21.15.83:48176) 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 -- 172.21.15.83:0/1914674639 learned_addr learned my addr 172.21.15.83:0/1914674639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26b7fff640 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd48971800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd489712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 -- 172.21.15.83:0/1914674639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 msgr2=0x55cd489712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd489712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 -- 172.21.15.83:0/1914674639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 msgr2=0x55cd48971800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd48971800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26affff640 1 -- 172.21.15.83:0/1914674639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd48635650 con 0x55cd485a9390 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.085+0000 7f26b5ffb640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 0x7f26a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 0x7f26a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26affff640 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f26a000bdc0 tx=0x7f26a000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624eb892b370fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26b5ffb640 1 -- 172.21.15.83:0/2080994663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26b0108c50 con 0x7f26b8108d10 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26ae7fc640 1 -- 172.21.15.83:0/1914674639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00120d0 con 0x55cd485a9390 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26ae7fc640 1 -- 172.21.15.83:0/1914674639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a0005020 con 0x55cd485a9390 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.086+0000 7f26ae7fc640 1 -- 172.21.15.83:0/1914674639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00049a0 con 0x55cd485a9390 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- 172.21.15.83:0/1914674639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 msgr2=0x55cd48971d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 0x7f26a40b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55cd4896ff40 tx=0x7f26a800b420 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f26a000bdc0 tx=0x7f26a000b650 comp rx=0 tx=0).stop 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- 172.21.15.83:0/1914674639 shutdown_connections 2024-09-21T03:32:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48971d40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd48971800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 --2- 172.21.15.83:0/1914674639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd489712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- 172.21.15.83:0/1914674639 >> 172.21.15.83:0/1914674639 conn(0x55cd485be460 msgr2=0x55cd4896f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- 172.21.15.83:0/1914674639 shutdown_connections 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- 172.21.15.83:0/1914674639 wait complete. 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 -- start start 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.087+0000 7f26c4be3740 1 Processor -- start 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd4897c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd48988dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd48738700 con 0x55cd485a9390 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd48872160 con 0x55cd485a11b0 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd4897faa0 con 0x55cd485ca070 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26c4be3740 10 client.0 _ll_register_callbacks cb 0x55cd4888b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:47.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48178/0 (socket says 172.21.15.83:48178) 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 -- 172.21.15.83:0/2600638524 learned_addr learned my addr 172.21.15.83:0/2600638524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd4897c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd48988dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 -- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 msgr2=0x55cd48988dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd48988dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 -- 172.21.15.83:0/2600638524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 msgr2=0x55cd4897c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd4897c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 -- 172.21.15.83:0/2600638524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4889fc90 con 0x55cd485a9390 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd48988dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26affff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004540 tx=0x7f26a0004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c581a59e71de7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.088+0000 7f26977fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00120d0 con 0x55cd485a9390 2024-09-21T03:32:47.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26a00053e0 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a0005020 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f26a00055e0 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 10 client.18057 fetch_fsmap learned FSMap version 59 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cd4898b810 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a0005020 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f26a0003920 con 0x55cd485a9390 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 10 client.18057 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 10 client.18057 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 20 client.18057 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485889f0 0x55cd4898ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 4 client.18057 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.089+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd488e9df0 con 0x55cd485889f0 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.090+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485889f0 0x55cd4898ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.090+0000 7f26977fe640 10 client.18057 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.090+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485889f0 0x55cd4898ece0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd83e7a0e7abbad6 server_cookie=503ecd72d54a9527 in_seq=0 out_seq=0 2024-09-21T03:32:47.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26977fe640 1 -- 172.21.15.83:0/2600638524 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd488e9df0 con 0x55cd485889f0 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26977fe640 10 client.18057 handle_command_reply: tid=0 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 client.18057 shutdown 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 msgr2=0x55cd4897cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004540 tx=0x7f26a0004330 comp rx=0 tx=0).stop 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 shutdown_connections 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485889f0 0x55cd4898ece0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485ca070 0x55cd48988dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd4897cef0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26977fe640 0 client.18057 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2600638524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485a11b0 0x55cd4897c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 >> 172.21.15.83:0/2600638524 conn(0x55cd485be460 msgr2=0x55cd48974450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 shutdown_connections 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 1 -- 172.21.15.83:0/2600638524 wait complete. 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.091+0000 7f26c4be3740 20 client.18057 trim_cache size 0 max 16384 2024-09-21T03:32:47.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.104+0000 7f26c4be3740 1 Processor -- start 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.104+0000 7f26c4be3740 1 -- start start 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48972cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48988cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81b17f0 con 0x55cd485a9390 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81b3480 con 0x55cd485889f0 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26c4be3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81b1ef0 con 0x55cd485a11b0 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.83:38914/0 (socket says 172.21.15.83:38914) 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 -- 172.21.15.83:0/2035025906 learned_addr learned my addr 172.21.15.83:0/2035025906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26bcc0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48988cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26b7fff640 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48972cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 -- 172.21.15.83:0/2035025906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 msgr2=0x55cd48972cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48972cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 -- 172.21.15.83:0/2035025906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 msgr2=0x55cd48988cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48988cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26affff640 1 -- 172.21.15.83:0/2035025906 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4896a5d0 con 0x55cd485a11b0 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.105+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48988cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.106+0000 7f26affff640 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26a0006010 tx=0x7f26a0005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12548f4191c041e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.106+0000 7f26adffb640 1 -- 172.21.15.83:0/2035025906 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0032040 con 0x55cd485a11b0 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.106+0000 7f26adffb640 1 -- 172.21.15.83:0/2035025906 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000bbd0 con 0x55cd485a11b0 2024-09-21T03:32:47.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.106+0000 7f26adffb640 1 -- 172.21.15.83:0/2035025906 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0004950 con 0x55cd485a11b0 2024-09-21T03:32:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 -- 172.21.15.83:0/2035025906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 msgr2=0x55cd4897e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26a0006010 tx=0x7f26a0005b90 comp rx=0 tx=0).stop 2024-09-21T03:32:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 -- 172.21.15.83:0/2035025906 shutdown_connections 2024-09-21T03:32:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a9390 0x55cd48988cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a11b0 0x55cd4897e5e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 --2- 172.21.15.83:0/2035025906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48972cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 -- 172.21.15.83:0/2035025906 >> 172.21.15.83:0/2035025906 conn(0x55cd485be460 msgr2=0x55cd4897ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 -- 172.21.15.83:0/2035025906 shutdown_connections 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.107+0000 7f26c4be3740 1 -- 172.21.15.83:0/2035025906 wait complete. 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 -- start start 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 Processor -- start 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48980f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a9390 0x55cd489819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81b2cf0 con 0x55cd485a11b0 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81b2620 con 0x55cd485889f0 2024-09-21T03:32:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.108+0000 7f26c4be3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81af7f0 con 0x55cd485a9390 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26c4be3740 10 client.0 _ll_register_callbacks cb 0x7f26a800b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48188/0 (socket says 172.21.15.83:48188) 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26affff640 1 -- 172.21.15.83:0/191409842 learned_addr learned my addr 172.21.15.83:0/191409842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48980f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26bcc0c640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a9390 0x55cd489819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 -- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a9390 msgr2=0x55cd489819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a9390 0x55cd489819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 -- 172.21.15.83:0/191409842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 msgr2=0x55cd48981460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 -- 172.21.15.83:0/191409842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd489f8810 con 0x55cd485889f0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26affff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26b7fff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48980f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55cd4898f350 tx=0x7f26a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f0e0e3d0b918d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.109+0000 7f26977fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.112+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8028600 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.112+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26a80287a0 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.112+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a000bbd0 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.116+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f26a80289a0 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.116+0000 7f26c4be3740 10 client.27575 fetch_fsmap learned FSMap version 60 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.116+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55cd489f9af0 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.116+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000bbd0 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 60) ==== 8394+0+0 (secure 0 0 0) 0x7f26a8005020 con 0x55cd485889f0 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 10 client.27575 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 10 client.27575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 20 client.27575 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485ca070 0x55cd489fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 4 client.27575 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 --> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd488e58a0 con 0x55cd485ca070 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26affff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485ca070 0x55cd489fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26977fe640 10 client.27575 ms_handle_connect on v2:172.21.15.193:6838/357057598 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.117+0000 7f26affff640 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485ca070 0x55cd489fc280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b1638fdb396d0dc server_cookie=b28c0465c2bc3bd2 in_seq=0 out_seq=0 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26977fe640 1 -- 172.21.15.83:0/191409842 <== mds.1 v2:172.21.15.193:6838/357057598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd488e58a0 con 0x55cd485ca070 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26977fe640 10 client.27575 handle_command_reply: tid=0 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 client.27575 shutdown 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 msgr2=0x55cd48980f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48980f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55cd4898f350 tx=0x7f26a80046c0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 shutdown_connections 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:6838/357057598,v1:172.21.15.193:6839/357057598] conn(0x55cd485ca070 0x55cd489fc280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd485a9390 0x55cd489819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd485a11b0 0x55cd48981460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26977fe640 0 client.27575 ms_handle_reset on v2:172.21.15.193:6838/357057598 2024-09-21T03:32:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 --2- 172.21.15.83:0/191409842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd485889f0 0x55cd48980f20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 >> 172.21.15.83:0/191409842 conn(0x55cd485be460 msgr2=0x55cd48974b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 shutdown_connections 2024-09-21T03:32:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 1 -- 172.21.15.83:0/191409842 wait complete. 2024-09-21T03:32:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.118+0000 7f26c4be3740 20 client.27575 trim_cache size 0 max 16384 2024-09-21T03:32:47.126 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 msgr2=0x7f26a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 0x7f26a40b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55cd4896ff40 tx=0x7f26a800b420 comp rx=0 tx=0).stop 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 msgr2=0x7f26b8199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f26b0005cc0 tx=0x7f26b002f380 comp rx=0 tx=0).stop 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 shutdown_connections 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26a40b6680 0x7f26a40b8b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b8108d10 0x7f26b8199b80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b81083c0 0x7f26b8199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 --2- 172.21.15.83:0/2080994663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26b8102a80 0x7f26b8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:47.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 >> 172.21.15.83:0/2080994663 conn(0x7f26b8079740 msgr2=0x7f26b81044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:47.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 shutdown_connections 2024-09-21T03:32:47.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:47.119+0000 7f26be66f640 1 -- 172.21.15.83:0/2080994663 wait complete. 2024-09-21T03:32:47.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:47 smithi193 ceph-mon[35224]: pgmap v828: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 411 KiB/s wr, 55 op/s 2024-09-21T03:32:47.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:47 smithi083 ceph-mon[31219]: pgmap v828: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 411 KiB/s wr, 55 op/s 2024-09-21T03:32:47.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:47 smithi142 ceph-mon[34512]: pgmap v828: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 411 KiB/s wr, 55 op/s 2024-09-21T03:32:47.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:32:48.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:48.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:48 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:48.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:48 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:48.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:48 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 -- 172.21.15.83:0/3397381945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec108cf0 msgr2=0x7f26ec10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 --2- 172.21.15.83:0/3397381945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec108cf0 0x7f26ec10f520 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ca0 tx=0x7f26e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 -- 172.21.15.83:0/3397381945 shutdown_connections 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 --2- 172.21.15.83:0/3397381945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec108cf0 0x7f26ec10f520 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 --2- 172.21.15.83:0/3397381945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec1083a0 0x7f26ec1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 --2- 172.21.15.83:0/3397381945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 0x7f26ec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 -- 172.21.15.83:0/3397381945 >> 172.21.15.83:0/3397381945 conn(0x7f26ec079730 msgr2=0x7f26ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 -- 172.21.15.83:0/3397381945 shutdown_connections 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.732+0000 7f26f4534640 1 -- 172.21.15.83:0/3397381945 wait complete. 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 Processor -- start 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 -- start start 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 0x7f26ec19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec108cf0 0x7f26ec19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec113a80 con 0x7f26ec1083a0 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec113900 con 0x7f26ec102a60 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f4534640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec113c00 con 0x7f26ec108cf0 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48200/0 (socket says 172.21.15.83:48200) 2024-09-21T03:32:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 -- 172.21.15.83:0/1894244646 learned_addr learned my addr 172.21.15.83:0/1894244646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f22d0640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 0x7f26ec19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec108cf0 0x7f26ec19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 -- 172.21.15.83:0/1894244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec108cf0 msgr2=0x7f26ec19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec108cf0 0x7f26ec19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 -- 172.21.15.83:0/1894244646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 msgr2=0x7f26ec19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 0x7f26ec19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.733+0000 7f26f1acf640 1 -- 172.21.15.83:0/1894244646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec1a0380 con 0x7f26ec1083a0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26f1acf640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f26e000e9f0 tx=0x7f26e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26eb7fe640 1 -- 172.21.15.83:0/1894244646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e000ccc0 con 0x7f26ec1083a0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26eb7fe640 1 -- 172.21.15.83:0/1894244646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e00054d0 con 0x7f26ec1083a0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec1a0670 con 0x7f26ec1083a0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26eb7fe640 1 -- 172.21.15.83:0/1894244646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0010430 con 0x7f26ec1083a0 2024-09-21T03:32:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.734+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec1a6dd0 con 0x7f26ec1083a0 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.736+0000 7f26faaa8740 1 Processor -- start 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 -- start start 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26eb7fe640 1 -- 172.21.15.83:0/1894244646 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f26e00046c0 con 0x7f26ec1083a0 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa443be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa444120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558caa3b9ea0 con 0x558caa09c070 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558caa3a92d0 con 0x558caa07b390 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26faaa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558caa3ad0e0 con 0x558caa0731b0 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48216/0 (socket says 172.21.15.83:48216) 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 -- 172.21.15.83:0/2439951656 learned_addr learned my addr 172.21.15.83:0/2439951656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f22d0640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa443be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f1acf640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa444120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 -- 172.21.15.83:0/2439951656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 msgr2=0x558caa443be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa443be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 -- 172.21.15.83:0/2439951656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 msgr2=0x558caa444120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa444120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f2ad1640 1 -- 172.21.15.83:0/2439951656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caa107650 con 0x558caa09c070 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.737+0000 7f26f22d0640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa443be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f26e4005cc0 tx=0x7f26e4009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdb62be9af4b4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26eb7fe640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 0x7f26d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26e8ff9640 1 -- 172.21.15.83:0/2439951656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4031cf0 con 0x558caa09c070 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26f22d0640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 0x7f26d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26e8ff9640 1 -- 172.21.15.83:0/2439951656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e4036070 con 0x558caa09c070 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26e8ff9640 1 -- 172.21.15.83:0/2439951656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004040 con 0x558caa09c070 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.738+0000 7f26eb7fe640 1 -- 172.21.15.83:0/1894244646 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26e00e7a60 con 0x7f26ec1083a0 2024-09-21T03:32:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- 172.21.15.83:0/2439951656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 msgr2=0x558caa4436a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26f22d0640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 0x7f26d40b8c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x558caa442320 tx=0x7f26dc00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f26e4005cc0 tx=0x7f26e4009440 comp rx=0 tx=0).stop 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- 172.21.15.83:0/2439951656 shutdown_connections 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa444120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa443be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2439951656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa4436a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- 172.21.15.83:0/2439951656 >> 172.21.15.83:0/2439951656 conn(0x558caa090460 msgr2=0x558caa441590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- 172.21.15.83:0/2439951656 shutdown_connections 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- 172.21.15.83:0/2439951656 wait complete. 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 -- start start 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.739+0000 7f26faaa8740 1 Processor -- start 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa44edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa44f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558caa20a700 con 0x558caa09c070 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558caa344160 con 0x558caa07b390 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558caa451e20 con 0x558caa0731b0 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26faaa8740 10 client.0 _ll_register_callbacks cb 0x558caa35d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f1acf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa44f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48228/0 (socket says 172.21.15.83:48228) 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 -- 172.21.15.83:0/3695667098 learned_addr learned my addr 172.21.15.83:0/3695667098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f22d0640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa44edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 -- 172.21.15.83:0/3695667098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 msgr2=0x558caa44edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa44edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 -- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 msgr2=0x558caa44f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa44f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 -- 172.21.15.83:0/3695667098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caa371c90 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f1acf640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa44f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ca0 tx=0x7f26e40039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96662ced4288eaf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.740+0000 7f26d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4049040 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26e4031cf0 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e4036070 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f26e405b050 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 10 client.18075 fetch_fsmap learned FSMap version 60 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558caa461f70 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e4036070 con 0x558caa09c070 2024-09-21T03:32:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 8394+0+0 (secure 0 0 0) 0x7f26e404c070 con 0x558caa09c070 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 10 client.18075 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 10 client.18075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 20 client.18075 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa05a9f0 0x558caa4652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 4 client.18075 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.741+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558caa3bbdf0 con 0x558caa05a9f0 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.742+0000 7f26f22d0640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa05a9f0 0x558caa4652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.742+0000 7f26d1ffb640 10 client.18075 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.742+0000 7f26f22d0640 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa05a9f0 0x558caa4652d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886a5ec5dd4049d server_cookie=5d37aa0b6e1c700a in_seq=0 out_seq=0 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26d1ffb640 1 -- 172.21.15.83:0/3695667098 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558caa3bbdf0 con 0x558caa05a9f0 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26d1ffb640 10 client.18075 handle_command_reply: tid=0 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 client.18075 shutdown 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 msgr2=0x558caa45f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009ca0 tx=0x7f26e40039f0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 shutdown_connections 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa05a9f0 0x558caa4652d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa09c070 0x558caa45f550 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa07b390 0x558caa44f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26d1ffb640 0 client.18075 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 --2- 172.21.15.83:0/3695667098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa44edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 >> 172.21.15.83:0/3695667098 conn(0x558caa090460 msgr2=0x558caa446880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 shutdown_connections 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 1 -- 172.21.15.83:0/3695667098 wait complete. 2024-09-21T03:32:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.743+0000 7f26faaa8740 20 client.18075 trim_cache size 0 max 16384 2024-09-21T03:32:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.756+0000 7f26faaa8740 1 Processor -- start 2024-09-21T03:32:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.756+0000 7f26faaa8740 1 -- start start 2024-09-21T03:32:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.756+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa450c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.756+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa45f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26faaa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1ad360 con 0x558caa07b390 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26faaa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1af020 con 0x558caa05a9f0 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26faaa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1ada90 con 0x558caa0731b0 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48238/0 (socket says 172.21.15.83:48238) 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 -- 172.21.15.83:0/44315875 learned_addr learned my addr 172.21.15.83:0/44315875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f1acf640 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa45f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f22d0640 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa450c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 -- 172.21.15.83:0/44315875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 msgr2=0x558caa45f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa45f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 -- 172.21.15.83:0/44315875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 msgr2=0x558caa450c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa450c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 -- 172.21.15.83:0/44315875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caa450070 con 0x558caa07b390 2024-09-21T03:32:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26f2ad1640 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26e4002230 tx=0x7f26e402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cacbdf58f24deb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26d3fff640 1 -- 172.21.15.83:0/44315875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4003a60 con 0x558caa07b390 2024-09-21T03:32:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26d3fff640 1 -- 172.21.15.83:0/44315875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e4036070 con 0x558caa07b390 2024-09-21T03:32:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.757+0000 7f26d3fff640 1 -- 172.21.15.83:0/44315875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e40316b0 con 0x558caa07b390 2024-09-21T03:32:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- 172.21.15.83:0/44315875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 msgr2=0x558caa44dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26e4002230 tx=0x7f26e402bfd0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- 172.21.15.83:0/44315875 shutdown_connections 2024-09-21T03:32:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa07b390 0x558caa44dd40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa0731b0 0x558caa45f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 --2- 172.21.15.83:0/44315875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa450c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- 172.21.15.83:0/44315875 >> 172.21.15.83:0/44315875 conn(0x558caa090460 msgr2=0x558caa44d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- 172.21.15.83:0/44315875 shutdown_connections 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- 172.21.15.83:0/44315875 wait complete. 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.758+0000 7f26faaa8740 1 -- start start 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 Processor -- start 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa468c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa07b390 0x558caa466800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1ae890 con 0x558caa0731b0 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1ae1c0 con 0x558caa05a9f0 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1ab360 con 0x558caa07b390 2024-09-21T03:32:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26faaa8740 10 client.0 _ll_register_callbacks cb 0x7f26e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48252/0 (socket says 172.21.15.83:48252) 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 -- 172.21.15.83:0/2544468837 learned_addr learned my addr 172.21.15.83:0/2544468837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f22d0640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa468c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 -- 172.21.15.83:0/2544468837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa07b390 msgr2=0x558caa466800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa07b390 0x558caa466800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 -- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 msgr2=0x558caa468c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa468c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.759+0000 7f26f1acf640 1 -- 172.21.15.83:0/2544468837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caa466f90 con 0x558caa0731b0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26f1acf640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ad600 tx=0x7f26e0004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5820e949d0e5ae63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00a97a0 con 0x558caa0731b0 2024-09-21T03:32:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f26e00a9460 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e4036070 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f26e00ad070 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 10 client.18081 fetch_fsmap learned FSMap version 60 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x558caa4cfb60 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e4036070 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 8394+0+0 (secure 0 0 0) 0x7f26e00ac5d0 con 0x558caa0731b0 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 10 client.18081 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 20 client.18081 populate_metadata read hostname 'smithi083' 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa09c070 0x558caa4d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 4 client.18081 mds_command: new command op to 24647 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.760+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 --> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558caa3b78a0 con 0x558caa09c070 2024-09-21T03:32:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26f22d0640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa09c070 0x558caa4d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26d1ffb640 10 client.18081 ms_handle_connect on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26f22d0640 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa09c070 0x558caa4d23e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e5eeeb7dbcfe78 server_cookie=d0863a6b76b54bf5 in_seq=0 out_seq=0 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26d1ffb640 1 -- 172.21.15.83:0/2544468837 <== mds.0 v2:172.21.15.142:6838/3152549242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558caa3b78a0 con 0x558caa09c070 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26d1ffb640 10 client.18081 handle_command_reply: tid=0 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 client.18081 shutdown 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 msgr2=0x558caa466160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ad600 tx=0x7f26e0004a60 comp rx=0 tx=0).stop 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 shutdown_connections 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:6838/3152549242,v1:172.21.15.142:6839/3152549242] conn(0x558caa09c070 0x558caa4d23e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558caa07b390 0x558caa466800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558caa0731b0 0x558caa466160 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 --2- 172.21.15.83:0/2544468837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558caa05a9f0 0x558caa468c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 >> 172.21.15.83:0/2544468837 conn(0x558caa090460 msgr2=0x558caa4509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.761+0000 7f26d1ffb640 0 client.18081 ms_handle_reset on v2:172.21.15.142:6838/3152549242 2024-09-21T03:32:48.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 shutdown_connections 2024-09-21T03:32:48.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26faaa8740 1 -- 172.21.15.83:0/2544468837 wait complete. 2024-09-21T03:32:48.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26faaa8740 20 client.18081 trim_cache size 0 max 16384 2024-09-21T03:32:48.769 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-09-21T03:32:48.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 msgr2=0x7f26d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 0x7f26d40b8c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x558caa442320 tx=0x7f26dc00b390 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 msgr2=0x7f26ec19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f26e000e9f0 tx=0x7f26e000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 shutdown_connections 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f26d40b6700 0x7f26d40b8c10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26ec108cf0 0x7f26ec19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26ec1083a0 0x7f26ec19a9b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 --2- 172.21.15.83:0/1894244646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec102a60 0x7f26ec19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 >> 172.21.15.83:0/1894244646 conn(0x7f26ec079730 msgr2=0x7f26ec105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 shutdown_connections 2024-09-21T03:32:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:48.762+0000 7f26f4534640 1 -- 172.21.15.83:0/1894244646 wait complete. 2024-09-21T03:32:49.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:49 smithi142 ceph-mon[34512]: pgmap v829: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 123 KiB/s wr, 23 op/s 2024-09-21T03:32:49.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:49 smithi083 ceph-mon[31219]: pgmap v829: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 123 KiB/s wr, 23 op/s 2024-09-21T03:32:49.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:49 smithi193 ceph-mon[35224]: pgmap v829: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 123 KiB/s wr, 23 op/s 2024-09-21T03:32:49.618 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T03:32:49.619 INFO:tasks.fwd_scrub:done joining 2024-09-21T03:32:49.619 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:32:49.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:32:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 -- 172.21.15.83:0/3675965649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92881030b0 msgr2=0x7f92881034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 --2- 172.21.15.83:0/3675965649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92881030b0 0x7f92881034c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9284009900 tx=0x7f928402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 -- 172.21.15.83:0/3675965649 shutdown_connections 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 --2- 172.21.15.83:0/3675965649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9288105a00 0x7f928810f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 --2- 172.21.15.83:0/3675965649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881050b0 0x7f92881054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 --2- 172.21.15.83:0/3675965649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92881030b0 0x7f92881034c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 -- 172.21.15.83:0/3675965649 >> 172.21.15.83:0/3675965649 conn(0x7f9288079cb0 msgr2=0x7f928807a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 -- 172.21.15.83:0/3675965649 shutdown_connections 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.745+0000 7f928f630640 1 -- 172.21.15.83:0/3675965649 wait complete. 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 Processor -- start 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 -- start start 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 0x7f928819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 0x7f928819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112960 con 0x7f9288105a00 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92881127e0 con 0x7f92881050b0 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928f630640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112ae0 con 0x7f92881030b0 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928dbcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928dbcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53406/0 (socket says 172.21.15.83:53406) 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.746+0000 7f928dbcd640 1 -- 172.21.15.83:0/550699825 learned_addr learned my addr 172.21.15.83:0/550699825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928d3cc640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 0x7f928819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928cbcb640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 0x7f928819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 -- 172.21.15.83:0/550699825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 msgr2=0x7f928819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 0x7f928819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 -- 172.21.15.83:0/550699825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 msgr2=0x7f928819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 0x7f928819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 -- 172.21.15.83:0/550699825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92881a6540 con 0x7f9288105a00 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928d3cc640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 0x7f928819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928cbcb640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 0x7f928819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928dbcd640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f928000b970 tx=0x7f928000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280019070 con 0x7f9288105a00 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9280014440 con 0x7f9288105a00 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92881a6830 con 0x7f9288105a00 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92800075d0 con 0x7f9288105a00 2024-09-21T03:32:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.747+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881a6db0 con 0x7f9288105a00 2024-09-21T03:32:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.748+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92881034c0 con 0x7f9288105a00 2024-09-21T03:32:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.749+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f928001c070 con 0x7f9288105a00 2024-09-21T03:32:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.750+0000 7f927a7fc640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 0x7f92740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.750+0000 7f928d3cc640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 0x7f92740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.750+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f92800e7100 con 0x7f9288105a00 2024-09-21T03:32:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.751+0000 7f928d3cc640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 0x7f92740b8b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9284000c00 tx=0x7f9284031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.753+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92800ad6a0 con 0x7f9288105a00 2024-09-21T03:32:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.934+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f928802e640 con 0x7f9288105a00 2024-09-21T03:32:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.935+0000 7f927a7fc640 1 -- 172.21.15.83:0/550699825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23038 (secure 0 0 0) 0x7f92800b1a80 con 0x7f9288105a00 2024-09-21T03:32:49.937 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:49.937 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:32:14.065661+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:32:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 msgr2=0x7f92740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 0x7f92740b8b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9284000c00 tx=0x7f9284031400 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 msgr2=0x7f928819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f928000b970 tx=0x7f928000be30 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 shutdown_connections 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f92740b6680 0x7f92740b8b90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9288105a00 0x7f928819bcb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92881050b0 0x7f928819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 --2- 172.21.15.83:0/550699825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881030b0 0x7f928819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 >> 172.21.15.83:0/550699825 conn(0x7f9288079cb0 msgr2=0x7f9288106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 shutdown_connections 2024-09-21T03:32:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:49.937+0000 7f928f630640 1 -- 172.21.15.83:0/550699825 wait complete. 2024-09-21T03:32:49.951 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:09:57.414337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:14:33.119862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:14:35.204610+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:32:49.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:32:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 -- 172.21.15.83:0/1759784482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb120108c10 msgr2=0x7fb12010f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 --2- 172.21.15.83:0/1759784482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb120108c10 0x7fb12010f440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ca0 tx=0x7fb11802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 -- 172.21.15.83:0/1759784482 shutdown_connections 2024-09-21T03:32:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 --2- 172.21.15.83:0/1759784482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb120108c10 0x7fb12010f440 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 --2- 172.21.15.83:0/1759784482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1201082c0 0x7fb1201086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 --2- 172.21.15.83:0/1759784482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb120102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 -- 172.21.15.83:0/1759784482 >> 172.21.15.83:0/1759784482 conn(0x7fb1200fe3b0 msgr2=0x7fb120100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 -- 172.21.15.83:0/1759784482 shutdown_connections 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.062+0000 7fb126f19640 1 -- 172.21.15.83:0/1759784482 wait complete. 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 Processor -- start 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 -- start start 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb12019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 0x7fb12019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1201128d0 con 0x7fb1201082c0 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb120112750 con 0x7fb120108c10 2024-09-21T03:32:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb126f19640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb120112a50 con 0x7fb120102990 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53432/0 (socket says 172.21.15.83:53432) 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 -- 172.21.15.83:0/4207728842 learned_addr learned my addr 172.21.15.83:0/4207728842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb1254b6640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 0x7fb12019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb124cb5640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb12019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 -- 172.21.15.83:0/4207728842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 msgr2=0x7fb12019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb12019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 -- 172.21.15.83:0/4207728842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 msgr2=0x7fb12019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 0x7fb12019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 -- 172.21.15.83:0/4207728842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1201a6440 con 0x7fb1201082c0 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb124cb5640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb12019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb1254b6640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 0x7fb12019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.063+0000 7fb11ffff640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c970 tx=0x7fb10c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.064+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c007bb0 con 0x7fb1201082c0 2024-09-21T03:32:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.064+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10c007d50 con 0x7fb1201082c0 2024-09-21T03:32:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.064+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201a6730 con 0x7fb1201082c0 2024-09-21T03:32:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.064+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00f450 con 0x7fb1201082c0 2024-09-21T03:32:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.064+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201a6c80 con 0x7fb1201082c0 2024-09-21T03:32:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.065+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb120102da0 con 0x7fb1201082c0 2024-09-21T03:32:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.066+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb10c004eb0 con 0x7fb1201082c0 2024-09-21T03:32:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.067+0000 7fb11dffb640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 0x7fb1100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.067+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7fb10c0e74d0 con 0x7fb1201082c0 2024-09-21T03:32:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.067+0000 7fb124cb5640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 0x7fb1100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.067+0000 7fb124cb5640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 0x7fb1100b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb1140058c0 tx=0x7fb114005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.071+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb10c014030 con 0x7fb1201082c0 2024-09-21T03:32:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.250+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb12002e640 con 0x7fb1201082c0 2024-09-21T03:32:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.250+0000 7fb11dffb640 1 -- 172.21.15.83:0/4207728842 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fb10c0ada70 con 0x7fb1201082c0 2024-09-21T03:32:50.252 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-09-21T03:32:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.252+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 msgr2=0x7fb1100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.252+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 0x7fb1100b8b90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb1140058c0 tx=0x7fb114005d50 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.252+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 msgr2=0x7fb12019b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.252+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c970 tx=0x7fb10c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 shutdown_connections 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fb1100b6680 0x7fb1100b8b90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb120108c10 0x7fb12019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1201082c0 0x7fb12019b6c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 --2- 172.21.15.83:0/4207728842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb120102990 0x7fb12019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 >> 172.21.15.83:0/4207728842 conn(0x7fb1200fe3b0 msgr2=0x7fb120107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 shutdown_connections 2024-09-21T03:32:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.253+0000 7fb126f19640 1 -- 172.21.15.83:0/4207728842 wait complete. 2024-09-21T03:32:50.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:50 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/550699825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:32:50.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:32:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/1129534927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d48102990 msgr2=0x7f8d48102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/1129534927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d48102990 0x7f8d48102da0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009900 tx=0x7f8d3802a410 comp rx=0 tx=0).stop 2024-09-21T03:32:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/1129534927 shutdown_connections 2024-09-21T03:32:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/1129534927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48108c10 0x7f8d4810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/1129534927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d481082c0 0x7f8d481086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/1129534927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d48102990 0x7f8d48102da0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/1129534927 >> 172.21.15.83:0/1129534927 conn(0x7f8d480fe3b0 msgr2=0x7f8d48100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/1129534927 shutdown_connections 2024-09-21T03:32:50.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.376+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/1129534927 wait complete. 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 Processor -- start 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 -- start start 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48102990 0x7f8d48199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d48108c10 0x7f8d4819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481128d0 con 0x7f8d481082c0 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48112750 con 0x7f8d48102990 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4ce4d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d48112a50 con 0x7f8d48108c10 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d45d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d45d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53448/0 (socket says 172.21.15.83:53448) 2024-09-21T03:32:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d45d9b640 1 -- 172.21.15.83:0/2008257336 learned_addr learned my addr 172.21.15.83:0/2008257336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d46d9d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d48108c10 0x7f8d4819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.377+0000 7f8d4659c640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48102990 0x7f8d48199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 -- 172.21.15.83:0/2008257336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d48108c10 msgr2=0x7f8d4819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d48108c10 0x7f8d4819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 -- 172.21.15.83:0/2008257336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48102990 msgr2=0x7f8d48199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48102990 0x7f8d48199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 -- 172.21.15.83:0/2008257336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4819f1b0 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d45d9b640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c007920 tx=0x7f8d3c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c019070 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c005020 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4819f4a0 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c004a20 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.378+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4806a720 con 0x7f8d481082c0 2024-09-21T03:32:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.379+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d48103c00 con 0x7f8d481082c0 2024-09-21T03:32:50.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.380+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8d3c01c070 con 0x7f8d481082c0 2024-09-21T03:32:50.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.381+0000 7f8d377fe640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 0x7f8d300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.381+0000 7f8d4659c640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 0x7f8d300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.381+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7f8d3c0e8680 con 0x7f8d481082c0 2024-09-21T03:32:50.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.382+0000 7f8d4659c640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 0x7f8d300b8b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031440 tx=0x7f8d38005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.384+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d3c0b3000 con 0x7f8d481082c0 2024-09-21T03:32:50.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:50 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/550699825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:32:50.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:50 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/550699825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:32:50.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.564+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d4806b140 con 0x7f8d481082c0 2024-09-21T03:32:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.565+0000 7f8d377fe640 1 -- 172.21.15.83:0/2008257336 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f8d3c0b7f40 con 0x7f8d481082c0 2024-09-21T03:32:50.568 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-09-21T03:32:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 msgr2=0x7f8d300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 0x7f8d300b8b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031440 tx=0x7f8d38005e80 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 msgr2=0x7f8d481998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c007920 tx=0x7f8d3c007de0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 shutdown_connections 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f8d300b6680 0x7f8d300b8b90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d48108c10 0x7f8d4819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d481082c0 0x7f8d481998b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 --2- 172.21.15.83:0/2008257336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d48102990 0x7f8d48199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 >> 172.21.15.83:0/2008257336 conn(0x7f8d480fe3b0 msgr2=0x7f8d481007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 shutdown_connections 2024-09-21T03:32:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.567+0000 7f8d4ce4d640 1 -- 172.21.15.83:0/2008257336 wait complete. 2024-09-21T03:32:50.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:32:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 -- 172.21.15.83:0/2493730122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664108c10 msgr2=0x7ff66410f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 --2- 172.21.15.83:0/2493730122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664108c10 0x7ff66410f440 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009ca0 tx=0x7ff65c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 -- 172.21.15.83:0/2493730122 shutdown_connections 2024-09-21T03:32:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 --2- 172.21.15.83:0/2493730122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664108c10 0x7ff66410f440 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 --2- 172.21.15.83:0/2493730122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 0x7ff6641086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 --2- 172.21.15.83:0/2493730122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664102990 0x7ff664102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 -- 172.21.15.83:0/2493730122 >> 172.21.15.83:0/2493730122 conn(0x7ff6640fe3b0 msgr2=0x7ff664100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 -- 172.21.15.83:0/2493730122 shutdown_connections 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 -- 172.21.15.83:0/2493730122 wait complete. 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.694+0000 7ff66984d640 1 Processor -- start 2024-09-21T03:32:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 -- start start 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 0x7ff66419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 0x7ff66419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641128d0 con 0x7ff664102990 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff664112750 con 0x7ff6641082c0 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff66984d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff664112a50 con 0x7ff664108c10 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53466/0 (socket says 172.21.15.83:53466) 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 -- 172.21.15.83:0/2221472091 learned_addr learned my addr 172.21.15.83:0/2221472091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff65a7fc640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 0x7ff66419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff6637fe640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 0x7ff66419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 -- 172.21.15.83:0/2221472091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 msgr2=0x7ff66419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 0x7ff66419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 -- 172.21.15.83:0/2221472091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 msgr2=0x7ff66419b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 0x7ff66419b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff662ffd640 1 -- 172.21.15.83:0/2221472091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6641a6460 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.695+0000 7ff6637fe640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 0x7ff66419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff662ffd640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9f0 tx=0x7ff65400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400cd80 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a6750 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6540054d0 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654010690 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.696+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a6ca0 con 0x7ff664102990 2024-09-21T03:32:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.697+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664102da0 con 0x7ff664102990 2024-09-21T03:32:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.698+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff654004e30 con 0x7ff664102990 2024-09-21T03:32:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.699+0000 7ff66884b640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 0x7ff6500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.699+0000 7ff65a7fc640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 0x7ff6500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.699+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7ff6540e7d60 con 0x7ff664102990 2024-09-21T03:32:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.700+0000 7ff65a7fc640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 0x7ff6500b8b90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff664101ab0 tx=0x7ff64c00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:50.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.702+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff654056710 con 0x7ff664102990 2024-09-21T03:32:50.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.881+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff66406ab50 con 0x7ff664102990 2024-09-21T03:32:50.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.881+0000 7ff66884b640 1 -- 172.21.15.83:0/2221472091 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7ff6540ae300 con 0x7ff664102990 2024-09-21T03:32:50.885 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 msgr2=0x7ff6500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 0x7ff6500b8b90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff664101ab0 tx=0x7ff64c00b390 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 msgr2=0x7ff66419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9f0 tx=0x7ff65400edf0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 shutdown_connections 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7ff6500b6680 0x7ff6500b8b90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff664108c10 0x7ff66419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6641082c0 0x7ff66419b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 --2- 172.21.15.83:0/2221472091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff664102990 0x7ff66419b1a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 >> 172.21.15.83:0/2221472091 conn(0x7ff6640fe3b0 msgr2=0x7ff664105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 shutdown_connections 2024-09-21T03:32:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:50.884+0000 7ff66984d640 1 -- 172.21.15.83:0/2221472091 wait complete. 2024-09-21T03:32:50.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.003+0000 7fd93880e640 1 -- 172.21.15.83:0/2109110650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 msgr2=0x7fd93410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.003+0000 7fd93880e640 1 --2- 172.21.15.83:0/2109110650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93410d770 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009900 tx=0x7fd91c02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 -- 172.21.15.83:0/2109110650 shutdown_connections 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 --2- 172.21.15.83:0/2109110650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd93410dd40 0x7fd934110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 --2- 172.21.15.83:0/2109110650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93410d770 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 --2- 172.21.15.83:0/2109110650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9341048e0 0x7fd934104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 -- 172.21.15.83:0/2109110650 >> 172.21.15.83:0/2109110650 conn(0x7fd9340fe350 msgr2=0x7fd9341007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 -- 172.21.15.83:0/2109110650 shutdown_connections 2024-09-21T03:32:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 -- 172.21.15.83:0/2109110650 wait complete. 2024-09-21T03:32:51.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.004+0000 7fd93880e640 1 Processor -- start 2024-09-21T03:32:51.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 -- start start 2024-09-21T03:32:51.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9341048e0 0x7fd93419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd93410dd40 0x7fd93419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341139a0 con 0x7fd934105230 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113820 con 0x7fd93410dd40 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93880e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113b20 con 0x7fd9341048e0 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53488/0 (socket says 172.21.15.83:53488) 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 -- 172.21.15.83:0/1941639029 learned_addr learned my addr 172.21.15.83:0/1941639029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd932d9d640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd93410dd40 0x7fd93419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd93259c640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9341048e0 0x7fd93419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 -- 172.21.15.83:0/1941639029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9341048e0 msgr2=0x7fd93419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9341048e0 0x7fd93419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 -- 172.21.15.83:0/1941639029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd93410dd40 msgr2=0x7fd93419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd93410dd40 0x7fd93419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 -- 172.21.15.83:0/1941639029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9341a0330 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.005+0000 7fd931d9b640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd91c006010 tx=0x7fd91c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.006+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c03f030 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.006+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91c003aa0 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.006+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9341a05c0 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.006+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c035070 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.006+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a6db0 con 0x7fd934105230 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.007+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93406c030 con 0x7fd934105230 2024-09-21T03:32:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.008+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd91c042030 con 0x7fd934105230 2024-09-21T03:32:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.009+0000 7fd92b7fe640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 0x7fd9200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.009+0000 7fd93259c640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 0x7fd9200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.009+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8441+0+0 (secure 0 0 0) 0x7fd91c107ff0 con 0x7fd934105230 2024-09-21T03:32:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.010+0000 7fd93259c640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 0x7fd9200b8b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd924005fd0 tx=0x7fd924005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.012+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd91c0ce590 con 0x7fd934105230 2024-09-21T03:32:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.241+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd9341a6f80 con 0x7fd934105230 2024-09-21T03:32:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.249+0000 7fd92b7fe640 1 -- 172.21.15.83:0/1941639029 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834051 (secure 0 0 0) 0x7fd91c0d2970 con 0x7fd934105230 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 24, 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 14217, 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:51.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1298689131 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1298689131 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/1298689131", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-09-21T03:07:59.080322+0000", 2024-09-21T03:32:51.262 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24130, 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.265 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.267 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:32:51.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.275 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.278 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-09-21T03:32:51.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:51.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:32:51.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:32:51.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.286 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.288 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.297 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-09-21T03:32:51.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-09-21T03:32:51.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-09-21T03:32:51.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.315 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nginx": { 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nginx", 2024-09-21T03:32:51.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nginx container image", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:32:51.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba_metrics": { 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:32:51.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.329 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-09-21T03:32:51.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.335 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-09-21T03:32:51.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:32:51.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-09-21T03:32:51.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-09-21T03:32:51.346 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:32:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_interval": { 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-09-21T03:32:51.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-09-21T03:32:51.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.363 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-09-21T03:32:51.371 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:32:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:32:51.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.378 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:32:51.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:32:51.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-09-21T03:32:51.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:32:51.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:32:51.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.390 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.391 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.392 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.393 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.400 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:32:51.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:32:51.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.411 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:32:51.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.413 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-09-21T03:32:51.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-09-21T03:32:51.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "sso_oauth2": { 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sso_oauth2", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-09-21T03:32:51.434 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.441 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:32:51.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.449 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.467 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.474 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.476 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.482 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.485 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.490 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:32:51.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.497 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:32:51.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.506 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:32:51.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.511 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-09-21T03:32:51.514 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.515 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.521 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.543 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.557 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-09-21T03:32:51.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.575 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-09-21T03:32:51.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:32:51.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-09-21T03:32:51.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:32:51.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-09-21T03:32:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.601 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max_requests": { 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_requests", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.610 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.617 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-09-21T03:32:51.621 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.622 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-09-21T03:32:51.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-09-21T03:32:51.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-09-21T03:32:51.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smb", 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "internal_store_backend": { 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "internal_store_backend", 2024-09-21T03:32:51.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "update_orchestration": { 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "update_orchestration", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:32:51.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-09-21T03:32:51.655 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.656 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.677 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.680 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-09-21T03:32:51.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-09-21T03:32:51.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.690 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.701 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.704 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.705 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.706 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.708 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-09-21T03:32:51.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-09-21T03:32:51.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-09-21T03:32:51.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:32:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.718 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.721 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-09-21T03:32:51.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-09-21T03:32:51.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-09-21T03:32:51.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24161, 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.729 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.733 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:32:51.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.743 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:51.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:32:51.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:32:51.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:32:51.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:32:51.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.752 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-09-21T03:32:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-09-21T03:32:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-09-21T03:32:51.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-09-21T03:32:51.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-09-21T03:32:51.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:32:51.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:32:51.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.785 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nginx": { 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nginx", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nginx container image", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-09-21T03:32:51.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.791 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-09-21T03:32:51.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba_metrics": { 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:32:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-09-21T03:32:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-09-21T03:32:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:32:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-09-21T03:32:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-09-21T03:32:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:32:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-09-21T03:32:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-09-21T03:32:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-09-21T03:32:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-09-21T03:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-09-21T03:32:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:32:51.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:32:51.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_interval": { 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.830 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-09-21T03:32:51.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.836 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.838 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.846 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.848 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.852 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.859 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.860 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:32:51.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:32:51.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:32:51.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:32:51.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:32:51.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:32:51.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.871 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.872 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.880 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.881 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.882 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:32:51.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:32:51.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:32:51.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:32:51.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.890 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.892 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.893 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.895 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "sso_oauth2": { 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sso_oauth2", 2024-09-21T03:32:51.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:51.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.907 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.908 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.915 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.922 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.925 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.926 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.928 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-09-21T03:32:51.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-09-21T03:32:51.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-09-21T03:32:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-09-21T03:32:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:32:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:32:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.965 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.966 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:32:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:32:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-09-21T03:32:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-09-21T03:32:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:32:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-09-21T03:32:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max_requests": { 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_requests", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-09-21T03:32:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-09-21T03:32:52.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-09-21T03:32:52.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-09-21T03:32:52.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-09-21T03:32:52.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-09-21T03:32:52.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smb", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "internal_store_backend": { 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "internal_store_backend", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.115 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.116 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "update_orchestration": { 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "update_orchestration", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-09-21T03:32:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.124 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.125 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.131 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-09-21T03:32:52.134 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.135 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.138 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.141 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.144 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.146 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-09-21T03:32:52.149 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-09-21T03:32:52.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-09-21T03:32:52.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-09-21T03:32:52.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-09-21T03:32:52.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.162 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.163 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.165 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.174 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.181 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-09-21T03:32:52.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.184 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:32:52.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-09-21T03:32:52.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.190 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-09-21T03:32:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "restful", 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_schedule" 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-09-21T03:32:52.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-09-21T03:32:52.218 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:32:52.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:52.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:32:52.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:32:52.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:32:52.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.226 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.227 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-09-21T03:32:52.233 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-09-21T03:32:52.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-09-21T03:32:52.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.254 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-09-21T03:32:52.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-09-21T03:32:52.256 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.257 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.258 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nginx": { 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nginx", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nginx container image", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:32:52.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:32:52.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba_metrics": { 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:32:52.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:32:52.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-09-21T03:32:52.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:32:52.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-09-21T03:32:52.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.284 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:32:52.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-09-21T03:32:52.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.302 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_keepalive_interval": { 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:32:52.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:32:52.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-09-21T03:32:52.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.313 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:32:52.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.324 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.325 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.326 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.327 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.340 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:32:52.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.344 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.359 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.363 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-09-21T03:32:52.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-09-21T03:32:52.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-09-21T03:32:52.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-09-21T03:32:52.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.370 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.372 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "sso_oauth2": { 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sso_oauth2", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-09-21T03:32:52.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-09-21T03:32:52.385 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-09-21T03:32:52.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.387 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.391 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-09-21T03:32:52.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-09-21T03:32:52.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-09-21T03:32:52.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-09-21T03:32:52.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-09-21T03:32:52.399 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.400 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-09-21T03:32:52.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.416 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-09-21T03:32:52.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.423 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.434 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.442 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.449 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:32:52.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-09-21T03:32:52.463 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.464 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.480 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-09-21T03:32:52.482 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.487 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.490 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.491 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.493 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-09-21T03:32:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.497 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.504 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.506 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.509 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.514 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.515 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.517 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.519 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.520 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-09-21T03:32:52.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:32:52.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-09-21T03:32:52.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:52.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-09-21T03:32:52.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.532 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.537 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.538 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-09-21T03:32:52.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.543 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.546 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max_requests": { 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_requests", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.548 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.552 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.559 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.562 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.569 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-09-21T03:32:52.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.572 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.573 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smb", 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "internal_store_backend": { 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "internal_store_backend", 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.589 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "update_orchestration": { 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "update_orchestration", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.593 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.596 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.603 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.612 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-09-21T03:32:52.615 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.618 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.637 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-09-21T03:32:52.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.649 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.653 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.655 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-09-21T03:32:52.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-09-21T03:32:52.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-09-21T03:32:52.662 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-09-21T03:32:52.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-09-21T03:32:52.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.665 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.666 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-09-21T03:32:52.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "sqlite3_killpoint": { 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "dev", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.673 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-09-21T03:32:52.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-09-21T03:32:52.675 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-09-21T03:32:52.676 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-09-21T03:32:52.677 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-09-21T03:32:52.678 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-09-21T03:32:52.679 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 5, 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3367477869 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 488129215 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1478872012 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 883570598 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 181500415 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-21T03:32:52.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2132424413 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-21T03:32:52.684 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 msgr2=0x7fd9200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 0x7fd9200b8b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd924005fd0 tx=0x7fd924005850 comp rx=0 tx=0).stop 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 msgr2=0x7fd93419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd91c006010 tx=0x7fd91c004060 comp rx=0 tx=0).stop 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 shutdown_connections 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fd9200b6680 0x7fd9200b8b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd93410dd40 0x7fd93419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934105230 0x7fd93419a9a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 --2- 172.21.15.83:0/1941639029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9341048e0 0x7fd93419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 >> 172.21.15.83:0/1941639029 conn(0x7fd9340fe350 msgr2=0x7fd934107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 shutdown_connections 2024-09-21T03:32:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:32:51.255+0000 7fd93880e640 1 -- 172.21.15.83:0/1941639029 wait complete. 2024-09-21T03:32:52.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:51 smithi083 ceph-mon[31219]: pgmap v830: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-09-21T03:32:52.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:51 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4207728842' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:51 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2008257336' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:51 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/2221472091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.731 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:32:52.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:32:52.732 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:52 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1941639029' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:32:52.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:51 smithi193 ceph-mon[35224]: pgmap v830: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:51 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4207728842' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:51 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2008257336' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:51 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/2221472091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:52 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1941639029' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:51 smithi142 ceph-mon[34512]: pgmap v830: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:51 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4207728842' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:51 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2008257336' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:51 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/2221472091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:32:52.733 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:52 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1941639029' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:32:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:53.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:53 smithi083 ceph-mon[31219]: pgmap v831: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 103 KiB/s wr, 18 op/s 2024-09-21T03:32:53.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:53 smithi142 ceph-mon[34512]: pgmap v831: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 103 KiB/s wr, 18 op/s 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2423, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2423, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3952337, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 769691, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.301056807, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.106242530, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.065636760, 2024-09-21T03:32:53.537 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.111955479, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2422, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.149167941, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000061588 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:32:53.538 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1014, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1238, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 249924, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 899292, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.130091518, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.058064712, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.022975404, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.038655943, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1236, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.057342521, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000046393 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.539 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 577408 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 363, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 324, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1120508, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 187822, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-09-21T03:32:53.540 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.054865851, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.028296991, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.034942943, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000019878, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 323, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.015991429, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000049509 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1120240, 2024-09-21T03:32:53.541 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 187504 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5749, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.188313744, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000032755 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.542 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 546, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 28, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 518, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 506, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-09-21T03:32:53.544 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 16, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-09-21T03:32:53.545 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:32:53.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-09-21T03:32:53.547 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:32:53.548 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_succeeded": 0, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_write_failed": 0 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-09-21T03:32:53.549 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 1905, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 3, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-09-21T03:32:53.550 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 7434108, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 7433534, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 1905 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 425, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 738, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 313, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 25, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 35, 2024-09-21T03:32:53.551 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 10, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 546, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 870, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 324, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 120188, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 320684032 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-09-21T03:32:53.552 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-09-21T03:32:53.553 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.555 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.556 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-09-21T03:32:53.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-09-21T03:32:53.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.559 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-09-21T03:32:53.560 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.561 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.562 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:32:53.563 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 3, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-09-21T03:32:53.564 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2053, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:32:53.565 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:32:53.566 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:32:53.567 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 208408, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 545, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-21T03:32:53.568 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 7968, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 176, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 1485933, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 21969, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.569 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3432, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3432, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3432, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 5.167877822, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001505791 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-09-21T03:32:53.570 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3432, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3433 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3432, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3432, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2089, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1341, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-09-21T03:32:53.571 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-09-21T03:32:53.572 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-09-21T03:32:53.573 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 97, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 22, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 5, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 7, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 2, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-09-21T03:32:53.574 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-09-21T03:32:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3796, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 4959753, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3796, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3796, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 4959753, 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:53.576 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:53.577 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3432, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3239809, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3432, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3239809, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-09-21T03:32:53.578 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3432, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3432, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3432, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3432, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-21T03:32:53.580 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x5575cae5d450": { 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:53.581 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "trackedop": { 2024-09-21T03:32:53.582 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-09-21T03:32:53.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:53.583 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:53.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:53 smithi193 ceph-mon[35224]: pgmap v831: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 103 KiB/s wr, 18 op/s 2024-09-21T03:32:54.248 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T03:32:54.248 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T03:32:54.248 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-21T03:32:54.248 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:32:54.248 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:32:54.248 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:32:54.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:32:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:54.994 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001175834, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000249255, 2024-09-21T03:32:54.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:32:54.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 232, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3264, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 201664, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.022263653, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.016449089, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.003696895, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 231, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.018929292, 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000081944 2024-09-21T03:32:54.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 201664 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 307, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 302, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 862608, 2024-09-21T03:32:54.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 186160, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.053839296, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.024560405, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.036492551, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 301, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.018508235, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000061489 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:54.999 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 862608, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 186160 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:32:55.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-21T03:32:55.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-21T03:32:55.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 29, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-09-21T03:32:55.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 968, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:55.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 309, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 819214, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 309, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 309, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 819214, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:55.005 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:55.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:55 smithi142 ceph-mon[34512]: pgmap v832: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 118 KiB/s wr, 21 op/s 2024-09-21T03:32:55.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:55 smithi193 ceph-mon[35224]: pgmap v832: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 118 KiB/s wr, 21 op/s 2024-09-21T03:32:55.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:55 smithi083 ceph-mon[31219]: pgmap v832: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 118 KiB/s wr, 21 op/s 2024-09-21T03:32:55.658 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:32:55.658 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:32:55.658 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T03:32:55.658 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:32:55.658 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:32:55.658 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:32:55.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:32:55.907 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1083, 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1085, 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 363833, 2024-09-21T03:32:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 344395, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.148788067, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.068101356, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.032039687, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.037870487, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1083, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.074236240, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000068546 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2272, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2497, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3382382, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1209467, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-09-21T03:32:56.493 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.232316953, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.091676262, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.048246283, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.078657784, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2495, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.148783188, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000059632 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 487232 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 370, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 329, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1124804, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 189557, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-09-21T03:32:56.495 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.058869861, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.027764766, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.041550837, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000061715, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 329, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.024338359, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000073976 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1123456, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 188592 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5669, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.161057838, 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000028410 2024-09-21T03:32:56.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-09-21T03:32:56.498 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 647, 2024-09-21T03:32:56.499 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 20, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 627, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 618, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 25, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-09-21T03:32:56.500 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-09-21T03:32:56.501 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:32:56.502 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-09-21T03:32:56.503 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:32:56.504 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_started": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_succeeded": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "uninline_write_failed": 0 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-09-21T03:32:56.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 1659, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 3, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 7000484, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 7000295, 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-09-21T03:32:56.506 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 1659 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 526, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 654, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 128, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 34, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 41, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 7, 2024-09-21T03:32:56.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 647, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 752, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 105, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 123276, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 320679936 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-09-21T03:32:56.508 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.509 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.510 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.511 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.512 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-09-21T03:32:56.514 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-09-21T03:32:56.519 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2055, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:32:56.520 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:32:56.521 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:32:56.522 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 220229, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 617, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 1, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 7968, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 176, 2024-09-21T03:32:56.523 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 1777599, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 23794, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3357, 2024-09-21T03:32:56.524 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3357, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3357, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 7.282827909, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002169445 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3357, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3358 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.525 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3357, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3357, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2043, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1312, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-09-21T03:32:56.526 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:32:56.527 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-09-21T03:32:56.528 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 97, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 22, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 6, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 8, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 2, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-09-21T03:32:56.529 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-09-21T03:32:56.530 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3721, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 4514744, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3721, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3721, 2024-09-21T03:32:56.531 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 4514744, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:56.532 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3357, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2806185, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3357, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2806185, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-09-21T03:32:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3357, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3357, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3357, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3357, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:56.535 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55ee64579520": { 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:56.536 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "trackedop": { 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:56.537 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:57.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:57 smithi083 ceph-mon[31219]: pgmap v833: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T03:32:57.253 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T03:32:57.254 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T03:32:57.254 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T03:32:57.254 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:32:57.254 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:32:57.254 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:32:57.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:32:57.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:57 smithi142 ceph-mon[34512]: pgmap v833: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T03:32:57.498 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:57.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:57 smithi193 ceph-mon[35224]: pgmap v833: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-21T03:32:57.992 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001213076, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000361183, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 312, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 305, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 865824, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 187248, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.055745552, 2024-09-21T03:32:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.024625395, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.039803155, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 305, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.018594784, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000060966 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 865824, 2024-09-21T03:32:57.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 187248 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 232, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 201568, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.024000435, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.019750027, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001767177, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:32:57.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 230, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.019337914, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000084077 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 201568 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-09-21T03:32:57.997 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:32:57.998 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:32:57.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-09-21T03:32:58.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148721, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:32:58.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 968, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-09-21T03:32:58.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 310, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 819344, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 310, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 310, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 819344, 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-09-21T03:32:58.003 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-09-21T03:32:58.004 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-09-21T03:32:58.004 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-09-21T03:32:58.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:58.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-21T03:32:58.004 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-21T03:32:58.698 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:32:58.698 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:32:58.699 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T03:32:58.699 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:32:58.699 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:32:58.699 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:32:58.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:32:59.337 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:32:59.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:32:59 smithi142 ceph-mon[34512]: pgmap v834: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 17 op/s 2024-09-21T03:32:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:32:59 smithi193 ceph-mon[35224]: pgmap v834: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 17 op/s 2024-09-21T03:32:59.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:32:59 smithi083 ceph-mon[31219]: pgmap v834: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 17 op/s 2024-09-21T03:33:00.201 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 8663, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 10238, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4385499, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 11059857, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 157, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.755037787, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.325687557, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.395116620, 2024-09-21T03:33:00.202 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.077193720, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10240, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.857087097, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000181356 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11297, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.002105643, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000186 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1160288, 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 223376 2024-09-21T03:33:00.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 6804, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 8060, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 2545737, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 19042491, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.592563161, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.284356156, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.192753431, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.103387921, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8058, 2024-09-21T03:33:00.204 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.905512155, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000236474 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5326, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001172217, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000220 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 498960 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 13643, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 16821, 2024-09-21T03:33:00.205 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4836507, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 21188332, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 20, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 8, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 1.303459176, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.610340038, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.655471260, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.191437196, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16817, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.555281943, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000092482 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.206 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21881, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004279960, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000195 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 560496, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 188112 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:00.207 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6420, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.066063882, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001100633 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 233, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.010987151, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000047155 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-21T03:33:00.208 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 3187, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 2946, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2946, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 395.175622824, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.134139722 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 241, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 6, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 56, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 5, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 463, 2024-09-21T03:33:00.209 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 37, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 376, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 50, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 410, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 1660, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 269, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 269, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 66, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 11115, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 10470, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 217, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 2, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:00.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 5, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 1687, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 40, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 1663, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 19, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 828, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 7656, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 8, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 1001, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 1303538210, 2024-09-21T03:33:00.211 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 7656, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 637, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 19, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 618, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 7019, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 15, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 1304, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 2020, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 10, 2024-09-21T03:33:00.212 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 2748, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 1164, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 63, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 274 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 7, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 57, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 4, 2024-09-21T03:33:00.213 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 67, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 61, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 4, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:00.214 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 19, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 40, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 5, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_started": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_write_failed": 0 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 4088, 2024-09-21T03:33:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 4088, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 10, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 10, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-21T03:33:00.216 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 35771145, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1950, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 27.738496127, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.014224869 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 447, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 2055, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 1608, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 126, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 142, 2024-09-21T03:33:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 16, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 469, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 2401, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 1932, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 269, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 1671, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 1402, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 137404, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 352202752 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 3461, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 133, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 3187, 2024-09-21T03:33:00.218 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 305, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 275, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1023, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5.074751446, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004960656 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1007, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 13.958558680, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013861527 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.011384602, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003794867 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 277, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 374.968458109, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 1.353676744 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:00.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000198785, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000198785 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:00.222 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.002695703, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000168481 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 37, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.304674443, 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008234444 2024-09-21T03:33:00.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.121094800, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007568425 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 372, 2024-09-21T03:33:00.224 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.078532389, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000211108 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 78, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.150928520, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001934981 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 49, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.225637764, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004604852 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:00.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016070728, 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008035364 2024-09-21T03:33:00.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.052242528, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005224252 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11, 2024-09-21T03:33:00.227 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.080904716, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007354974 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-21T03:33:00.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 9, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.023076085, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002564009 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.229 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 35, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.106413526, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003040386 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-21T03:33:00.230 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 198, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 99, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 968, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:00.231 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:00.232 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:00.233 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 384276, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 504, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:00.234 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 1911845, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 24236, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-21T03:33:00.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 10659, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 32246963, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 37, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 10659, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10659, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 69.824753431, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006550779 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.236 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10659, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 10931 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 10659, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 7731, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 2928, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 11, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 44, 2024-09-21T03:33:00.237 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 1975, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 94, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 4, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 595, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 7675, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 22, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:00.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 10, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-21T03:33:00.239 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 97, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-21T03:33:00.240 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 22, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 390, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 18, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 93 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-21T03:33:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 13, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 75, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 61, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:00.242 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 29106, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 9063506, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 29106, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 29106, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 9063506, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 10659, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 32253290, 2024-09-21T03:33:00.244 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 10659, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 32253290, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:00.245 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 10659, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 10659, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 10659, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 10659, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:00.246 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 61, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 6577, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 61, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 25, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6577, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x56555a29d380": { 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4088, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 31576841, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4088, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:00.248 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1950, 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 31576841, 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 28 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:00.249 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-21T03:33:01.264 INFO:tasks.check_counter:Daemon mds.b mds.exported=40 2024-09-21T03:33:01.264 INFO:tasks.check_counter:Daemon mds.b mds.imported=19 2024-09-21T03:33:01.264 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T03:33:01.264 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-21T03:33:01.264 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-09-21T03:33:01.264 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:33:01.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:33:01.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:01 smithi142 ceph-mon[34512]: pgmap v835: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T03:33:01.528 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:33:01.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:01 smithi193 ceph-mon[35224]: pgmap v835: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T03:33:01.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:01 smithi083 ceph-mon[31219]: pgmap v835: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2013, 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1974, 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 10017442, 2024-09-21T03:33:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 714142, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.234336483, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.104862699, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.078272876, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.048504425, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1973, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.114057615, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000057809 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1122368, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 191664 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1282, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1511, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 15185475, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 988690, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.133577015, 2024-09-21T03:33:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.050743874, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.038634711, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.035624605, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1509, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.081028705, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000053696 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 581584 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 777, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 777, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 8604153, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 246269, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.079415279, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.027803154, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.025279636, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.021180169, 2024-09-21T03:33:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 776, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.037169378, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000047898 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-21T03:33:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6238, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.157180138, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000025197 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 550, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 8, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 542, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 538, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-21T03:33:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 64, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-21T03:33:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 1001, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 1303538210, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-21T03:33:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_started": 0, 2024-09-21T03:33:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_write_failed": 0 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 4094, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-21T03:33:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 10, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 35779943, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 35775780, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 4094 2024-09-21T03:33:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 460, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 2074, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 1614, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 118, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 134, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 16, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 550, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 2178, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 1628, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-21T03:33:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 122492, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 323256320 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-21T03:33:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-21T03:33:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-21T03:33:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2283, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 229793, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 675, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 1838961, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 19062, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-21T03:33:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 3696, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 3696, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3696, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 15.169738103, 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004104366 2024-09-21T03:33:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3696, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3697 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 3696, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 3696, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2153, 2024-09-21T03:33:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1540, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2024-09-21T03:33:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-21T03:33:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-21T03:33:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 97, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 22, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 7, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 7, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-09-21T03:33:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4068, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 33419164, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4068, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4068, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 33419164, 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3696, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 31655998, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3696, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 31655998, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3696, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 3696, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3696, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 3696, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55e5fbbe7380": { 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-21T03:33:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:02.159 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-21T03:33:03.275 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:33:03.276 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:33:03.276 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-21T03:33:03.276 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:03.276 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:03.276 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:33:03.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:33:03.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:03 smithi142 ceph-mon[34512]: pgmap v836: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 75 KiB/s wr, 16 op/s 2024-09-21T03:33:03.537 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:33:03.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:03 smithi193 ceph-mon[35224]: pgmap v836: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 75 KiB/s wr, 16 op/s 2024-09-21T03:33:03.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:03 smithi083 ceph-mon[31219]: pgmap v836: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 75 KiB/s wr, 16 op/s 2024-09-21T03:33:04.126 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-21T03:33:04.126 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 5466, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 4370, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 3811182, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 6403244, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.474015487, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.231643768, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.243358063, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.035809159, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4373, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.258751381, 2024-09-21T03:33:04.127 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000059170 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6669, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001791823, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000268 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1147280, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 219360 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1920, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2144, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 735373, 2024-09-21T03:33:04.128 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 2082473, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 13, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.170844625, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.080284396, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.056445346, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.030453002, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2142, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.145121263, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000067750 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1485, 2024-09-21T03:33:04.129 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000238673, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000160 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 584016 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 5023, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 6595, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1713198, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 3623651, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 51, 2024-09-21T03:33:04.130 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 11, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.434251552, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.215467371, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.207664673, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.034551008, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6590, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.449370173, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000068189 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7934, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.002037994, 2024-09-21T03:33:04.131 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000256 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1985, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.049462167, 2024-09-21T03:33:04.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001032474 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 791, 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.038407767, 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000048555 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.133 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 1468, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 1419, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1419, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 25.952655643, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.018289397 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 49, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 1, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 19, 2024-09-21T03:33:04.134 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 1, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 373, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 49, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 309, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 15, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 304, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 416, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 226, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 226, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 33, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 4103, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 3815, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 48, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 35, 2024-09-21T03:33:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 82, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 303, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 9, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 387, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 15, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 569, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 1454, 2024-09-21T03:33:04.136 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 1006, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 1316304154, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 1454, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 1454, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 11, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 1178, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 994, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 1843, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 389, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 160, 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 860 2024-09-21T03:33:04.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 42, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 15, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 3, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 214, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 203, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 7, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 8, 2024-09-21T03:33:04.139 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 9, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 1, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_started": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_write_failed": 0 2024-09-21T03:33:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 2300, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 2300, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 4, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 4, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-21T03:33:04.141 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 8956272, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 893, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 11.857754519, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013278560 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 338, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 921, 2024-09-21T03:33:04.142 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 583, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 58, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 68, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 10, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 379, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 1014, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 635, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 226, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 771, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 545, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 129044, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 335429632 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-21T03:33:04.143 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 1889, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 218, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 1468, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 32, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 343, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 202, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.178890510, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000885596 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 559, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 23.836196229, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.042640780 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.144 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 13, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.218005500, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.016769653 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-21T03:33:04.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 267, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.819413052, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003068962 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 139, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.072835928, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000523999 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005125497, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000465954 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-21T03:33:04.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 213, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.775678560, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003641683 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-21T03:33:04.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-21T03:33:04.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 15, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.046510367, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003100691 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 124, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 124, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2285, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-21T03:33:04.156 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:04.157 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 467241, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 466, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 528, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 6, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 1322240, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 19811, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-21T03:33:04.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 3101, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 4932026, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 29, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 3101, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3101, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 19.662656703, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006340747 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-21T03:33:04.161 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3101, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3306 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 3101, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1555, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 1546, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 100, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 993, 2024-09-21T03:33:04.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 146, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 217, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1454, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:04.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-21T03:33:04.164 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 97, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 22, 2024-09-21T03:33:04.165 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 265, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 99 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.166 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 203, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 203, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 12405, 2024-09-21T03:33:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 5118643, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 12405, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 12405, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 5118643, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3101, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 4952905, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3101, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4952905, 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:04.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3101, 2024-09-21T03:33:04.170 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 3101, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3101, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 3101, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 203, 2024-09-21T03:33:04.171 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 20895, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 203, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 100, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 20895, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x559b31eeb520": { 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 4018, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:04.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2300, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 4761968, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2300, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 893, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4757950, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-09-21T03:33:04.173 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-21T03:33:04.174 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:04.174 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-21T03:33:05.050 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-09-21T03:33:05.050 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2024-09-21T03:33:05.050 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T03:33:05.050 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:05.050 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:05.051 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:33:05.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:33:05.302 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:33:05.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:05 smithi142 ceph-mon[34512]: pgmap v837: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-09-21T03:33:05.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:05 smithi193 ceph-mon[35224]: pgmap v837: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-09-21T03:33:05.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:05 smithi083 ceph-mon[31219]: pgmap v837: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1205, 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1206, 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4507117, 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 382936, 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 7, 2024-09-21T03:33:05.852 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.116233998, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.051778691, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.026972868, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.028246600, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1204, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.046411242, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038547 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:05.854 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1342, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1301, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1927719, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 501253, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.146181935, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.059491441, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.058155629, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.031277633, 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:05.855 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1300, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.060284632, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000046372 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1123024, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 193424 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:05.856 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1174, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1404, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 386802, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 955632, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.158079013, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.070901833, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.035085900, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.040624314, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1403, 2024-09-21T03:33:05.857 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.112600150, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000080256 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 584208 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-21T03:33:05.858 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5698, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.161026997, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000028260 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-21T03:33:05.859 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 568, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 16, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 552, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-21T03:33:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 546, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 15, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-21T03:33:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:33:05.863 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:05.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-09-21T03:33:05.865 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-21T03:33:05.866 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_started": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: "uninline_write_failed": 0 2024-09-21T03:33:05.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 2293, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-21T03:33:05.868 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 4, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 8952254, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 8951680, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 2293 2024-09-21T03:33:05.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 310, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 900, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 590, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 25, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 35, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 10, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 568, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 956, 2024-09-21T03:33:05.870 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 388, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 120416, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 319635456 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-21T03:33:05.871 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.873 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.874 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-21T03:33:05.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.876 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.876 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.876 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-21T03:33:05.876 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-21T03:33:05.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.879 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-21T03:33:05.882 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.883 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2289, 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:05.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:05.885 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:05.886 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 210908, 2024-09-21T03:33:05.887 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 379, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 176, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:05.888 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 1193452, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13653, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 3340, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-09-21T03:33:05.889 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 3340, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3340, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 29.447491106, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008816614 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3340, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3341 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 3340, 2024-09-21T03:33:05.890 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 3340, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2003, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1335, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-21T03:33:05.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-21T03:33:05.892 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-21T03:33:05.893 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 97, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 22, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 6, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 7, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 1, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-21T03:33:05.894 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:05.895 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 3717, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 6465212, 2024-09-21T03:33:05.896 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 3717, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3717, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6465212, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:05.897 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3340, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 4757955, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3340, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4757955, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.898 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 3340, 2024-09-21T03:33:05.899 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 3340, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3340, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 3340, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:05.900 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.901 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55d2e17196c0": { 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-21T03:33:05.902 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-21T03:33:05.903 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-21T03:33:06.708 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T03:33:06.708 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T03:33:06.708 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T03:33:06.708 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:06.708 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:06.708 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:33:06.709 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:33:07.003 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:07.400 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:07 smithi193 ceph-mon[35224]: pgmap v838: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-09-21T03:33:07.425 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:07 smithi083 ceph-mon[31219]: pgmap v838: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-09-21T03:33:07.427 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:07 smithi142 ceph-mon[34512]: pgmap v838: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-09-21T03:33:07.687 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T03:33:07.687 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 6357, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 7471, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2033085, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 7001169, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 52, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 8, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.597067331, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.282151799, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.231678854, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.085266388, 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:07.688 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7470, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.559174098, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000208724 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6660, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001604291, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000240 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 4528, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 1888 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3032, 2024-09-21T03:33:07.689 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3031, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2077574, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1276086, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.282692353, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.132676364, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.128489337, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.033359736, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3030, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.393455373, 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000129853 2024-09-21T03:33:07.690 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2768, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000551384, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000199 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1145952, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 223600 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 6574, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 6094, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3228250, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3253155, 2024-09-21T03:33:07.691 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 12, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.566273208, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.270920380, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.259059899, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.047155199, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6092, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.620573775, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000101867 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8823, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002054315, 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000232 2024-09-21T03:33:07.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 493776 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2962, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.164991274, 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001406141 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.693 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 62, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002836437, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045748 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 1100, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 1047, 2024-09-21T03:33:07.694 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1047, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 14.038838178, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.013408632 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 53, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 7, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 19, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 1, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 552, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 74, 2024-09-21T03:33:07.695 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 467, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 11, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 445, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 390, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 228, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 228, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 28, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 6440, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 6153, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 51, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 54, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:07.696 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 40, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 512, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 9, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 313, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 16, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 568, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 4045, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1006, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1316304154, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-21T03:33:07.697 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 4045, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 4045, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 36, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1422, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 898, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:07.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 2017, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 278, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 54, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 119 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 60, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 45, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 10, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 44, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 14, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 25, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 13, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 9, 2024-09-21T03:33:07.700 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 1, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 1935, 2024-09-21T03:33:07.701 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1935, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 3, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 3, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T03:33:07.702 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 7567514, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 937, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 10.969805506, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011707369 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.703 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 453, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 769, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 316, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 55, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 62, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 7, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 555, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 982, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 427, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 214, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 587, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 373, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 131452, 2024-09-21T03:33:07.704 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 335421440 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 1755, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 392, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 1100, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 70, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 691, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.024686464, 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002742940 2024-09-21T03:33:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 663, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 12.530878841, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.018900269 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.706 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 29, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.262780912, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009061410 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 204, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.182439885, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000894313 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.707 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:07.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.709 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 38, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.026408845, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000694969 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 17, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.008893803, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000523164 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 43, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.846753357, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.019691938 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.155996071, 2024-09-21T03:33:07.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003545365 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T03:33:07.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 115, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 57, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 973, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:07.718 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:07.719 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:07.720 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 550117, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 584, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 352, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 4, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:07.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 176, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 1746152, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 28111, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:07.722 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 5302, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 3405723, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 5302, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5302, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 22.525646272, 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004248518 2024-09-21T03:33:07.723 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5302, 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 5513 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 5302, 2024-09-21T03:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 4068, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1234, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 7, 2024-09-21T03:33:07.725 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 944, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 73, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 27, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 4045, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-21T03:33:07.726 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T03:33:07.727 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 97, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 22, 2024-09-21T03:33:07.728 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 290, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 23, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 74 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.729 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 14, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 14, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 15959, 2024-09-21T03:33:07.730 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5876895, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 15959, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 15959, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5876895, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.731 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5302, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3407112, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5302, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3407112, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.732 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5302, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 5302, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5302, 2024-09-21T03:33:07.733 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5302, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 14, 2024-09-21T03:33:07.734 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1726, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 14, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 7, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1726, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.735 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x564a1117b380": { 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 117895, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1935, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 3373210, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:07.736 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1935, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 937, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3255315, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 1 2024-09-21T03:33:07.737 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:07.738 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T03:33:08.823 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2024-09-21T03:33:08.823 INFO:tasks.check_counter:Daemon mds.c mds.imported=16 2024-09-21T03:33:08.823 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-21T03:33:08.823 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:08.823 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:08.823 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:33:08.823 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:33:09.097 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: pgmap v839: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:09.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:09 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: pgmap v839: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:09.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:09 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:33:09.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: pgmap v839: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:09.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:09 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' 2024-09-21T03:33:09.958 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T03:33:09.958 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:09.958 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4519, 2024-09-21T03:33:09.958 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4515, 2024-09-21T03:33:09.958 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1675146, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 5183396, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 54, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 9, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.397152323, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.180910892, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.150095845, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.061173068, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4511, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.530643261, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000117633 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.959 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4270, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000885248, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000207 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 6971, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 6959, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3126649, 2024-09-21T03:33:09.960 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3628373, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.578950181, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.289557704, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.277054730, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.040627474, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6958, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.265997033, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000181948 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9378, 2024-09-21T03:33:09.961 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002067233, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000220 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 494368 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3892, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4186, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2292455, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1726140, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.423836558, 2024-09-21T03:33:09.962 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.218162883, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.162663400, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.045020951, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4187, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.506276802, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000120916 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3646, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001059749, 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000290 2024-09-21T03:33:09.963 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1141328, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 226528 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2956, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.209924653, 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001424196 2024-09-21T03:33:09.964 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000020638, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000010319 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 712, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 684, 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T03:33:09.965 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 684, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 20.185355965, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.029510754 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 28, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 2, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 28, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 2, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 1, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 684, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 87, 2024-09-21T03:33:09.966 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 583, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 14, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 496, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 175, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 245, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 245, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 37, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 5982, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 5736, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 28, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 61, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:09.967 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 29, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 791, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 3, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 108, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 13, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 514, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 4100, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1006, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1316304154, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 4100, 2024-09-21T03:33:09.968 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 4100, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 36, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1203, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 710, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 1642, 2024-09-21T03:33:09.969 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 129, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 34, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 94 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 68, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 50, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 4, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:09.970 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 33, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 22, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 24, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 3, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T03:33:09.971 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 3, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 1689, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1689, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T03:33:09.972 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 3, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 3, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 7073003, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 872, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 7.052689763, 2024-09-21T03:33:09.973 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008087946 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 582, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 693, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 111, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 71, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 77, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 6, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 695, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 883, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 188, 2024-09-21T03:33:09.974 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 225, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 640, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 415, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 129632, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 336470016 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 1327, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 643, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 712, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 84, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 832, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4, 2024-09-21T03:33:09.975 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.009056627, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002264156 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 335, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 18.462684620, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.055112491 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.976 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 29, 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.250177914, 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008626824 2024-09-21T03:33:09.977 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 182, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.122501726, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000673086 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.978 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000120343, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000120343 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.979 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.980 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.980 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:09.980 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.980 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.981 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T03:33:09.982 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 36, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.078923005, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002192305 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 25, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.026173866, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001046954 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 39, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.038356242, 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.026624519 2024-09-21T03:33:09.983 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.984 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.985 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.986 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.987 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 33, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.197361622, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005980655 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:09.988 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 98, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 49, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 971, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:09.989 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:09.991 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 799932, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 741, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T03:33:09.992 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 176, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T03:33:09.993 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2193841, 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 38399, 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T03:33:09.994 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 5211, 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 3025844, 2024-09-21T03:33:09.995 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-21T03:33:09.996 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 5211, 2024-09-21T03:33:09.997 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T03:33:09.997 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5211, 2024-09-21T03:33:09.997 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 18.136715201, 2024-09-21T03:33:09.997 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003480467 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5211, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 5385 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 5211, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 4104, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1107, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-21T03:33:09.998 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 872, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 62, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 10, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 4100, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-21T03:33:09.999 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T03:33:10.000 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T03:33:10.001 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T03:33:10.002 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 98, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-21T03:33:10.003 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 250, 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 6, 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 55 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T03:33:10.004 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-21T03:33:10.005 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 15378, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5684634, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 15378, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 15378, 2024-09-21T03:33:10.006 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5684634, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:10.007 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5211, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3025722, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5211, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3025722, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:10.008 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5211, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 5211, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5211, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5211, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:10.009 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T03:33:10.010 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55c17cd23520": { 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 61039, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1689, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2878699, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1689, 2024-09-21T03:33:10.011 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 872, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2817660, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 2 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:10.012 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T03:33:10.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:10 smithi142 ceph-mon[34512]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:33:10.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:10 smithi142 ceph-mon[34512]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:33:10.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:10 smithi142 ceph-mon[34512]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:33:10.436 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:10 smithi193 ceph-mon[35224]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:33:10.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:10 smithi083 ceph-mon[31219]: Adjusting osd_memory_target on smithi083 to 176.0M 2024-09-21T03:33:10.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:10 smithi083 ceph-mon[31219]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:33:10.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:10 smithi083 ceph-mon[31219]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:33:10.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:10 smithi193 ceph-mon[35224]: Unable to set osd_memory_target on smithi083 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-21T03:33:10.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:10 smithi193 ceph-mon[35224]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:33:11.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:11 smithi193 ceph-mon[35224]: pgmap v841: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-09-21T03:33:11.667 INFO:tasks.check_counter:Daemon mds.f mds.exported=3 2024-09-21T03:33:11.667 INFO:tasks.check_counter:Daemon mds.f mds.imported=13 2024-09-21T03:33:11.667 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-21T03:33:11.667 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:11.667 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:11.667 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:33:11.668 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:33:11.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:11 smithi083 ceph-mon[31219]: pgmap v841: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-09-21T03:33:11.924 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:11.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:11 smithi142 ceph-mon[34512]: pgmap v841: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 34 op/s 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 7833, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 6951, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3131801, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 8241422, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 49, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 8, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.692120327, 2024-09-21T03:33:12.516 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.317318038, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.358261431, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.067585297, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6950, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.036484381, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000149134 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9812, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002632009, 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000268 2024-09-21T03:33:12.517 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 7048, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 8236, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2318306, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 4287527, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 16, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.566851206, 2024-09-21T03:33:12.518 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.304917906, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.291031200, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.017986260, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8234, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.999815266, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000121425 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11564, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003097714, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000267 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:33:12.519 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 585792 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1877, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1799, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1762698, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 935929, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.186089773, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.086232379, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.096342748, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.016185807, 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:12.520 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1798, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.194531892, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000108193 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1685, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000304822, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000180 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1156320, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 232544 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.521 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3239, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.668099278, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000823741 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.522 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 16, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000288655, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000018040 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 1557, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 1508, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1508, 2024-09-21T03:33:12.523 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 10.114132944, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.006706984 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 49, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 3, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 28, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 2, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 534, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 70, 2024-09-21T03:33:12.524 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 347, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 117, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 433, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 351, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 48, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 48, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 24, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 5482, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 4546, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 42, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 44, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:12.525 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 209, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 611, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 6, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 211, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 4, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 203, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 1565, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1006, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1316304154, 2024-09-21T03:33:12.526 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 1565, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 18, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 18, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 1547, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 13, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1470, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 1341, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-09-21T03:33:12.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 2370, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 554, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 66, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 653 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 44, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 54, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 4, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:12.528 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 149, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 1, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 145, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 17, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:12.529 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 7, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 2, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 2902, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T03:33:12.530 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 2902, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 5, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 5, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T03:33:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 9138119, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1434, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 12.169025275, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008486070 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.532 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 285, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 467, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 182, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 56, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 60, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 4, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 543, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 942, 2024-09-21T03:33:12.533 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 399, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 48, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 359, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 311, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 127440, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 334385152 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T03:33:12.534 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 2714, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 479, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 1557, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 315, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 764, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 66, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.072820363, 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001103338 2024-09-21T03:33:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 289, 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6.981522598, 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.024157517 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.536 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 154, 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.504064728, 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003273147 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.537 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 502, 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.498118028, 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000992266 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T03:33:12.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.540 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:12.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.541 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T03:33:12.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.544 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 161, 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.197865408, 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001228977 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 23, 2024-09-21T03:33:12.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.014436186, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000627660 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 164, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.551063876, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009457706 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.548 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.550 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 149, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.294241757, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001974776 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.551 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 149, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 74, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1633, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:12.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:12.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 1116623, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 534, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T03:33:12.555 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7968, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 176, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 1391179, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 43791, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:12.556 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3310, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 5254854, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 30, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3310, 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T03:33:12.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3310, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 19.726688967, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005959724 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3310, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3547 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3310, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1572, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1738, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T03:33:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 1435, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 66, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 1, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 15, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1565, 2024-09-21T03:33:12.559 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T03:33:12.560 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T03:33:12.561 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 98, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 357, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 9, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 68 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:12.562 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 1, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 1, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 1, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 1, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:12.563 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 16754, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5679304, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 16754, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 16754, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5679304, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.564 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3310, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 5254804, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3310, 2024-09-21T03:33:12.565 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5254804, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:12.566 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3310, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3310, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3310, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3310, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.567 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 133, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:12.568 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 133, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x563e22a7f520": { 2024-09-21T03:33:12.569 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2902, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 4943815, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2902, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1434, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 4943815, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:12.570 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 1 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:12.571 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T03:33:13.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:13 smithi193 ceph-mon[35224]: pgmap v842: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T03:33:13.769 INFO:tasks.check_counter:Daemon mds.i mds.exported=6 2024-09-21T03:33:13.769 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-09-21T03:33:13.769 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T03:33:13.769 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:13.769 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:13.769 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:33:13.769 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:33:13.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:13 smithi083 ceph-mon[31219]: pgmap v842: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T03:33:13.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:13 smithi142 ceph-mon[34512]: pgmap v842: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-09-21T03:33:14.027 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3437, 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3439, 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 5906902, 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1092973, 2024-09-21T03:33:14.592 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.360831613, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.130864178, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.078525698, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.132493758, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3437, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.154744744, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045023 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.593 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 238, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 876, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 586474, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-09-21T03:33:14.594 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.016061524, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.012414486, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.001724233, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000082272, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 236, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.023520908, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000099664 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.595 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 585776 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 381, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 340, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1127916, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 196286, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.055830992, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.024726108, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.042887153, 2024-09-21T03:33:14.596 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000034666, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 340, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.022801452, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000067063 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1127648, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 195968 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.597 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5876, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.152731874, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000025992 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T03:33:14.598 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.599 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 529, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 9, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 520, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-21T03:33:14.600 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 517, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 14, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-21T03:33:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T03:33:14.602 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:33:14.603 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:33:14.604 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T03:33:14.605 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 2902, 2024-09-21T03:33:14.606 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 5, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 9138119, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 9129014, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.607 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 2902 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 248, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 438, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 190, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 26, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 34, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 8, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 529, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 801, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 272, 2024-09-21T03:33:14.608 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 119340, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 319627264 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.609 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T03:33:14.610 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.611 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T03:33:14.612 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T03:33:14.613 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.616 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.617 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.618 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.619 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.620 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.622 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T03:33:14.623 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2301, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:33:14.624 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:33:14.625 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:33:14.626 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:33:14.627 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 194949, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 377, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8088, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 179, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:33:14.628 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 1099935, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 31591, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T03:33:14.629 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3440, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3440, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3440, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 16.878736756, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004906609 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3440, 2024-09-21T03:33:14.630 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3441 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3440, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3440, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 2058, 2024-09-21T03:33:14.631 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1380, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T03:33:14.632 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:33:14.633 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:33:14.634 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T03:33:14.635 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T03:33:14.636 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 98, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 4, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 5, 2024-09-21T03:33:14.637 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:33:14.638 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:14.639 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3818, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 6669971, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3818, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3818, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6669971, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.640 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3440, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 4944594, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3440, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 4944594, 2024-09-21T03:33:14.641 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:14.642 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3440, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3440, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3440, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3440, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.644 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5621918832b0": { 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T03:33:14.645 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T03:33:14.646 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T03:33:15.439 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T03:33:15.439 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T03:33:15.439 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-21T03:33:15.439 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:33:15.439 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:33:15.439 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:33:15.449 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:33:15.449 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:33:15.450 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:33:15.492 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-21T03:33:15.492 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:33:15.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:33:15.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:15 smithi193 ceph-mon[35224]: pgmap v843: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:15.854 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:15.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:15 smithi083 ceph-mon[31219]: pgmap v843: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:15.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:15 smithi142 ceph-mon[34512]: pgmap v843: 97 pgs: 97 active+clean; 4.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1372331442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 msgr2=0x7f12b410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1372331442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b410f530 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009ca0 tx=0x7f12ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1372331442 shutdown_connections 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1372331442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b410f530 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1372331442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1372331442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1372331442 >> 172.21.15.83:0/1372331442 conn(0x7f12b4079710 msgr2=0x7f12b4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1372331442 shutdown_connections 2024-09-21T03:33:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.348+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1372331442 wait complete. 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 Processor -- start 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 -- start start 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12b41129c0 con 0x7f12b4108d00 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4112840 con 0x7f12b4102a70 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12bb4ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4112b40 con 0x7f12b41083b0 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36908/0 (socket says 172.21.15.83:36908) 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 -- 172.21.15.83:0/1888870602 learned_addr learned my addr 172.21.15.83:0/1888870602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9248640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b8a47640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 -- 172.21.15.83:0/1888870602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 msgr2=0x7f12b419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 -- 172.21.15.83:0/1888870602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 msgr2=0x7f12b419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9a49640 1 -- 172.21.15.83:0/1888870602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41a25a0 con 0x7f12b4108d00 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b9248640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.349+0000 7f12b8a47640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12b9a49640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0098d0 tx=0x7f12ac0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac003b70 con 0x7f12b4108d00 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a2830 con 0x7f12b4108d00 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ac003d10 con 0x7f12b4108d00 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac02fa20 con 0x7f12b4108d00 2024-09-21T03:33:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.350+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a8ff0 con 0x7f12b4108d00 2024-09-21T03:33:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.351+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4103cf0 con 0x7f12b4108d00 2024-09-21T03:33:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.353+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f12ac02fc20 con 0x7f12b4108d00 2024-09-21T03:33:16.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.354+0000 7f12aa7fc640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 0x7f12a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:16.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.354+0000 7f12b9248640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 0x7f12a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:16.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.354+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8485+0+0 (secure 0 0 0) 0x7f12ac108ca0 con 0x7f12b4108d00 2024-09-21T03:33:16.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.355+0000 7f12b9248640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 0x7f12a00b8b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005d00 tx=0x7f12b0005c70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:16.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.357+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12ac0393b0 con 0x7f12b4108d00 2024-09-21T03:33:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.549+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12b4102e80 con 0x7f12b4108d00 2024-09-21T03:33:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.550+0000 7f12aa7fc640 1 -- 172.21.15.83:0/1888870602 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v98) ==== 81+0+884 (secure 0 0 0) 0x7f12b4102e80 con 0x7f12b4108d00 2024-09-21T03:33:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1976219591 2024-09-22T03:07:59.080195+0000 2024-09-21T03:33:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3984963619 2024-09-22T03:06:36.366094+0000 2024-09-21T03:33:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/3130587124 2024-09-22T03:07:01.129903+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2346813165 2024-09-22T03:07:59.080195+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1246429082 2024-09-22T03:06:36.366094+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/493695771 2024-09-22T03:07:59.080195+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1246429082 2024-09-22T03:06:36.366094+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/211211530 2024-09-22T03:07:01.129903+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1905391534 2024-09-22T03:06:36.366094+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/817883806 2024-09-22T03:06:36.366094+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3621693467 2024-09-22T03:07:01.129903+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2962867597 2024-09-22T03:07:01.129903+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3776769458 2024-09-22T03:07:59.080195+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/3130587124 2024-09-22T03:07:01.129903+0000 2024-09-21T03:33:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/493695771 2024-09-22T03:07:59.080195+0000 2024-09-21T03:33:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:listed 15 entries 2024-09-21T03:33:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 msgr2=0x7f12a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 0x7f12a00b8b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005d00 tx=0x7f12b0005c70 comp rx=0 tx=0).stop 2024-09-21T03:33:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 msgr2=0x7f12b41a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0098d0 tx=0x7f12ac0317b0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 shutdown_connections 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f12a00b6680 0x7f12a00b8b90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b4108d00 0x7f12b41a1e70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b41083b0 0x7f12b419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 --2- 172.21.15.83:0/1888870602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b4102a70 0x7f12b419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 >> 172.21.15.83:0/1888870602 conn(0x7f12b4079710 msgr2=0x7f12b4106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 shutdown_connections 2024-09-21T03:33:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:16.552+0000 7f12bb4ac640 1 -- 172.21.15.83:0/1888870602 wait complete. 2024-09-21T03:33:17.285 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:33:17.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:17.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /tmp/cephfs 2024-09-21T03:33:17.495 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:33:17.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:17.495 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-21T03:33:17.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:17.547 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-09-21T03:33:17.621 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-21T03:33:17.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:17.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-21T03:33:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:17 smithi083 ceph-mon[31219]: pgmap v844: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:17.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:17 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/1888870602' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:33:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:33:17.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-09-21T03:33:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:17 smithi142 ceph-mon[34512]: pgmap v844: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:17 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/1888870602' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:33:17.805 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:33:17.815 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:33:17.815 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:33:17.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:17.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:17 smithi193 ceph-mon[35224]: pgmap v844: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-21T03:33:17.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:17 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/1888870602' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:33:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 -- 172.21.15.83:0/370715177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 msgr2=0x7f95dc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 --2- 172.21.15.83:0/370715177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc10f520 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009ca0 tx=0x7f95d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 -- 172.21.15.83:0/370715177 shutdown_connections 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 --2- 172.21.15.83:0/370715177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc10f520 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 --2- 172.21.15.83:0/370715177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc1083a0 0x7f95dc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 --2- 172.21.15.83:0/370715177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc102a60 0x7f95dc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 -- 172.21.15.83:0/370715177 >> 172.21.15.83:0/370715177 conn(0x7f95dc079730 msgr2=0x7f95dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 -- 172.21.15.83:0/370715177 shutdown_connections 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.242+0000 7f95e16e5640 1 -- 172.21.15.83:0/370715177 wait complete. 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 Processor -- start 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 -- start start 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc102a60 0x7f95dc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 0x7f95dc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc112960 con 0x7f95dc108cf0 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1127e0 con 0x7f95dc102a60 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95e16e5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc112ae0 con 0x7f95dc1083a0 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48444/0 (socket says 172.21.15.83:48444) 2024-09-21T03:33:19.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95db7fe640 1 -- 172.21.15.83:0/114282956 learned_addr learned my addr 172.21.15.83:0/114282956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.243+0000 7f95daffd640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc102a60 0x7f95dc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95d27fc640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 0x7f95dc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 -- 172.21.15.83:0/114282956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 msgr2=0x7f95dc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 0x7f95dc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 -- 172.21.15.83:0/114282956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc102a60 msgr2=0x7f95dc19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc102a60 0x7f95dc19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 -- 172.21.15.83:0/114282956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc1a0350 con 0x7f95dc108cf0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95d27fc640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 0x7f95dc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95db7fe640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f95d40098d0 tx=0x7f95d40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4003b70 con 0x7f95dc108cf0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc1a05e0 con 0x7f95dc108cf0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d4003d10 con 0x7f95dc108cf0 2024-09-21T03:33:19.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d402fa20 con 0x7f95dc108cf0 2024-09-21T03:33:19.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.244+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc1a6da0 con 0x7f95dc108cf0 2024-09-21T03:33:19.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.245+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95dc102e70 con 0x7f95dc108cf0 2024-09-21T03:33:19.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.247+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f95d402fbe0 con 0x7f95dc108cf0 2024-09-21T03:33:19.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.248+0000 7f95d8ff9640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 0x7f95c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:19.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.248+0000 7f95daffd640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 0x7f95c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:19.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.248+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8441+0+0 (secure 0 0 0) 0x7f95d4108ba0 con 0x7f95dc108cf0 2024-09-21T03:33:19.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.248+0000 7f95daffd640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 0x7f95c80b8b90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005fd0 tx=0x7f95cc00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:19.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.251+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95d40cf140 con 0x7f95dc108cf0 2024-09-21T03:33:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.469+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95dc19b510 con 0x7f95dc108cf0 2024-09-21T03:33:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.470+0000 7f95d8ff9640 1 -- 172.21.15.83:0/114282956 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11995 (secure 0 0 0) 0x7f95d40d3520 con 0x7f95dc108cf0 2024-09-21T03:33:19.472 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:33:19.472 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"btime":"2024-09-21T03:32:47:106513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:32:46.272413+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:33:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 msgr2=0x7f95c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 0x7f95c80b8b90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005fd0 tx=0x7f95cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 msgr2=0x7f95dc19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f95d40098d0 tx=0x7f95d40317b0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 shutdown_connections 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f95c80b6680 0x7f95c80b8b90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95dc108cf0 0x7f95dc19fc20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95dc1083a0 0x7f95dc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 --2- 172.21.15.83:0/114282956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95dc102a60 0x7f95dc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:19.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 >> 172.21.15.83:0/114282956 conn(0x7f95dc079730 msgr2=0x7f95dc06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:19.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 shutdown_connections 2024-09-21T03:33:19.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:19.472+0000 7f95e16e5640 1 -- 172.21.15.83:0/114282956 wait complete. 2024-09-21T03:33:19.782 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:19 smithi083 ceph-mon[31219]: pgmap v845: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 14 op/s 2024-09-21T03:33:19.782 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:19 smithi083 ceph-mon[31219]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:33:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:19 smithi193 ceph-mon[35224]: pgmap v845: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 14 op/s 2024-09-21T03:33:19.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:19 smithi193 ceph-mon[35224]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:33:19.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:19 smithi142 ceph-mon[34512]: pgmap v845: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 14 op/s 2024-09-21T03:33:19.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:19 smithi142 ceph-mon[34512]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:33:20.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:20.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:20 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/114282956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:20.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:20 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/114282956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:20.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:20 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/114282956' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 -- 172.21.15.83:0/2377232572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998103b60 msgr2=0x7f8998103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 --2- 172.21.15.83:0/2377232572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998103b60 0x7f8998103f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8994009900 tx=0x7f899402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 -- 172.21.15.83:0/2377232572 shutdown_connections 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 --2- 172.21.15.83:0/2377232572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998109df0 0x7f8998110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 --2- 172.21.15.83:0/2377232572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 0x7f89981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 --2- 172.21.15.83:0/2377232572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998103b60 0x7f8998103f70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 -- 172.21.15.83:0/2377232572 >> 172.21.15.83:0/2377232572 conn(0x7f899807b850 msgr2=0x7f899807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 -- 172.21.15.83:0/2377232572 shutdown_connections 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.901+0000 7f899fa91640 1 -- 172.21.15.83:0/2377232572 wait complete. 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 Processor -- start 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 -- start start 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 0x7f899819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 0x7f899819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113ab0 con 0x7f8998109df0 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113930 con 0x7f8998103b60 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899fa91640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113c30 con 0x7f89981094a0 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.902+0000 7f899e02e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48470/0 (socket says 172.21.15.83:48470) 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 -- 172.21.15.83:0/4219710015 learned_addr learned my addr 172.21.15.83:0/4219710015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899d02c640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 0x7f899819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899d82d640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 0x7f899819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 -- 172.21.15.83:0/4219710015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 msgr2=0x7f899819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 0x7f899819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 -- 172.21.15.83:0/4219710015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 msgr2=0x7f899819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 0x7f899819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 -- 172.21.15.83:0/4219710015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981a0470 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899d82d640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 0x7f899819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899e02e640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f899000cad0 tx=0x7f899000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899000ae20 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8990005020 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89981a0760 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899000f6b0 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.903+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a6ec0 con 0x7f8998109df0 2024-09-21T03:33:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.904+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8998103f70 con 0x7f8998109df0 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.906+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f89900051d0 con 0x7f8998109df0 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.906+0000 7f898effd640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 0x7f89840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.906+0000 7f899d82d640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 0x7f89840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.907+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8441+0+0 (secure 0 0 0) 0x7f89900e7660 con 0x7f8998109df0 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.907+0000 7f899d82d640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 0x7f89840b8c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8994031440 tx=0x7f8994009680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:20.911+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8990014030 con 0x7f8998109df0 2024-09-21T03:33:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.127+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f899802e640 con 0x7f8998109df0 2024-09-21T03:33:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.128+0000 7f898effd640 1 -- 172.21.15.83:0/4219710015 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11995 (secure 0 0 0) 0x7f89900adc00 con 0x7f8998109df0 2024-09-21T03:33:21.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:33:21.130 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"btime":"2024-09-21T03:32:47:106513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14784,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3297642632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3297642632},{"type":"v1","addr":"172.21.15.83:6835","nonce":3297642632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14796,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3179576739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3179576739},{"type":"v1","addr":"172.21.15.83:6837","nonce":3179576739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:14:37.183732+0000","modified":"2024-09-21T03:32:46.272413+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24647,"mds_1":24628,"mds_2":24620,"mds_3":24613,"mds_4":24601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/3046510766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3046510766},{"type":"v1","addr":"172.21.15.83:6839","nonce":3046510766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2540451962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2540451962},{"type":"v1","addr":"172.21.15.83:6841","nonce":2540451962}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/1409264490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1409264490},{"type":"v1","addr":"172.21.15.142:6833","nonce":1409264490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24571":{"gid":24571,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/817680520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":817680520},{"type":"v1","addr":"172.21.15.193:6833","nonce":817680520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/3055596594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3055596594},{"type":"v1","addr":"172.21.15.142:6835","nonce":3055596594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24601":{"gid":24601,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6835/1130551949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1130551949},{"type":"v1","addr":"172.21.15.193:6835","nonce":1130551949}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.193:6837/3456789844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":3456789844},{"type":"v1","addr":"172.21.15.193:6837","nonce":3456789844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.142:6837/1691832808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1691832808},{"type":"v1","addr":"172.21.15.142:6837","nonce":1691832808}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/357057598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":357057598},{"type":"v1","addr":"172.21.15.193:6839","nonce":357057598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.142:6839/3152549242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3152549242},{"type":"v1","addr":"172.21.15.142:6839","nonce":3152549242}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24647,"qdb_cluster":[24601,24613,24620,24628,24647]},"id":1}]} 2024-09-21T03:33:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2024-09-21T03:33:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 msgr2=0x7f89840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 0x7f89840b8c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8994031440 tx=0x7f8994009680 comp rx=0 tx=0).stop 2024-09-21T03:33:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 msgr2=0x7f899819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f899000cad0 tx=0x7f899000cf90 comp rx=0 tx=0).stop 2024-09-21T03:33:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 shutdown_connections 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f89840b6700 0x7f89840b8c10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8998109df0 0x7f899819fd40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89981094a0 0x7f899819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 --2- 172.21.15.83:0/4219710015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8998103b60 0x7f899819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 >> 172.21.15.83:0/4219710015 conn(0x7f899807b850 msgr2=0x7f89981075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 shutdown_connections 2024-09-21T03:33:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:21.130+0000 7f899fa91640 1 -- 172.21.15.83:0/4219710015 wait complete. 2024-09-21T03:33:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:21 smithi193 ceph-mon[35224]: pgmap v847: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-09-21T03:33:21.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:21 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4219710015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:21.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:21 smithi083 ceph-mon[31219]: pgmap v847: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-09-21T03:33:21.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:21 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4219710015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:21.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:21 smithi142 ceph-mon[34512]: pgmap v847: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-09-21T03:33:21.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:21 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4219710015' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:21.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:33:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:22.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:22 smithi083 ceph-mon[31219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.663+0000 7f11e2bc7640 1 -- 172.21.15.83:0/533207331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108420 msgr2=0x7f11dc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.663+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/533207331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108420 0x7f11dc108830 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 -- 172.21.15.83:0/533207331 shutdown_connections 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/533207331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108d70 0x7f11dc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/533207331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108420 0x7f11dc108830 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/533207331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 0x7f11dc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 -- 172.21.15.83:0/533207331 >> 172.21.15.83:0/533207331 conn(0x7f11dc079770 msgr2=0x7f11dc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 -- 172.21.15.83:0/533207331 shutdown_connections 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 -- 172.21.15.83:0/533207331 wait complete. 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 Processor -- start 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.664+0000 7f11e2bc7640 1 -- start start 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 0x7f11dc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108420 0x7f11dc19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc112a30 con 0x7f11dc108d70 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1128b0 con 0x7f11dc102ae0 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e2bc7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc112bb0 con 0x7f11dc108420 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48482/0 (socket says 172.21.15.83:48482) 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 -- 172.21.15.83:0/674609539 learned_addr learned my addr 172.21.15.83:0/674609539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e0963640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 0x7f11dc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11dbfff640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108420 0x7f11dc19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 -- 172.21.15.83:0/674609539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108420 msgr2=0x7f11dc19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108420 0x7f11dc19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 -- 172.21.15.83:0/674609539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 msgr2=0x7f11dc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 0x7f11dc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 -- 172.21.15.83:0/674609539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc1a6650 con 0x7f11dc108d70 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11e1164640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f11d400eaa0 tx=0x7f11d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.665+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d400cdb0 con 0x7f11dc108d70 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.666+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d4005020 con 0x7f11dc108d70 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.666+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc1a6940 con 0x7f11dc108d70 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.666+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d40116d0 con 0x7f11dc108d70 2024-09-21T03:33:22.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.666+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc1a6e50 con 0x7f11dc108d70 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.667+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc103d60 con 0x7f11dc108d70 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.669+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f11d40051d0 con 0x7f11dc108d70 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.670+0000 7f11d9ffb640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 0x7f11c80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.670+0000 7f11e0963640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 0x7f11c80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.670+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8441+0+0 (secure 0 0 0) 0x7f11d40e7ac0 con 0x7f11dc108d70 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.670+0000 7f11e0963640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 0x7f11c80b8bc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11dc06c6c0 tx=0x7f11cc00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:22.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.673+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11d4016030 con 0x7f11dc108d70 2024-09-21T03:33:22.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:22 smithi193 ceph-mon[35224]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:33:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.854+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11dc06bf70 con 0x7f11dc108d70 2024-09-21T03:33:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.855+0000 7f11d9ffb640 1 -- 172.21.15.83:0/674609539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23038 (secure 0 0 0) 0x7f11dc06bf70 con 0x7f11dc108d70 2024-09-21T03:33:22.857 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-21T03:33:22.857 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"5108497e-77c6-11ef-baf5-efdc52797490","created":"2024-09-21T03:06:13.570352+0000","modified":"2024-09-21T03:33:19.089235+0000","last_up_change":"2024-09-21T03:13:47.223310+0000","last_in_change":"2024-09-21T03:13:23.568829+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:09:57.414337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:14:33.119862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:14:35.204610+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f96fcc94-0c90-4aaa-83e9-93e0c3cdc835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6803","nonce":4290610745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6805","nonce":4290610745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6809","nonce":4290610745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4290610745},{"type":"v1","addr":"172.21.15.83:6807","nonce":4290610745}]},"public_addr":"172.21.15.83:6803/4290610745","cluster_addr":"172.21.15.83:6805/4290610745","heartbeat_back_addr":"172.21.15.83:6809/4290610745","heartbeat_front_addr":"172.21.15.83:6807/4290610745","state":["exists","up"]},{"osd":1,"uuid":"949b227b-4c32-4ca5-a1db-df21fe923196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6801","nonce":3543209885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6803","nonce":3543209885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6807","nonce":3543209885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3543209885},{"type":"v1","addr":"172.21.15.142:6805","nonce":3543209885}]},"public_addr":"172.21.15.142:6801/3543209885","cluster_addr":"172.21.15.142:6803/3543209885","heartbeat_back_addr":"172.21.15.142:6807/3543209885","heartbeat_front_addr":"172.21.15.142:6805/3543209885","state":["exists","up"]},{"osd":2,"uuid":"49bca58f-0193-459d-915a-a295270c7fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6801","nonce":905402231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6803","nonce":905402231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6807","nonce":905402231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":905402231},{"type":"v1","addr":"172.21.15.193:6805","nonce":905402231}]},"public_addr":"172.21.15.193:6801/905402231","cluster_addr":"172.21.15.193:6803/905402231","heartbeat_back_addr":"172.21.15.193:6807/905402231","heartbeat_front_addr":"172.21.15.193:6805/905402231","state":["exists","up"]},{"osd":3,"uuid":"280e4e6e-e486-492c-b238-36b38206052c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6811","nonce":4077666910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6813","nonce":4077666910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6817","nonce":4077666910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4077666910},{"type":"v1","addr":"172.21.15.83:6815","nonce":4077666910}]},"public_addr":"172.21.15.83:6811/4077666910","cluster_addr":"172.21.15.83:6813/4077666910","heartbeat_back_addr":"172.21.15.83:6817/4077666910","heartbeat_front_addr":"172.21.15.83:6815/4077666910","state":["exists","up"]},{"osd":4,"uuid":"0c6f14a7-871f-4d09-a704-71022463636c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6809","nonce":4220832012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6811","nonce":4220832012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6815","nonce":4220832012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4220832012},{"type":"v1","addr":"172.21.15.142:6813","nonce":4220832012}]},"public_addr":"172.21.15.142:6809/4220832012","cluster_addr":"172.21.15.142:6811/4220832012","heartbeat_back_addr":"172.21.15.142:6815/4220832012","heartbeat_front_addr":"172.21.15.142:6813/4220832012","state":["exists","up"]},{"osd":5,"uuid":"0de58bca-db3b-4600-83bb-9df3665dc790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6809","nonce":2703219317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6811","nonce":2703219317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6815","nonce":2703219317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2703219317},{"type":"v1","addr":"172.21.15.193:6813","nonce":2703219317}]},"public_addr":"172.21.15.193:6809/2703219317","cluster_addr":"172.21.15.193:6811/2703219317","heartbeat_back_addr":"172.21.15.193:6815/2703219317","heartbeat_front_addr":"172.21.15.193:6813/2703219317","state":["exists","up"]},{"osd":6,"uuid":"2a5a5708-35f7-4e4b-a46c-3c1afab03a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6819","nonce":2541470109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6821","nonce":2541470109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6825","nonce":2541470109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":2541470109},{"type":"v1","addr":"172.21.15.83:6823","nonce":2541470109}]},"public_addr":"172.21.15.83:6819/2541470109","cluster_addr":"172.21.15.83:6821/2541470109","heartbeat_back_addr":"172.21.15.83:6825/2541470109","heartbeat_front_addr":"172.21.15.83:6823/2541470109","state":["exists","up"]},{"osd":7,"uuid":"07842e54-9f66-42d4-9a74-2458dbdd8c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6817","nonce":3641783594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6819","nonce":3641783594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6823","nonce":3641783594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3641783594},{"type":"v1","addr":"172.21.15.142:6821","nonce":3641783594}]},"public_addr":"172.21.15.142:6817/3641783594","cluster_addr":"172.21.15.142:6819/3641783594","heartbeat_back_addr":"172.21.15.142:6823/3641783594","heartbeat_front_addr":"172.21.15.142:6821/3641783594","state":["exists","up"]},{"osd":8,"uuid":"2f61dfd6-a1c7-4a71-8b8b-f6d19c7ac558","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6817","nonce":1886733655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6819","nonce":1886733655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6823","nonce":1886733655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1886733655},{"type":"v1","addr":"172.21.15.193:6821","nonce":1886733655}]},"public_addr":"172.21.15.193:6817/1886733655","cluster_addr":"172.21.15.193:6819/1886733655","heartbeat_back_addr":"172.21.15.193:6823/1886733655","heartbeat_front_addr":"172.21.15.193:6821/1886733655","state":["exists","up"]},{"osd":9,"uuid":"fc8eed7c-f5c5-4eee-8d08-0ba29e4db1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6827","nonce":400576662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6829","nonce":400576662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6833","nonce":400576662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":400576662},{"type":"v1","addr":"172.21.15.83:6831","nonce":400576662}]},"public_addr":"172.21.15.83:6827/400576662","cluster_addr":"172.21.15.83:6829/400576662","heartbeat_back_addr":"172.21.15.83:6833/400576662","heartbeat_front_addr":"172.21.15.83:6831/400576662","state":["exists","up"]},{"osd":10,"uuid":"f60a1b3a-0053-460c-848e-aa177514017d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6825","nonce":2781310607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6827","nonce":2781310607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6831","nonce":2781310607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2781310607},{"type":"v1","addr":"172.21.15.142:6829","nonce":2781310607}]},"public_addr":"172.21.15.142:6825/2781310607","cluster_addr":"172.21.15.142:6827/2781310607","heartbeat_back_addr":"172.21.15.142:6831/2781310607","heartbeat_front_addr":"172.21.15.142:6829/2781310607","state":["exists","up"]},{"osd":11,"uuid":"920f94f1-463c-4ffd-bd02-2470631a6746","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6825","nonce":1672057754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6827","nonce":1672057754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6831","nonce":1672057754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1672057754},{"type":"v1","addr":"172.21.15.193:6829","nonce":1672057754}]},"public_addr":"172.21.15.193:6825/1672057754","cluster_addr":"172.21.15.193:6827/1672057754","heartbeat_back_addr":"172.21.15.193:6831/1672057754","heartbeat_front_addr":"172.21.15.193:6829/1672057754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:04.579923+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:29.775779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:09:54.233110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:20.211984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:10:45.505275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:11.326920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:11:38.197783+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:03.690299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:28.707253+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:12:54.532610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:20.076497+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:13:45.503864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1976219591":"2024-09-22T03:07:59.080195+0000","172.21.15.83:0/3984963619":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2346813165":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6800/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:6800/493695771":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/1246429082":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/211211530":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/1905391534":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/817883806":"2024-09-22T03:06:36.366094+0000","172.21.15.83:0/3621693467":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/2962867597":"2024-09-22T03:07:01.129903+0000","172.21.15.83:0/3776769458":"2024-09-22T03:07:59.080195+0000","172.21.15.83:6801/3130587124":"2024-09-22T03:07:01.129903+0000","172.21.15.83:6801/493695771":"2024-09-22T03:07:59.080195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:33:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 msgr2=0x7f11c80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 0x7f11c80b8bc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11dc06c6c0 tx=0x7f11cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:33:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 msgr2=0x7f11dc19be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f11d400eaa0 tx=0x7f11d400ef60 comp rx=0 tx=0).stop 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 shutdown_connections 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f11c80b66b0 0x7f11c80b8bc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11dc108d70 0x7f11dc19be40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc108420 0x7f11dc19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.857+0000 7f11e2bc7640 1 --2- 172.21.15.83:0/674609539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11dc102ae0 0x7f11dc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.858+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 >> 172.21.15.83:0/674609539 conn(0x7f11dc079770 msgr2=0x7f11dc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.858+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 shutdown_connections 2024-09-21T03:33:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:22.858+0000 7f11e2bc7640 1 -- 172.21.15.83:0/674609539 wait complete. 2024-09-21T03:33:22.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:22 smithi142 ceph-mon[34512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T03:33:23.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:23 smithi083 ceph-mon[31219]: pgmap v848: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-21T03:33:23.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:23 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/674609539' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:33:23.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:33:23.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:23 smithi193 ceph-mon[35224]: pgmap v848: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-21T03:33:23.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:23 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/674609539' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:33:23.888 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:23.930 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:23 smithi142 ceph-mon[34512]: pgmap v848: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-21T03:33:23.930 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:23 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/674609539' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:33:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 -- 172.21.15.83:0/264926973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 msgr2=0x7fde1410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 --2- 172.21.15.83:0/264926973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1410f540 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009ca0 tx=0x7fde0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 -- 172.21.15.83:0/264926973 shutdown_connections 2024-09-21T03:33:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 --2- 172.21.15.83:0/264926973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1410f540 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 --2- 172.21.15.83:0/264926973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde141083c0 0x7fde141087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 --2- 172.21.15.83:0/264926973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde14102a80 0x7fde14102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 -- 172.21.15.83:0/264926973 >> 172.21.15.83:0/264926973 conn(0x7fde14079740 msgr2=0x7fde14079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 -- 172.21.15.83:0/264926973 shutdown_connections 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.447+0000 7fde1a50c640 1 -- 172.21.15.83:0/264926973 wait complete. 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 Processor -- start 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 -- start start 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde14102a80 0x7fde1419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 0x7fde1419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde14113ae0 con 0x7fde14108d10 2024-09-21T03:33:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde14113960 con 0x7fde141083c0 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde1a50c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fde14113c60 con 0x7fde14102a80 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde18aa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde18aa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48500/0 (socket says 172.21.15.83:48500) 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.448+0000 7fde18aa9640 1 -- 172.21.15.83:0/4276438741 learned_addr learned my addr 172.21.15.83:0/4276438741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde13fff640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde14102a80 0x7fde1419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde137fe640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 0x7fde1419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 -- 172.21.15.83:0/4276438741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde14102a80 msgr2=0x7fde1419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde14102a80 0x7fde1419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 -- 172.21.15.83:0/4276438741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 msgr2=0x7fde1419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 0x7fde1419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 -- 172.21.15.83:0/4276438741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde141a0420 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde137fe640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 0x7fde1419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde18aa9640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005cc0 tx=0x7fde0c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c003bf0 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c003d90 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde141a06b0 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c02fbc0 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.449+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde141a6e70 con 0x7fde14108d10 2024-09-21T03:33:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.450+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde14103d00 con 0x7fde14108d10 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.452+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fde0c02fd60 con 0x7fde14108d10 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.452+0000 7fde117fa640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 0x7fde000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.453+0000 7fde13fff640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 0x7fde000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.453+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8441+0+0 (secure 0 0 0) 0x7fde0c108cf0 con 0x7fde14108d10 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.453+0000 7fde13fff640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 0x7fde000b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fde04005fd0 tx=0x7fde04005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.457+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde0c0cf290 con 0x7fde14108d10 2024-09-21T03:33:24.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.674+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde1419b5e0 con 0x7fde14108d10 2024-09-21T03:33:24.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.676+0000 7fde117fa640 1 -- 172.21.15.83:0/4276438741 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7fde0c0098b0 con 0x7fde14108d10 2024-09-21T03:33:24.678 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:33:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.680+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 msgr2=0x7fde000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.680+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 0x7fde000b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fde04005fd0 tx=0x7fde04005850 comp rx=0 tx=0).stop 2024-09-21T03:33:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.680+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 msgr2=0x7fde1419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.680+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005cc0 tx=0x7fde0c02f380 comp rx=0 tx=0).stop 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 shutdown_connections 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fde000b6680 0x7fde000b8b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde14108d10 0x7fde1419fcf0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde141083c0 0x7fde1419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 --2- 172.21.15.83:0/4276438741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde14102a80 0x7fde1419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 >> 172.21.15.83:0/4276438741 conn(0x7fde14079740 msgr2=0x7fde14105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 shutdown_connections 2024-09-21T03:33:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:24.681+0000 7fde1a50c640 1 -- 172.21.15.83:0/4276438741 wait complete. 2024-09-21T03:33:25.065 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c[64498]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.065 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c[64498]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.065 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-mds[64502]: -- 172.21.15.193:0/3113637787 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbc1415c20 con 0x55dbc55bda80 2024-09-21T03:33:25.065 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c[64498]: starting mds.c at 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-mds[64219]: -- 172.21.15.193:0/1013765627 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb923c1860 con 0x55bb96569a80 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: starting mds.f at 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-mds[64772]: -- 172.21.15.193:0/2068138787 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596a4a39860 con 0x5596a8be1a80 2024-09-21T03:33:25.066 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: starting mds.i at 2024-09-21T03:33:25.067 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.067 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.067 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-mds[63950]: -- 172.21.15.193:0/3164902181 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a47c53860 con 0x559a4bdfb180 2024-09-21T03:33:25.067 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:33:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: starting mds.l at 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a[31215]: 2024-09-21T03:33:24.675+0000 7f95a3519640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: starting mds.g at 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a[74408]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a[74408]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.123 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-mds[74413]: --2- 172.21.15.83:0/1499373681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5565fd5ed600 0x5565fe1dc580 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x5565fd5feba0 tx=0x5565fd524540 comp rx=0 tx=0).write_message sending message m=0x5565fd4b2540 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T03:33:25.124 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-mds[74413]: -- 172.21.15.83:0/1499373681 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565f9445860 con 0x5565fd5ed600 2024-09-21T03:33:25.124 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:33:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a[74408]: starting mds.a at 2024-09-21T03:33:25.177 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.177 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.177 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-mds[62976]: -- 172.21.15.142:0/499688028 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56469d1bdc20 con 0x5646a1365600 2024-09-21T03:33:25.177 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: starting mds.b at 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-mds[62152]: -- 172.21.15.142:0/1849781502 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55675cda7860 con 0x556760f4f600 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: starting mds.k at 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h[62701]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h[62701]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.178 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h[62701]: starting mds.h at 2024-09-21T03:33:25.179 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e[62428]: ignoring --setuser ceph since I am not root 2024-09-21T03:33:25.179 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e[62428]: ignoring --setgroup ceph since I am not root 2024-09-21T03:33:25.179 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-mds[62432]: -- 172.21.15.142:0/3141149562 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559b39672ea0 con 0x559b3d811180 2024-09-21T03:33:25.179 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-mds[62432]: -- 172.21.15.142:0/3141149562 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b39669860 con 0x559b3d811180 2024-09-21T03:33:25.179 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:33:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e[62428]: starting mds.e at 2024-09-21T03:33:25.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:33:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: pgmap v849: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-09-21T03:33:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:33:25.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:33:25.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:33:25.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:33:25.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T03:33:25.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:25 smithi193 ceph-mon[35224]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: pgmap v849: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T03:33:25.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:25 smithi083 ceph-mon[31219]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:33:25.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: pgmap v849: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4276438741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T03:33:25.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:25 smithi142 ceph-mon[34512]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:33:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.395+0000 7fef2c835640 1 -- 172.21.15.83:0/2252161465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28108cf0 msgr2=0x7fef2810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.395+0000 7fef2c835640 1 --2- 172.21.15.83:0/2252161465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28108cf0 0x7fef2810f520 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fef20009ca0 tx=0x7fef2002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 -- 172.21.15.83:0/2252161465 shutdown_connections 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 --2- 172.21.15.83:0/2252161465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef28108cf0 0x7fef2810f520 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 --2- 172.21.15.83:0/2252161465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef281083a0 0x7fef281087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 --2- 172.21.15.83:0/2252161465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 0x7fef28102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 -- 172.21.15.83:0/2252161465 >> 172.21.15.83:0/2252161465 conn(0x7fef28079720 msgr2=0x7fef28079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 -- 172.21.15.83:0/2252161465 shutdown_connections 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 -- 172.21.15.83:0/2252161465 wait complete. 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.396+0000 7fef2c835640 1 Processor -- start 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 -- start start 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 0x7fef2819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 0x7fef2819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112960 con 0x7fef281083a0 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef281127e0 con 0x7fef28108cf0 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2c835640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112ae0 con 0x7fef28102a60 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48570/0 (socket says 172.21.15.83:48570) 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 -- 172.21.15.83:0/4048763464 learned_addr learned my addr 172.21.15.83:0/4048763464 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef2659c640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 0x7fef2819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef26d9d640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 0x7fef2819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 -- 172.21.15.83:0/4048763464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 msgr2=0x7fef2819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 0x7fef2819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 -- 172.21.15.83:0/4048763464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 msgr2=0x7fef2819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 0x7fef2819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 -- 172.21.15.83:0/4048763464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef281a03d0 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef26d9d640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 0x7fef2819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.397+0000 7fef25d9b640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00e9c0 tx=0x7fef1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.398+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00cd80 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.398+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c0054d0 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.398+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c010430 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.398+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef281a06c0 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.398+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a6e20 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.399+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef28103ce0 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.400+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fef1c004fb0 con 0x7fef281083a0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.401+0000 7fef177fe640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 0x7fef100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.401+0000 7fef2659c640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 0x7fef100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.401+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8827+0+0 (secure 0 0 0) 0x7fef1c0e7eb0 con 0x7fef281083a0 2024-09-21T03:33:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.401+0000 7fef2659c640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 0x7fef100b8b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef18009bd0 tx=0x7fef18005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.404+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef1c056710 con 0x7fef281083a0 2024-09-21T03:33:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:26.622+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fef2819b590 con 0x7fef281083a0 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: pgmap v851: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6838/1456598773,v1:172.21.15.83:6840/1456598773] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.83:6839/2895630631,v1:172.21.15.83:6841/2895630631] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6832/1934949583,v1:172.21.15.193:6834/1934949583] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6833/2877712992,v1:172.21.15.193:6835/2877712992] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6832/495312795,v1:172.21.15.142:6833/495312795] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6834/1793956760,v1:172.21.15.142:6835/1793956760] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6836/1955682022,v1:172.21.15.193:6837/1955682022] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.193:6838/4097240604,v1:172.21.15.193:6839/4097240604] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6836/3946833490,v1:172.21.15.142:6837/3946833490] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: mds.? [v2:172.21.15.142:6838/2249247190,v1:172.21.15.142:6839/2249247190] up:boot 2024-09-21T03:33:26.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:33:26.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:26 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:33:27.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: pgmap v851: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-21T03:33:27.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6838/1456598773,v1:172.21.15.83:6840/1456598773] up:boot 2024-09-21T03:33:27.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.83:6839/2895630631,v1:172.21.15.83:6841/2895630631] up:boot 2024-09-21T03:33:27.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6832/1934949583,v1:172.21.15.193:6834/1934949583] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6833/2877712992,v1:172.21.15.193:6835/2877712992] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6832/495312795,v1:172.21.15.142:6833/495312795] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6834/1793956760,v1:172.21.15.142:6835/1793956760] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6836/1955682022,v1:172.21.15.193:6837/1955682022] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.193:6838/4097240604,v1:172.21.15.193:6839/4097240604] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6836/3946833490,v1:172.21.15.142:6837/3946833490] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: mds.? [v2:172.21.15.142:6838/2249247190,v1:172.21.15.142:6839/2249247190] up:boot 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:33:27.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:33:27.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:33:27.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:33:27.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:33:27.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:33:27.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:26 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: pgmap v851: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6838/1456598773,v1:172.21.15.83:6840/1456598773] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.83:6839/2895630631,v1:172.21.15.83:6841/2895630631] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6832/1934949583,v1:172.21.15.193:6834/1934949583] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6833/2877712992,v1:172.21.15.193:6835/2877712992] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6832/495312795,v1:172.21.15.142:6833/495312795] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6834/1793956760,v1:172.21.15.142:6835/1793956760] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6836/1955682022,v1:172.21.15.193:6837/1955682022] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.193:6838/4097240604,v1:172.21.15.193:6839/4097240604] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6836/3946833490,v1:172.21.15.142:6837/3946833490] up:boot 2024-09-21T03:33:27.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: mds.? [v2:172.21.15.142:6838/2249247190,v1:172.21.15.142:6839/2249247190] up:boot 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='mgr.14217 172.21.15.83:0/2329401724' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:33:27.179 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:26 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:33:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.522+0000 7fef177fe640 1 -- 172.21.15.83:0/4048763464 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7fef1c0ae2d0 con 0x7fef281083a0 2024-09-21T03:33:27.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 msgr2=0x7fef100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:27.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 0x7fef100b8b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef18009bd0 tx=0x7fef18005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:27.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 msgr2=0x7fef2819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00e9c0 tx=0x7fef1c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 shutdown_connections 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fef100b6680 0x7fef100b8b90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef28108cf0 0x7fef2819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef281083a0 0x7fef2819aa00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 --2- 172.21.15.83:0/4048763464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef28102a60 0x7fef2819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.525+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 >> 172.21.15.83:0/4048763464 conn(0x7fef28079720 msgr2=0x7fef28101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.526+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 shutdown_connections 2024-09-21T03:33:27.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:27.526+0000 7fef2c835640 1 -- 172.21.15.83:0/4048763464 wait complete. 2024-09-21T03:33:27.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:27 smithi083 ceph-mon[31219]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:33:27.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:27 smithi083 ceph-mon[31219]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:33:27.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:27 smithi083 ceph-mon[31219]: Cluster is now healthy 2024-09-21T03:33:27.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:27 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:33:27.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:27 smithi083 ceph-mon[31219]: fsmap 12 up:standby 2024-09-21T03:33:28.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:27 smithi193 ceph-mon[35224]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:33:28.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:27 smithi193 ceph-mon[35224]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:33:28.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:27 smithi193 ceph-mon[35224]: Cluster is now healthy 2024-09-21T03:33:28.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:27 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:33:28.066 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:27 smithi193 ceph-mon[35224]: fsmap 12 up:standby 2024-09-21T03:33:28.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:27 smithi142 ceph-mon[34512]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:33:28.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:27 smithi142 ceph-mon[34512]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:33:28.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:27 smithi142 ceph-mon[34512]: Cluster is now healthy 2024-09-21T03:33:28.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:27 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/4048763464' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:33:28.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:27 smithi142 ceph-mon[34512]: fsmap 12 up:standby 2024-09-21T03:33:28.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:33:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:29.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3451730599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc581049e0 msgr2=0x7fcc58104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3451730599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc581049e0 0x7fcc58104df0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc4802a410 comp rx=0 tx=0).stop 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3451730599 shutdown_connections 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3451730599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc5810de40 0x7fcc58110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3451730599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc5810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3451730599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc581049e0 0x7fcc58104df0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3451730599 >> 172.21.15.83:0/3451730599 conn(0x7fcc58079750 msgr2=0x7fcc58079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3451730599 shutdown_connections 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3451730599 wait complete. 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.240+0000 7fcc5e18b640 1 Processor -- start 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 -- start start 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc581049e0 0x7fcc58102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc58102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58113aa0 con 0x7fcc5810de40 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58113920 con 0x7fcc581049e0 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc5e18b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58113c20 con 0x7fcc58105330 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49114/0 (socket says 172.21.15.83:49114) 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 -- 172.21.15.83:0/3941292086 learned_addr learned my addr 172.21.15.83:0/3941292086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc577fe640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc581049e0 0x7fcc58102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc56ffd640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc58102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 -- 172.21.15.83:0/3941292086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 msgr2=0x7fcc58102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc58102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 -- 172.21.15.83:0/3941292086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc581049e0 msgr2=0x7fcc58102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc581049e0 0x7fcc58102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc57fff640 1 -- 172.21.15.83:0/3941292086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc581a8770 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.241+0000 7fcc56ffd640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc58102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc57fff640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000eb10 tx=0x7fcc5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50004b20 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50004cc0 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50010670 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc581a8a60 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.242+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a8f50 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.243+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5806c280 con 0x7fcc5810de40 2024-09-21T03:33:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.244+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcc50005020 con 0x7fcc5810de40 2024-09-21T03:33:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.245+0000 7fcc54ff9640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 0x7fcc400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.245+0000 7fcc577fe640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 0x7fcc400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.245+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8827+0+0 (secure 0 0 0) 0x7fcc500e7ed0 con 0x7fcc5810de40 2024-09-21T03:33:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.245+0000 7fcc577fe640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 0x7fcc400b8b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009230 tx=0x7fcc48031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.250+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc500ae2f0 con 0x7fcc5810de40 2024-09-21T03:33:29.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.426+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc5802e640 con 0x7fcc5810de40 2024-09-21T03:33:29.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.530+0000 7fcc54ff9640 1 -- 172.21.15.83:0/3941292086 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7fcc500b26d0 con 0x7fcc5810de40 2024-09-21T03:33:29.532 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:33:29.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 msgr2=0x7fcc400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 0x7fcc400b8b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009230 tx=0x7fcc48031400 comp rx=0 tx=0).stop 2024-09-21T03:33:29.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 msgr2=0x7fcc581031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000eb10 tx=0x7fcc5000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 shutdown_connections 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7fcc400b6680 0x7fcc400b8b90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5810de40 0x7fcc581031a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc58105330 0x7fcc58102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 --2- 172.21.15.83:0/3941292086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc581049e0 0x7fcc58102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 >> 172.21.15.83:0/3941292086 conn(0x7fcc58079750 msgr2=0x7fcc58100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.533+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 shutdown_connections 2024-09-21T03:33:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:29.534+0000 7fcc5e18b640 1 -- 172.21.15.83:0/3941292086 wait complete. 2024-09-21T03:33:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:29 smithi193 ceph-mon[35224]: pgmap v852: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T03:33:29.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:29 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:29.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:29 smithi083 ceph-mon[31219]: pgmap v852: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T03:33:29.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:29 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:29.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:29 smithi142 ceph-mon[34512]: pgmap v852: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-21T03:33:29.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:29 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:30.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5108497e-77c6-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:33:30.497 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:30 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:30 smithi193 ceph-mon[35224]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T03:33:30.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:30 smithi193 ceph-mon[35224]: pgmap v854: 65 pgs: 65 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:30.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:30 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:30.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:30 smithi083 ceph-mon[31219]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T03:33:30.873 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:30 smithi083 ceph-mon[31219]: pgmap v854: 65 pgs: 65 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:30.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:30 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3941292086' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:30.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:30 smithi142 ceph-mon[34512]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T03:33:30.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:30 smithi142 ceph-mon[34512]: pgmap v854: 65 pgs: 65 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 -- 172.21.15.83:0/1178137327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8105380 msgr2=0x7f63c810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/1178137327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8105380 0x7f63c810d8d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009900 tx=0x7f63bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 -- 172.21.15.83:0/1178137327 shutdown_connections 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/1178137327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c810de90 0x7f63c8110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/1178137327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8105380 0x7f63c810d8d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/1178137327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c8104a30 0x7f63c8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 -- 172.21.15.83:0/1178137327 >> 172.21.15.83:0/1178137327 conn(0x7f63c80796b0 msgr2=0x7f63c8079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 -- 172.21.15.83:0/1178137327 shutdown_connections 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.975+0000 7f63cfe4a640 1 -- 172.21.15.83:0/1178137327 wait complete. 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 Processor -- start 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 -- start start 2024-09-21T03:33:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c8105380 0x7f63c819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c810de90 0x7f63c819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113af0 con 0x7f63c8104a30 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113970 con 0x7f63c8105380 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cfe4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113c70 con 0x7f63c810de90 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49120/0 (socket says 172.21.15.83:49120) 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 -- 172.21.15.83:0/3816085 learned_addr learned my addr 172.21.15.83:0/3816085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63ce3e7640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c810de90 0x7f63c819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cd3e5640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c8105380 0x7f63c819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 -- 172.21.15.83:0/3816085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c810de90 msgr2=0x7f63c819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c810de90 0x7f63c819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 -- 172.21.15.83:0/3816085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c8105380 msgr2=0x7f63c819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c8105380 0x7f63c819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.976+0000 7f63cdbe6640 1 -- 172.21.15.83:0/3816085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c81a0350 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63cdbe6640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f63b800cce0 tx=0x7f63b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b800f650 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c81a0640 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b800fca0 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8018b40 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.977+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a6e60 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.978+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c806c240 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.979+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f63b8018ce0 con 0x7f63c8104a30 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.980+0000 7f63c6ffd640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 0x7f63b00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.980+0000 7f63cd3e5640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 0x7f63b00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.980+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8302+0+0 (secure 0 0 0) 0x7f63b80e7d50 con 0x7f63c8104a30 2024-09-21T03:33:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.981+0000 7f63cd3e5640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 0x7f63b00b8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009230 tx=0x7f63bc031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:30.983+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63b8014030 con 0x7f63c8104a30 2024-09-21T03:33:31.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.163+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63c8104e40 con 0x7f63c8104a30 2024-09-21T03:33:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.543+0000 7f63c6ffd640 1 -- 172.21.15.83:0/3816085 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f63b80ae410 con 0x7f63c8104a30 2024-09-21T03:33:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-09-21T03:33:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.547+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 msgr2=0x7f63b00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.547+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 0x7f63b00b8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009230 tx=0x7f63bc031400 comp rx=0 tx=0).stop 2024-09-21T03:33:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.547+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 msgr2=0x7f63c819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.547+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f63b800cce0 tx=0x7f63b800a620 comp rx=0 tx=0).stop 2024-09-21T03:33:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 shutdown_connections 2024-09-21T03:33:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:6800/1298689131,v1:172.21.15.83:6801/1298689131] conn(0x7f63b00b66b0 0x7f63b00b8bc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63c810de90 0x7f63c819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c8105380 0x7f63c819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 --2- 172.21.15.83:0/3816085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c8104a30 0x7f63c819a4a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 >> 172.21.15.83:0/3816085 conn(0x7f63c80796b0 msgr2=0x7f63c8107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.548+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 shutdown_connections 2024-09-21T03:33:31.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-21T03:33:31.549+0000 7f63cfe4a640 1 -- 172.21.15.83:0/3816085 wait complete. 2024-09-21T03:33:31.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:31 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:31.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:31 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:31.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:31 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:33:32.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:33:32.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:33:32.501 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:33:32.549 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:33:32.550 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:33:32.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:32 smithi083 ceph-mon[31219]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:32.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:32 smithi083 ceph-mon[31219]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T03:33:32.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:32 smithi083 ceph-mon[31219]: pgmap v856: 1 pgs: 1 active+clean; 577 KiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:32.590 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:33:32.620 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:33:32.726 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:33:32.727 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:33:32.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:33:32.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:32 smithi193 ceph-mon[35224]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:32.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:32 smithi193 ceph-mon[35224]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T03:33:32.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:32 smithi193 ceph-mon[35224]: pgmap v856: 1 pgs: 1 active+clean; 577 KiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:32.872 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:32 smithi083 systemd[1]: Stopping Ceph mon.a for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:32.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:32 smithi142 ceph-mon[34512]: from='client.? 172.21.15.83:0/3816085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:33:32.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:32 smithi142 ceph-mon[34512]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T03:33:32.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:32 smithi142 ceph-mon[34512]: pgmap v856: 1 pgs: 1 active+clean; 577 KiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:33:33.597 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a[31215]: 2024-09-21T03:33:33.269+0000 7f95a8d24640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:33.597 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a[31215]: 2024-09-21T03:33:33.269+0000 7f95a8d24640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:33:33.597 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:33 smithi083 podman[114090]: 2024-09-21 03:33:33.313624431 +0000 UTC m=+0.199226409 container died 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:33:34.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:33 smithi083 podman[114090]: 2024-09-21 03:33:33.886678992 +0000 UTC m=+0.772280971 container cleanup 9561cf4f54311550c14c0350676d60faea97af49f9fca7138ede8324bb8e8970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:33:34.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 21 03:33:33 smithi083 bash[114090]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:33:34.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:33:34.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:34.604 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:33:34.604 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:33:34.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:33:34.928 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:34 smithi142 systemd[1]: Stopping Ceph mon.b for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:35.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-b[34508]: 2024-09-21T03:33:34.974+0000 7efec0be4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:35.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-b[34508]: 2024-09-21T03:33:34.974+0000 7efec0be4640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:33:35.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:35 smithi142 podman[67524]: 2024-09-21 03:33:35.158050883 +0000 UTC m=+0.308028573 container died c3a7274f2e34863ae9ace9664e4467f3bf3f76fd6baaba6ea2dc2da3919a6cb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:33:35.828 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:35 smithi142 podman[67524]: 2024-09-21 03:33:35.525147004 +0000 UTC m=+0.675124691 container cleanup c3a7274f2e34863ae9ace9664e4467f3bf3f76fd6baaba6ea2dc2da3919a6cb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:33:35.828 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 21 03:33:35 smithi142 bash[67524]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:33:35.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:33:36.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:36.002 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:33:36.002 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:33:36.002 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:33:36.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 systemd[1]: Stopping Ceph mon.c for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:36.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-c[35220]: 2024-09-21T03:33:36.387+0000 7f95a3b18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:36.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-c[35220]: 2024-09-21T03:33:36.387+0000 7f95a3b18640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:33:36.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 podman[68848]: 2024-09-21 03:33:36.460365408 +0000 UTC m=+0.216894979 container died e19f79293868cba4d3df6bc66e0db85574a81cb0341530e99a8cc1d66305e086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:33:37.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 podman[68848]: 2024-09-21 03:33:36.889145659 +0000 UTC m=+0.645675218 container cleanup e19f79293868cba4d3df6bc66e0db85574a81cb0341530e99a8cc1d66305e086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:33:37.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 03:33:36 smithi193 bash[68848]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:33:37.392 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:33:37.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:37.443 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:33:37.443 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:33:37.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:33:37.812 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:33:37 smithi083 systemd[1]: Stopping Ceph mgr.x for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:38.077 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:33:37 smithi083 podman[114212]: 2024-09-21 03:33:37.818165579 +0000 UTC m=+0.169651199 container died f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:33:38.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:33:38 smithi083 podman[114212]: 2024-09-21 03:33:38.335350748 +0000 UTC m=+0.686836375 container cleanup f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:33:38.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:33:38 smithi083 bash[114212]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:33:38.623 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Sep 21 03:33:38 smithi083 podman[114226]: 2024-09-21 03:33:38.579116444 +0000 UTC m=+0.760135358 container remove f7ed1f718536b576f4a1d6805c347031ef27cc54aa63f0d6fe71904c07113b1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:33:38.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:33:38.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:38.869 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:33:38.869 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:33:38.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:33:39.177 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:38 smithi142 systemd[1]: Stopping Ceph mgr.y for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:39.783 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:39 smithi142 podman[67646]: 2024-09-21 03:33:39.444794053 +0000 UTC m=+0.377678506 container died d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:33:40.177 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:39 smithi142 podman[67646]: 2024-09-21 03:33:39.897948886 +0000 UTC m=+0.830833346 container cleanup d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:33:40.177 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:39 smithi142 bash[67646]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:33:40.466 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:40 smithi142 podman[67658]: 2024-09-21 03:33:40.207691377 +0000 UTC m=+0.759939496 container remove d3572e5ba61709d157b949a30a5e80468623769ccdc0ea6a6d62fd0e4fbdd4e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:33:40.466 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 21 03:33:40 smithi142 systemd[1]: ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-21T03:33:40.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:33:40.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:40.506 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:33:40.506 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:33:40.506 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:33:40.815 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:33:40 smithi193 systemd[1]: Stopping Ceph mgr.z for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:42.799 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:33:42 smithi193 podman[68970]: 2024-09-21 03:33:42.385174241 +0000 UTC m=+1.686231917 container died 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:33:43.065 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:33:42 smithi193 podman[68970]: 2024-09-21 03:33:42.9061015 +0000 UTC m=+2.207159170 container cleanup 310d420b04b9e6ddd1806f76d8677181e67732f9595641c57ff605537ebb065f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:33:43.065 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 03:33:42 smithi193 bash[68970]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:33:43.383 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:33:43.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:43.418 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:33:43.418 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:33:43.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:33:43.726 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:43 smithi083 systemd[1]: Stopping Ceph osd.0 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:44.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:33:43.724+0000 7f29b8b2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:44.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:33:43.724+0000 7f29b8b2e640 -1 osd.0 102 *** Got signal Terminated *** 2024-09-21T03:33:44.123 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0[48061]: 2024-09-21T03:33:43.724+0000 7f29b8b2e640 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:33:49.743 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:49 smithi083 podman[114343]: 2024-09-21 03:33:49.423809699 +0000 UTC m=+5.814002790 container died 7112a04ac180520fcfcec3dd1f7853707a10d7b3a97f655427804c93d4342be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:33:50.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:49 smithi083 podman[114343]: 2024-09-21 03:33:49.917096184 +0000 UTC m=+6.307289267 container cleanup 7112a04ac180520fcfcec3dd1f7853707a10d7b3a97f655427804c93d4342be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:33:50.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:49 smithi083 bash[114343]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:33:50.495 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114360]: 2024-09-21 03:33:50.185840369 +0000 UTC m=+0.762523475 container remove 7112a04ac180520fcfcec3dd1f7853707a10d7b3a97f655427804c93d4342be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:33:50.824 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.396587538 +0000 UTC m=+0.016035320 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:33:50.824 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.511813491 +0000 UTC m=+0.131261257 container create 534beee3b2dae778594e279fe74a5455280d1c7a3e7de70268b962bca9f1e7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:33:50.824 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.739563323 +0000 UTC m=+0.359011099 container init 534beee3b2dae778594e279fe74a5455280d1c7a3e7de70268b962bca9f1e7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:33:50.824 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.742751053 +0000 UTC m=+0.362198829 container start 534beee3b2dae778594e279fe74a5455280d1c7a3e7de70268b962bca9f1e7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:33:50.824 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.822948623 +0000 UTC m=+0.442396402 container attach 534beee3b2dae778594e279fe74a5455280d1c7a3e7de70268b962bca9f1e7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:33:51.122 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 21 03:33:50 smithi083 podman[114450]: 2024-09-21 03:33:50.97776773 +0000 UTC m=+0.597215496 container died 534beee3b2dae778594e279fe74a5455280d1c7a3e7de70268b962bca9f1e7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:33:51.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:33:51.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:51.683 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:33:51.683 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:33:51.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:33:52.004 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:51 smithi142 systemd[1]: Stopping Ceph osd.1 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:33:52.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:33:52.001+0000 7f5ce6d1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:33:52.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:33:52.001+0000 7f5ce6d1a640 -1 osd.1 102 *** Got signal Terminated *** 2024-09-21T03:33:52.428 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1[39268]: 2024-09-21T03:33:52.001+0000 7f5ce6d1a640 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:33:57.904 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:57 smithi142 podman[67784]: 2024-09-21 03:33:57.595096038 +0000 UTC m=+5.714843898 container died 03cebe8bf29dbaaa1e3f87b4825b10f1e1be2964cd77eb2d0fb97c98cf6d0b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:33:58.178 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67784]: 2024-09-21 03:33:58.010679029 +0000 UTC m=+6.130426889 container cleanup 03cebe8bf29dbaaa1e3f87b4825b10f1e1be2964cd77eb2d0fb97c98cf6d0b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:33:58.178 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 bash[67784]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:33:58.564 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67799]: 2024-09-21 03:33:58.253911709 +0000 UTC m=+0.654461302 container remove 03cebe8bf29dbaaa1e3f87b4825b10f1e1be2964cd77eb2d0fb97c98cf6d0b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T03:33:58.867 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67889]: 2024-09-21 03:33:58.466026728 +0000 UTC m=+0.015498490 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:33:58.867 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67889]: 2024-09-21 03:33:58.60478944 +0000 UTC m=+0.154261191 container create 4c49dc882aed9c95556fed2f88700c4a1940704857095deb5d6d0f0ae894e985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T03:33:59.124 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67889]: 2024-09-21 03:33:58.865782965 +0000 UTC m=+0.415254714 container init 4c49dc882aed9c95556fed2f88700c4a1940704857095deb5d6d0f0ae894e985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:33:59.124 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67889]: 2024-09-21 03:33:58.869306238 +0000 UTC m=+0.418777993 container start 4c49dc882aed9c95556fed2f88700c4a1940704857095deb5d6d0f0ae894e985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:33:59.124 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:58 smithi142 podman[67889]: 2024-09-21 03:33:58.99133557 +0000 UTC m=+0.540807401 container attach 4c49dc882aed9c95556fed2f88700c4a1940704857095deb5d6d0f0ae894e985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:33:59.428 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 21 03:33:59 smithi142 podman[67889]: 2024-09-21 03:33:59.123499631 +0000 UTC m=+0.672971391 container died 4c49dc882aed9c95556fed2f88700c4a1940704857095deb5d6d0f0ae894e985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:33:59.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:33:59.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:33:59.819 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:33:59.819 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:33:59.819 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:34:00.145 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:33:59 smithi193 systemd[1]: Stopping Ceph osd.2 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:00.566 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:34:00.143+0000 7fb5ee640640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:00.566 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:34:00.143+0000 7fb5ee640640 -1 osd.2 102 *** Got signal Terminated *** 2024-09-21T03:34:00.566 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2[40235]: 2024-09-21T03:34:00.143+0000 7fb5ee640640 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:06.065 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:05 smithi193 podman[69106]: 2024-09-21 03:34:05.790790747 +0000 UTC m=+5.777005335 container died 3da7ea20fb52a36e6cd80dc07a6dfd8941cdceddafff24fbf7d35a25932997b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:34:06.472 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:06 smithi193 podman[69106]: 2024-09-21 03:34:06.194186207 +0000 UTC m=+6.180400794 container cleanup 3da7ea20fb52a36e6cd80dc07a6dfd8941cdceddafff24fbf7d35a25932997b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:34:06.472 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:06 smithi193 bash[69106]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:34:06.811 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:06 smithi193 podman[69121]: 2024-09-21 03:34:06.471040622 +0000 UTC m=+0.678671583 container remove 3da7ea20fb52a36e6cd80dc07a6dfd8941cdceddafff24fbf7d35a25932997b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:34:07.065 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:06 smithi193 podman[69213]: 2024-09-21 03:34:06.713072247 +0000 UTC m=+0.016226931 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:07.065 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:06 smithi193 podman[69213]: 2024-09-21 03:34:06.830483359 +0000 UTC m=+0.133638033 container create 3d873a2f5429a274ad546f5f7530e6d7db2c90006fc6519783505a6f19250e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:07.337 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:07 smithi193 podman[69213]: 2024-09-21 03:34:07.096318584 +0000 UTC m=+0.399473260 container init 3d873a2f5429a274ad546f5f7530e6d7db2c90006fc6519783505a6f19250e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:34:07.338 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:07 smithi193 podman[69213]: 2024-09-21 03:34:07.100123119 +0000 UTC m=+0.403277797 container start 3d873a2f5429a274ad546f5f7530e6d7db2c90006fc6519783505a6f19250e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T03:34:07.338 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:07 smithi193 podman[69213]: 2024-09-21 03:34:07.200243484 +0000 UTC m=+0.503398167 container attach 3d873a2f5429a274ad546f5f7530e6d7db2c90006fc6519783505a6f19250e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:34:07.796 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 03:34:07 smithi193 podman[69213]: 2024-09-21 03:34:07.33654427 +0000 UTC m=+0.639698948 container died 3d873a2f5429a274ad546f5f7530e6d7db2c90006fc6519783505a6f19250e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T03:34:08.126 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:34:08.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:08.174 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:34:08.174 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:34:08.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:34:08.499 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:08 smithi083 systemd[1]: Stopping Ceph osd.3 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:08.757 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:08 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:34:08.496+0000 7fbc73ebf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:08.757 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:08 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:34:08.496+0000 7fbc73ebf640 -1 osd.3 102 *** Got signal Terminated *** 2024-09-21T03:34:08.757 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:08 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3[54598]: 2024-09-21T03:34:08.496+0000 7fbc73ebf640 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:10.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:10 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:10.118+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:10.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:10 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:10.516+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:10.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:10.663+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:11.550 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:11 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:11.155+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:11.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:11 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:11.548+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:11.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:11.655+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:12.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:11.764+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:12.588 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:12 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:12.141+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:12.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:12 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:12.586+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:12.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:12 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:12.624+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:13.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:12.752+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:13.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:13 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:13.103+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:13.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:13.104+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:13.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:13 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:13.582+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:13.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:13 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:13.619+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:13.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:14.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:14 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:14.070+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:14.373 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:14 smithi083 podman[114565]: 2024-09-21 03:34:14.090623133 +0000 UTC m=+5.722343134 container died ef482d744698568c28faedcc21bf866f5693e33372ecf1c50c2a51da304ddc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:34:14.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:14.129+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:14.851 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:14 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:14.579+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:14.851 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:14 smithi083 podman[114565]: 2024-09-21 03:34:14.548016743 +0000 UTC m=+6.179736796 container cleanup ef482d744698568c28faedcc21bf866f5693e33372ecf1c50c2a51da304ddc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:14.851 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:14 smithi083 bash[114565]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:34:14.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:14 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:14.612+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:14.727+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:15.123 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:14 smithi083 podman[114593]: 2024-09-21 03:34:14.849729737 +0000 UTC m=+0.760297981 container remove ef482d744698568c28faedcc21bf866f5693e33372ecf1c50c2a51da304ddc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:15.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:15 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:15.054+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:15.496 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.111025627 +0000 UTC m=+0.015490748 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:15.497 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.217376338 +0000 UTC m=+0.121841447 container create c6c356472cf4a7c3e4ab17640afe353b1aa0286ad87f5dfedb5cb86cb27baa93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:15.497 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.495442517 +0000 UTC m=+0.399907641 container init c6c356472cf4a7c3e4ab17640afe353b1aa0286ad87f5dfedb5cb86cb27baa93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:15.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:15.136+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:15.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:15 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:15.613+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:15.873 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.499099995 +0000 UTC m=+0.403565106 container start c6c356472cf4a7c3e4ab17640afe353b1aa0286ad87f5dfedb5cb86cb27baa93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:34:15.873 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.586817991 +0000 UTC m=+0.491283127 container attach c6c356472cf4a7c3e4ab17640afe353b1aa0286ad87f5dfedb5cb86cb27baa93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:34:15.873 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 21 03:34:15 smithi083 podman[114683]: 2024-09-21 03:34:15.690725523 +0000 UTC m=+0.595190639 container died c6c356472cf4a7c3e4ab17640afe353b1aa0286ad87f5dfedb5cb86cb27baa93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:34:15.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:15 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:15.590+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:15.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:16.311 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:16 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:16.034+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:16.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:34:16.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:16.496 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:34:16.496 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:34:16.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:34:16.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:16.183+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:16.927 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:16 smithi142 systemd[1]: Stopping Ceph osd.4 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:16.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:16 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:16.601+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:17.030 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:16 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:16.660+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:17.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:16.701+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:17.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:17 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:17.027+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:17.427 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:16 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:16.973+0000 7fb6da207640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:17.428 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:16 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:16.973+0000 7fb6da207640 -1 osd.4 102 *** Got signal Terminated *** 2024-09-21T03:34:17.428 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:16 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:16.973+0000 7fb6da207640 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:17.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:17.211+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:17.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:17 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:17.574+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:18.019 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:17 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:17.702+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:18.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:17.712+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:18.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:18 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:18.016+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:18.566 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:18.239+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:18.566 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:18.241+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:18.829 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:18 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:18.696+0000 7fb6d67f2640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.352694+0000 front 2024-09-21T03:33:54.352724+0000 (oldest deadline 2024-09-21T03:34:17.852858+0000) 2024-09-21T03:34:18.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:18 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:18.545+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:19.016 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:18 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:18.689+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:19.016 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:18 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:18.689+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:19.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:18.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:19.097 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:18 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:18.846+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:19.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:19 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:19.013+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:19.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:19 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:19.013+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:19.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:19 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:19.240+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:19.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:19 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:19.216+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:19.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:19 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:19.216+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:19.877 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:19 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:19.647+0000 7fb6d67f2640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.352694+0000 front 2024-09-21T03:33:54.352724+0000 (oldest deadline 2024-09-21T03:34:17.852858+0000) 2024-09-21T03:34:19.878 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:19 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:19.543+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:19.878 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:19 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:19.543+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:20.026 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:19 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:19.707+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:20.026 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:19 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:19.707+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:20.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:19 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:19.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:20.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:19 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:19.875+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:20.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:20 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:20.024+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:20.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:20 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:20.024+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:20.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:20 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:20.256+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:20.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:20 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:20.168+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:20.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:20 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:20.168+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:20.927 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:20.612+0000 7fb6d67f2640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.352694+0000 front 2024-09-21T03:33:54.352724+0000 (oldest deadline 2024-09-21T03:34:17.852858+0000) 2024-09-21T03:34:20.928 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:20.834+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:20.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:20.589+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:20.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:20 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:20.589+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:21.021 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:20 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:20.755+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:21.021 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:20 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:20.755+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:20 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:20.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:20 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:20.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:21.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:21.018+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:21.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:21.018+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:21.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:21 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:21.246+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:21.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:21 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:21.174+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:21.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:21 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:21.174+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:21.927 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:21 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4[45680]: 2024-09-21T03:34:21.600+0000 7fb6d67f2640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.352694+0000 front 2024-09-21T03:33:54.352724+0000 (oldest deadline 2024-09-21T03:34:17.852858+0000) 2024-09-21T03:34:21.928 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:21 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:21.810+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:21.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:21 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:21.623+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:21.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:21 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:21.623+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:22.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:21 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:21.604+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:22.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:21 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:21.604+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:22.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:21.978+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:22.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:21.978+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:22.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:21.773+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:22.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:21 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:21.773+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:22.563 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:22 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:22.246+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:22.563 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:22 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:22.215+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:22.563 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:22 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:22.215+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:22.806 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:22 smithi142 podman[67996]: 2024-09-21 03:34:22.524356207 +0000 UTC m=+5.834983948 container died 6368d8d0c4a3f2e1338fce63126cbfbbb011f9b9babbd357f8e13c35d39dd124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:34:22.807 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:22 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:22.576+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:22.807 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:22 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:22.576+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:22.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:22 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:22.561+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:22.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:22 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:22.561+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:23.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:22.948+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:23.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:22.948+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:23.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:22.818+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:23.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:22.818+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:23.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:22 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:22.804+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:23.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:22 smithi142 podman[67996]: 2024-09-21 03:34:22.998258478 +0000 UTC m=+6.308886216 container cleanup 6368d8d0c4a3f2e1338fce63126cbfbbb011f9b9babbd357f8e13c35d39dd124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:34:23.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:22 smithi142 bash[67996]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:34:23.544 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:23 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:23.223+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:23.545 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:23 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:23.172+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:23.545 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:23 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:23.172+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:23.604 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68024]: 2024-09-21 03:34:23.299935682 +0000 UTC m=+0.777552000 container remove 6368d8d0c4a3f2e1338fce63126cbfbbb011f9b9babbd357f8e13c35d39dd124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T03:34:23.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:23 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:23.543+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:23.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:23 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:23.543+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:23.896 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68117]: 2024-09-21 03:34:23.510276103 +0000 UTC m=+0.016071433 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:23.896 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68117]: 2024-09-21 03:34:23.634164619 +0000 UTC m=+0.139959922 container create 87e61615769326950881ea7b0ef0dfa6438acd9f5ba2104c78460963a93b88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:34:23.896 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:23 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:23.815+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:23.897 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:23 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:23.601+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:23.897 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:23 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:23.601+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:24.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:23 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:23.950+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:24.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:23 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:23.950+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:24.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:23 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:23.824+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:24.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:23 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:23.824+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:24.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68117]: 2024-09-21 03:34:23.894959858 +0000 UTC m=+0.400755165 container init 87e61615769326950881ea7b0ef0dfa6438acd9f5ba2104c78460963a93b88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:24.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68117]: 2024-09-21 03:34:23.898429151 +0000 UTC m=+0.404224464 container start 87e61615769326950881ea7b0ef0dfa6438acd9f5ba2104c78460963a93b88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T03:34:24.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:23 smithi142 podman[68117]: 2024-09-21 03:34:23.995231935 +0000 UTC m=+0.501027244 container attach 87e61615769326950881ea7b0ef0dfa6438acd9f5ba2104c78460963a93b88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:34:24.178 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 21 03:34:24 smithi142 podman[68117]: 2024-09-21 03:34:24.083796488 +0000 UTC m=+0.589591791 container died 87e61615769326950881ea7b0ef0dfa6438acd9f5ba2104c78460963a93b88ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:24.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:24.187+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:24.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:24.173+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:24.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:24.173+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:24.718 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:24.634+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:24.718 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:24.634+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:24.735 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:34:24.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:24.775 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:34:24.775 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:34:24.775 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:34:24.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:24.551+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:24.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:24 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:24.551+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:25.065 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:24 smithi193 systemd[1]: Stopping Ceph osd.5 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:25.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:24.919+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:25.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:24.919+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:25.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:24.800+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:25.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:24 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:24.800+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:25.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:24 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:24.829+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:25.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:25.165+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:25.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:25.217+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:25.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:25.217+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:25.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:25.231+0000 7f907e2b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:25.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:25.231+0000 7f907e2b3640 -1 osd.5 102 *** Got signal Terminated *** 2024-09-21T03:34:25.566 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:25.231+0000 7f907e2b3640 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:25.927 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:25 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:25.844+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:25.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:25 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:25.662+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:25.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:25 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:25.662+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:26.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:25.594+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:26.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:25 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:25.594+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:26.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:25.957+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:26.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:25.957+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:26.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:25.779+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:26.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:25.779+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:26.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:26 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:26.124+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:26.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:26 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:26.192+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:26.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:26 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:26.192+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:26.927 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:26 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:26.868+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:26.928 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:26 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:26.868+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:26.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:26 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:26.631+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:26.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:26 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:26.631+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:26.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:26 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:26.631+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:27.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:26 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:26.601+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:27.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:26 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:26.601+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:27.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:26 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:26.942+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:27.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:26 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:26.942+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:27.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:26 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:26.762+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:27.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:26 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:26.762+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:27.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:27.226+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:27.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:27.226+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:27.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:27.148+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:27.566 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:27.148+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:27.879 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:27 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:27.584+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:27.879 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:27 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:27.584+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:27.879 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:27 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:27.584+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:28.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:27.578+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:28.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:27 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:27.578+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:28.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:27 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:27.978+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:28.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:27 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:27.978+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:28.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:27 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:27.760+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:28.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:27 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:27.760+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:28.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:27 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:27.877+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:28.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:27 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:27.877+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:28.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:28.249+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:28.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:28.249+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:28.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:28.159+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:28.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:28.159+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:28.861 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:28.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:28.861 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:28.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:28.861 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:28.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:29.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:28.597+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:29.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:28 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:28.597+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:29.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:28 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:28.943+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:29.123 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:28 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:28.943+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:29.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:28 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:28.723+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:29.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:28 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:28.723+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:29.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:28 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:28.723+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:29.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:28.859+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:29.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:28 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:28.859+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:29.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:29.262+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.852306+0000 front 2024-09-21T03:33:46.852280+0000 (oldest deadline 2024-09-21T03:34:12.752336+0000) 2024-09-21T03:34:29.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5[46782]: 2024-09-21T03:34:29.262+0000 7f907a89e640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:52.752832+0000 front 2024-09-21T03:33:52.752906+0000 (oldest deadline 2024-09-21T03:34:18.652945+0000) 2024-09-21T03:34:29.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:29.174+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:29.566 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:29.174+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:29.899 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:29 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:29.536+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:29.900 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:29 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:29.536+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:29.900 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:29 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:29.536+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:29.971 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:29.711+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:29.971 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:29.711+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:29.971 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:29.711+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:30.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:29.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:30.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:29.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:30.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:29 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:29.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:30.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:29 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:29.897+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:30.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:29 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:29.897+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:30.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:29.969+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:30.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:29.969+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:30.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:29 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:29.969+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:30.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:30 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:30.148+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:30.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:30 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:30.148+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:30.896 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:30.559+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:30.896 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:30.559+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:30.896 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:30.559+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:30.921 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:30 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:30.634+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:30.921 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:30 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:30.634+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:30.921 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:30 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:30.634+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:30.946 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:30.679+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:30.947 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:30.679+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:30.947 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:30.679+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:31.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:30.894+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:31.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:30 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:30.894+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:31.236 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:30 smithi193 podman[69328]: 2024-09-21 03:34:30.920432184 +0000 UTC m=+5.943226618 container died 40cc946c838ee37fe6bd4e5ec4aa46c9b93ea45053eba6f9b03adb84e66c37c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:34:31.237 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:31 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:31.105+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:31.237 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:31 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:31.105+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:31.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:30.945+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:31.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:30.945+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:31.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:30 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:30.945+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:31.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:31 smithi193 podman[69328]: 2024-09-21 03:34:31.333185876 +0000 UTC m=+6.355980307 container cleanup 40cc946c838ee37fe6bd4e5ec4aa46c9b93ea45053eba6f9b03adb84e66c37c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:34:31.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:31 smithi193 bash[69328]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:34:31.853 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:31 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:31.593+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:31.854 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:31 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:31.593+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:31.854 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:31 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:31.593+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:31.914 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:31.642+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:31.914 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:31.642+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:31.914 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:31.642+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:32.003 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:31 smithi193 podman[69345]: 2024-09-21 03:34:31.601585487 +0000 UTC m=+0.678754355 container remove 40cc946c838ee37fe6bd4e5ec4aa46c9b93ea45053eba6f9b03adb84e66c37c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:34:32.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:31 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:31.632+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:32.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:31 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:31.632+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:32.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:31 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:31.632+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:32.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:31 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:31.851+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:32.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:31 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:31.851+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:32.289 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:32 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:32.125+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:32.290 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:32 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:32.125+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:32.290 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:31.905214161 +0000 UTC m=+0.015734193 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:32.290 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:32.027466362 +0000 UTC m=+0.137986392 container create 3979809a7c15a1042084b6d2c66c71ae28102af6c118f1dad116dc1073472db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:34:32.290 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:32.288599685 +0000 UTC m=+0.399119719 container init 3979809a7c15a1042084b6d2c66c71ae28102af6c118f1dad116dc1073472db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:34:32.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:31.912+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:32.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:31.912+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:32.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:31.912+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:32.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:32.291963815 +0000 UTC m=+0.402483851 container start 3979809a7c15a1042084b6d2c66c71ae28102af6c118f1dad116dc1073472db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:34:32.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:32.397151833 +0000 UTC m=+0.507671882 container attach 3979809a7c15a1042084b6d2c66c71ae28102af6c118f1dad116dc1073472db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:34:32.565 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 03:34:32 smithi193 podman[69434]: 2024-09-21 03:34:32.484232438 +0000 UTC m=+0.594752469 container died 3979809a7c15a1042084b6d2c66c71ae28102af6c118f1dad116dc1073472db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:34:32.889 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:32 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:32.580+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:32.889 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:32 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:32.580+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:32.889 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:32 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:32.580+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:32.942 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:32.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:32.942 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:32.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:32.942 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:32.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:33.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:32 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:32.604+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:33.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:32 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:32.604+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:33.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:32 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:32.604+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:33.129 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:34:33.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:32 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:32.887+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:33.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:32 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:32.887+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:33.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:33.209 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:34:33.209 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:34:33.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:34:33.236 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:32.940+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:33.236 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:32.940+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:33.236 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:32 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:32.940+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:33.538 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 systemd[1]: Stopping Ceph osd.6 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:33.538 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.535+0000 7f2b102d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:33.538 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.535+0000 7f2b102d9640 -1 osd.6 102 *** Got signal Terminated *** 2024-09-21T03:34:33.538 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.535+0000 7f2b102d9640 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:33.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:33 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:33.119+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:33.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:33 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:33.119+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:33.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:33.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:33.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:33.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:33.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:33.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:33.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:33 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:33.589+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:33.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:33 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:33.589+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:33.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:33 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:33.589+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:34.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:33 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:33.621+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:34.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:33 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:33.621+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:34.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:33 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:33.621+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:34.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.983+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:34.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.983+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:34.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:33.983+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:34.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:33 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:33.930+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:34.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:33 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:33.930+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:34.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:34 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:34.162+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:34.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:34 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:34.162+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:34.923 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:34.585+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:34.923 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:34.585+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:34.923 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:34.585+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:35.030 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:34 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:34.625+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:35.030 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:34 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:34.625+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:35.031 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:34 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:34.625+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:35.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:34 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:34.610+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:35.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:34 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:34.610+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:35.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:34 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:34.610+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:35.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:34.922+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:35.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:34.922+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:35.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:34 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:34.922+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:35.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:35.028+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:35.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:35.028+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:35.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:35.028+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:35.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:35.200+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:35.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:35.200+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:35.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:35.617+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:35.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:35.617+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:35.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:35 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:35.617+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:35.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:35.615+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:35.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:35.615+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:35.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:35.615+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:35.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:35.615+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:36.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:35.593+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:36.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:35.593+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:36.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:35.593+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:36.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:35 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:35.593+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:36.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:36.071+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:36.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:36.071+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:36.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:36.071+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:36.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:36.071+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.523082+0000 front 2024-09-21T03:34:12.523151+0000 (oldest deadline 2024-09-21T03:34:36.023229+0000) 2024-09-21T03:34:36.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:35.943+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:36.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:35.943+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:36.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:35 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:35.943+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:36.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:36.247+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:36.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:36.247+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:36.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:36.608+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:36.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:36.608+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:36.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:36.608+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:36.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:36.608+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:37.036 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:36.654+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:37.036 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:36.654+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:37.036 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:36 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:36.654+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:37.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:36.573+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:37.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:36.573+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:37.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:36.573+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:37.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:36 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:36.573+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:37.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:37.034+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:37.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:37.034+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:37.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:37.034+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:37.373 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:37.034+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.523082+0000 front 2024-09-21T03:34:12.523151+0000 (oldest deadline 2024-09-21T03:34:36.023229+0000) 2024-09-21T03:34:37.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:36.984+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:37.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:36.984+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:37.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:36 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:36.984+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:37.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:37.247+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:37.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:37.247+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:37.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:37.247+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:37.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:37 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:37.577+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:37.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:37 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:37.577+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:37.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:37 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:37.577+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:37.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:37 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:37.577+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:38.046 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:37.666+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:38.046 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:37.666+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:38.046 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:37.666+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:38.046 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:37 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:37.666+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:38.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:37.571+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:38.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:37.571+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:38.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:37.571+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:38.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:37 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:37.571+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:38.317 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:38.043+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:43.518354+0000 front 2024-09-21T03:33:43.518332+0000 (oldest deadline 2024-09-21T03:34:09.418438+0000) 2024-09-21T03:34:38.317 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:38.043+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.619992+0000 front 2024-09-21T03:33:54.619863+0000 (oldest deadline 2024-09-21T03:34:18.119678+0000) 2024-09-21T03:34:38.317 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:38.043+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.722462+0000 front 2024-09-21T03:34:03.722314+0000 (oldest deadline 2024-09-21T03:34:29.622181+0000) 2024-09-21T03:34:38.318 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6[61011]: 2024-09-21T03:34:38.043+0000 7f2b088bc640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.523082+0000 front 2024-09-21T03:34:12.523151+0000 (oldest deadline 2024-09-21T03:34:36.023229+0000) 2024-09-21T03:34:38.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:38.000+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:38.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:38.000+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:38.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:38.000+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:38.539 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:38.271+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:38.539 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:38.271+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:38.539 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:38.271+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:38.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:38.536+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:38.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:38.536+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:38.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:38.536+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:38.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:38 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:38.536+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:38.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:38.618+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:38.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:38.618+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:38.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:38.618+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:38.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:38 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:38.618+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:38.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:38.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:38.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:38.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:38.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:38.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:38.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:38 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:38.547+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:39.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:39.048+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:39.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:39.048+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:39.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:39.048+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:39.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:39.297+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:39.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:39.297+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:39.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:39.297+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:39.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:39.545+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:39.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:39.545+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:39.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:39.545+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:39.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:39 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:39.545+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:39.578 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:39 smithi083 podman[114784]: 2024-09-21 03:34:39.320449063 +0000 UTC m=+5.917958277 container died 919c38ab628021a110c316c4120febdb86b3001076a44f18a79fd0bdbed651ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:34:39.872 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:39 smithi083 podman[114784]: 2024-09-21 03:34:39.795102932 +0000 UTC m=+6.392612142 container cleanup 919c38ab628021a110c316c4120febdb86b3001076a44f18a79fd0bdbed651ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:34:39.872 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:39 smithi083 bash[114784]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:34:39.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:39.576+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:39.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:39.576+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:39.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:39.576+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:39.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:39 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:39.576+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:39.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:39.581+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:39.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:39.581+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:39.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:39.581+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:39.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:39.581+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:40.372 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114811]: 2024-09-21 03:34:40.063218301 +0000 UTC m=+0.744497762 container remove 919c38ab628021a110c316c4120febdb86b3001076a44f18a79fd0bdbed651ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:34:40.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.019+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:40.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.019+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:40.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.019+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:40.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:40.282+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:40.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:40.282+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:40.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:40.282+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:40.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:40.498+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:40.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:40.498+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:40.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:40.498+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:40.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:40 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:40.498+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:40.709 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:40 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:40.587+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:40.709 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:40 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:40.587+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:40.709 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:40 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:40.587+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:40.709 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:40 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:40.587+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:40.710 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.298213879 +0000 UTC m=+0.015736598 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:40.710 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.422648231 +0000 UTC m=+0.140170974 container create 3871a97a85c14aacc80a1a8a1ba8553689193744212a61ca9b3c16938f6b5ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T03:34:40.710 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.625252449 +0000 UTC m=+0.342775163 container init 3871a97a85c14aacc80a1a8a1ba8553689193744212a61ca9b3c16938f6b5ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:34:40.710 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.628541305 +0000 UTC m=+0.346064021 container start 3871a97a85c14aacc80a1a8a1ba8553689193744212a61ca9b3c16938f6b5ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:34:40.710 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.708335984 +0000 UTC m=+0.425858695 container attach 3871a97a85c14aacc80a1a8a1ba8553689193744212a61ca9b3c16938f6b5ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:34:40.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:40.535+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:40.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:40.535+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:40.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:40.535+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:40.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:40.535+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:41.122 INFO:journalctl@ceph.osd.6.smithi083.stdout:Sep 21 03:34:40 smithi083 podman[114899]: 2024-09-21 03:34:40.817933758 +0000 UTC m=+0.535456463 container died 3871a97a85c14aacc80a1a8a1ba8553689193744212a61ca9b3c16938f6b5ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-21T03:34:41.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.978+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:41.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.978+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:41.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:40 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:40.978+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:41.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:41.292+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:41.292+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:41.292+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:41.524+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:41.524+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:41.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:41.524+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:41.566 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:41 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:41.524+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:41.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:41.587 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:34:41.587 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:34:41.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:34:41.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:41 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:41.556+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:41.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:41 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:41.556+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:41.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:41 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:41.556+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:41.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:41 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:41.556+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:41.895 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 systemd[1]: Stopping Ceph osd.7 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:41.895 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:41.493+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:41.895 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:41.493+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:41.895 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:41.493+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:41.895 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:41.493+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:42.177 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.893+0000 7fb873125640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:42.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.893+0000 7fb873125640 -1 osd.7 102 *** Got signal Terminated *** 2024-09-21T03:34:42.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.893+0000 7fb873125640 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:42.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.971+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:42.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.971+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:42.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:41 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:41.971+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:42.815 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:42.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:42.815 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:42.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:42.815 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:42.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:42.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:42.574+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:42.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:42.574+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:42.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:42.574+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:42.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:42 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:42.574+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:42.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:42.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:42.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:42.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:42.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:42.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:42.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:42 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:42.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:42.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:42.464+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:42.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:42.464+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:42.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:42.464+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:42.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:42.464+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:43.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:42.951+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:43.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:42.951+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:43.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:42.951+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:43.614 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:43.354+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:43.614 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:43.354+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:43.614 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:43.354+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:43.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:43.529+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:43.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:43.529+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:43.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:43.529+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:43.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:43 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:43.529+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:43.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:43.451+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:43.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:43.451+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:43.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:43.451+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:43.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:43.451+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:44.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:43.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:44.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:43.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:44.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:43.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:44.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:43 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:43.612+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:44.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:43.968+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:44.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:43.968+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:44.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:43.968+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:44.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:43 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:43.968+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:20.721404+0000 front 2024-09-21T03:34:20.721430+0000 (oldest deadline 2024-09-21T03:34:43.621590+0000) 2024-09-21T03:34:44.658 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:44.326+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:44.658 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:44.326+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:44.659 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:44.326+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:44.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:44 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:44.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:44.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:44 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:44.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:44.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:44 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:44.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:44.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:44 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:44.570+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:44.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:44.453+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:44.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:44.453+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:44.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:44.453+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:44.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:44.453+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:44.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:44 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:44.453+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:45.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:44.656+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:45.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:44.656+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:45.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:44.656+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:45.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:44.656+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:45.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:44 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:44.656+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:45.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:45.018+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:45.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:45.018+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:45.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:45.018+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:45.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:45.018+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:20.721404+0000 front 2024-09-21T03:34:20.721430+0000 (oldest deadline 2024-09-21T03:34:43.621590+0000) 2024-09-21T03:34:45.706 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:45.372+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:45.706 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:45.372+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:45.706 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:45.372+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:45.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:45 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:45.610+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:45.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:45 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:45.610+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:45.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:45 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:45.610+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:45.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:45 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:45.610+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:45.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:45.445+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:45.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:45.445+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:45.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:45.445+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:45.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:45.445+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:45.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:45.445+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:46.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:45.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:46.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:45.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:46.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:45.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:46.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:45.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:46.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:45 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:45.704+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:46.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:46.052+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:56.817641+0000 front 2024-09-21T03:33:56.817680+0000 (oldest deadline 2024-09-21T03:34:17.917807+0000) 2024-09-21T03:34:46.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:46.052+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.818938+0000 front 2024-09-21T03:34:00.819023+0000 (oldest deadline 2024-09-21T03:34:26.119017+0000) 2024-09-21T03:34:46.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:46.052+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.620023+0000 front 2024-09-21T03:34:09.620084+0000 (oldest deadline 2024-09-21T03:34:34.920154+0000) 2024-09-21T03:34:46.428 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7[51868]: 2024-09-21T03:34:46.052+0000 7fb86f710640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:20.721404+0000 front 2024-09-21T03:34:20.721430+0000 (oldest deadline 2024-09-21T03:34:43.621590+0000) 2024-09-21T03:34:46.679 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:46.380+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:46.679 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:46.380+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:46.679 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:46.380+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:46.680 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:46.380+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:46.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:46.474+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:46.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:46.474+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:46.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:46.474+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:46.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:46.474+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:46.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:46 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:46.474+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:47.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:46.677+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:47.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:46.677+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:47.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:46.677+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:47.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:46.677+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:47.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:46 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:46.677+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:47.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:46 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:46.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:47.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:46 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:46.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:47.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:46 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:46.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:47.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:46 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:46.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:47.645 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:47.345+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:47.645 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:47.345+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:47.645 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:47.345+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:47.645 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:47.345+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:47 smithi142 podman[68222]: 2024-09-21 03:34:47.515332266 +0000 UTC m=+5.727619410 container died d4d56a66bb0ed26d799cd537d0efac92932b8f93af948711cbcc4d07fcc80cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:47 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:47.490+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:47 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:47.490+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:47 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:47.490+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:47 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:47.490+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:47.829 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:47 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:47.490+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:47.872 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:47 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:47.603+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:47.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:47 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:47.603+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:47.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:47 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:47.603+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:47.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:47 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:47.603+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:48.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:47.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:48.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:47.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:48.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:47.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:48.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:47.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:48.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:47 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:47.643+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:48.146 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:47 smithi142 podman[68222]: 2024-09-21 03:34:47.934766596 +0000 UTC m=+6.147053736 container cleanup d4d56a66bb0ed26d799cd537d0efac92932b8f93af948711cbcc4d07fcc80cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T03:34:48.146 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:47 smithi142 bash[68222]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:34:48.427 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68237]: 2024-09-21 03:34:48.144851439 +0000 UTC m=+0.626392295 container remove d4d56a66bb0ed26d799cd537d0efac92932b8f93af948711cbcc4d07fcc80cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:34:48.683 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:48.327+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:48.683 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:48.327+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:48.684 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:48.327+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:48.684 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:48.327+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:48.808 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.545680431 +0000 UTC m=+0.107652410 container create 9ab65e37447479ecfbaa8bb80d6209f6e48aa4e4e0c390b294f194ce68c045c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:34:48.808 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.454169772 +0000 UTC m=+0.016141763 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:48.808 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:48.541+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:48.808 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:48.541+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:48.808 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:48.541+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:48.809 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:48.541+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:48.809 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:48 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:48.541+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:49.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:48.681+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:49.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:48.681+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:49.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:48.681+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:49.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:48.681+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:49.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:48 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:48.681+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:49.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:48 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:48.643+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:49.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:48 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:48.643+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:49.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:48 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:48.643+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:49.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:48 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:48.643+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:49.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.806471202 +0000 UTC m=+0.368443180 container init 9ab65e37447479ecfbaa8bb80d6209f6e48aa4e4e0c390b294f194ce68c045c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:34:49.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.810611987 +0000 UTC m=+0.372583978 container start 9ab65e37447479ecfbaa8bb80d6209f6e48aa4e4e0c390b294f194ce68c045c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:34:49.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.881544953 +0000 UTC m=+0.443516934 container attach 9ab65e37447479ecfbaa8bb80d6209f6e48aa4e4e0c390b294f194ce68c045c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:34:49.178 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 21 03:34:48 smithi142 podman[68327]: 2024-09-21 03:34:48.995437593 +0000 UTC m=+0.557409586 container died 9ab65e37447479ecfbaa8bb80d6209f6e48aa4e4e0c390b294f194ce68c045c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:34:49.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:34:49.699 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:49.351+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:49.699 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:49.351+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:49.699 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:49.351+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:49.700 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:49.351+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:49.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:49.720 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:34:49.720 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:34:49.720 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:34:49.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:49 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:49.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:49.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:49 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:49.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:49.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:49 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:49.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:49.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:49 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:49.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:49.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:49 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:49.525+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:49.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:49 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:49.525+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:49.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:49 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:49.525+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:49.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:49 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:49.525+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:49.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:49 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:49.525+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:49.976 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:49 smithi193 systemd[1]: Stopping Ceph osd.8 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:49.976 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:49.725+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:49.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:49.725+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:49.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:49.725+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:49.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:49.725+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:49.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:49 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:49.725+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:50.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.037+0000 7fe7473f4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:50.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.037+0000 7fe7473f4640 -1 osd.8 102 *** Got signal Terminated *** 2024-09-21T03:34:50.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.037+0000 7fe7473f4640 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:50.698 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.375+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:50.698 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.375+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:50.699 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.375+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:50.699 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:50.375+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:50.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:50.486+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:50.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:50.486+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:50.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:50.486+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:50.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:50.486+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:50.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:50.486+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:51.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:50.696+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:51.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:50.696+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:51.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:50.696+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:51.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:50.696+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:51.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:50 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:50.696+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:51.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:50.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:51.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:50.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:51.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:50.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:51.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:50 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:50.648+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:51.690 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:51.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:51.690 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:51.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:51.690 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:51.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:51.690 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:51.331+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:51.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:51 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:51.482+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:51.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:51 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:51.482+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:51.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:51 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:51.482+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:51.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:51 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:51.482+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:51.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:51 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:51.482+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:52.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:51.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:52.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:51.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:52.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:51.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:52.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:51.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:52.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:51 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:51.688+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:52.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:51 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:51.651+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:52.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:51 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:51.651+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:52.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:51 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:51.651+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:52.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:51 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:51.651+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:52.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:52.370+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:52.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:52.370+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:52.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:52.371+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:52.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:52.371+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:52.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:52.371+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:28.325706+0000 front 2024-09-21T03:34:28.325736+0000 (oldest deadline 2024-09-21T03:34:51.825806+0000) 2024-09-21T03:34:52.927 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:52.437+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:52.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:52.437+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:52.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:52.437+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:52.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:52.437+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:52.928 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:52 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:52.437+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:53.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:52.671+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:53.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:52.671+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:53.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:52.671+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:53.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:52.671+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:53.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:52 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:52.671+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:53.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:52 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:52.664+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:53.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:52 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:52.664+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:53.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:52 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:52.664+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:53.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:52 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:52.664+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:53.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:52 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:52.665+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:53.672 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:53.386+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:53.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:53.386+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:53.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:53.386+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:53.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:53.386+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:53.673 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:53.386+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:28.325706+0000 front 2024-09-21T03:34:28.325736+0000 (oldest deadline 2024-09-21T03:34:51.825806+0000) 2024-09-21T03:34:53.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:53.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:53.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:53.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:53.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:53.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:53 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:53.394+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:54.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:54.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:54.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:54.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:54.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:54.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:53.670+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:54.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:53.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:54.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:53.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:54.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:53.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:54.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:53.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:54.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:53 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:53.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:54.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:54 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:54.374+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:54.712 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:54.409+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:55.120812+0000 front 2024-09-21T03:33:55.120877+0000 (oldest deadline 2024-09-21T03:34:18.020800+0000) 2024-09-21T03:34:54.712 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:54.409+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:00.921837+0000 front 2024-09-21T03:34:00.921872+0000 (oldest deadline 2024-09-21T03:34:26.221980+0000) 2024-09-21T03:34:54.712 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:54.409+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:13.123851+0000 front 2024-09-21T03:34:13.123698+0000 (oldest deadline 2024-09-21T03:34:37.223839+0000) 2024-09-21T03:34:54.713 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:54.409+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:21.924941+0000 front 2024-09-21T03:34:21.925063+0000 (oldest deadline 2024-09-21T03:34:45.424827+0000) 2024-09-21T03:34:54.713 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8[53403]: 2024-09-21T03:34:54.409+0000 7fe7439df640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:28.325706+0000 front 2024-09-21T03:34:28.325736+0000 (oldest deadline 2024-09-21T03:34:51.825806+0000) 2024-09-21T03:34:55.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:55.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:55.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:55.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:55.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:55.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:54 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:54.710+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:55.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:54.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:55.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:54.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:55.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:54.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:55.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:54.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:55.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:54 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:54.649+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:55.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:55.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:55.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:55.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:55.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:55.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:55 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:55.354+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:56.025 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:55 smithi193 podman[69537]: 2024-09-21 03:34:55.683418138 +0000 UTC m=+5.767345051 container died 390d18c296bb7d201c158d04fdb087a6545791214eda0c0356a48f779b24b6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:34:56.025 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:56.026 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:56.026 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:56.026 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:56.026 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:56.026 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:55 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:55.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:56.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:55.640+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:56.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:55.640+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:56.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:55.640+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:56.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:55.640+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:56.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:55 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:55.640+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:56.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:56 smithi193 podman[69537]: 2024-09-21 03:34:56.129902225 +0000 UTC m=+6.213829145 container cleanup 390d18c296bb7d201c158d04fdb087a6545791214eda0c0356a48f779b24b6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:34:56.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:56 smithi193 bash[69537]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:34:56.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:56.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:56.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:56.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:56.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:56.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:56 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:56.329+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:56.709 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:56 smithi193 podman[69563]: 2024-09-21 03:34:56.456341978 +0000 UTC m=+0.768742744 container remove 390d18c296bb7d201c158d04fdb087a6545791214eda0c0356a48f779b24b6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:34:57.002 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:56 smithi193 podman[69652]: 2024-09-21 03:34:56.668216086 +0000 UTC m=+0.015989358 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:34:57.002 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:56 smithi193 podman[69652]: 2024-09-21 03:34:56.765545363 +0000 UTC m=+0.113318625 container create 7d8c8adbac1ff7e4b6629ff15c6f5a1d2c03ed0cfc78fbf7d6cfd296798e73dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:57.003 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:56 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:56.707+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:57.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:56.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:57.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:56.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:57.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:56.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:57.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:56.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:57.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:56 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:56.622+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:57.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:57 smithi193 podman[69652]: 2024-09-21 03:34:57.001511925 +0000 UTC m=+0.349285187 container init 7d8c8adbac1ff7e4b6629ff15c6f5a1d2c03ed0cfc78fbf7d6cfd296798e73dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:34:57.315 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:57 smithi193 podman[69652]: 2024-09-21 03:34:57.004902483 +0000 UTC m=+0.352675749 container start 7d8c8adbac1ff7e4b6629ff15c6f5a1d2c03ed0cfc78fbf7d6cfd296798e73dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T03:34:57.316 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:57 smithi193 podman[69652]: 2024-09-21 03:34:57.101830708 +0000 UTC m=+0.449603972 container attach 7d8c8adbac1ff7e4b6629ff15c6f5a1d2c03ed0cfc78fbf7d6cfd296798e73dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:34:57.316 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 03:34:57 smithi193 podman[69652]: 2024-09-21 03:34:57.190995898 +0000 UTC m=+0.538769172 container died 7d8c8adbac1ff7e4b6629ff15c6f5a1d2c03ed0cfc78fbf7d6cfd296798e73dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:34:57.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:57.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:57.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:57.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:57.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:57.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:57 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:57.295+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:57.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:57.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:57.815 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:57.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:57.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:57.816 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:57 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:57.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:57.884 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:34:57.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:34:57.924 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:34:57.924 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:34:57.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:34:57.951 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:57.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:57.951 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:57.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:57.951 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:57.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:57.951 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:57.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:57.951 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:57.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:58.372 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:57 smithi083 systemd[1]: Stopping Ceph osd.9 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:34:58.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:58.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:58.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:58.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:58.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:58.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:58 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:58.278+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.480+0000 7fa11fe04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.480+0000 7fa11fe04640 -1 osd.9 102 *** Got signal Terminated *** 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.480+0000 7fa11fe04640 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:34:58.873 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:58 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:58.626+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:34:59.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:34:59.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:34:59.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:34:59.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:34:59.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:34:59.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:58.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:34:59.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:34:59.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:34:59.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:34:59.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:34:59.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:34:59.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:34:59 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:34:59.269+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:00.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:00.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:00.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:00.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:00.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:00.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:34:59 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:34:59.723+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:00.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:59 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:59.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:35:00.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:59 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:59.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:35:00.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:59 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:59.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:35:00.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:59 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:59.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:35:00.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:34:59 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:34:59.636+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:35:00.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:00.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:00.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:00.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:00.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:00.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:00 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:00.240+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:01.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:01.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:01.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:01.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:01.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:01.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:00 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:00.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:35:01.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:00 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:00.637+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:34.692337+0000 front 2024-09-21T03:34:34.692364+0000 (oldest deadline 2024-09-21T03:34:59.992476+0000) 2024-09-21T03:35:01.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:01.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:01 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:01.265+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:02.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:02.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:02.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:02.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:02.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:02.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:01.693+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:02.122 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:35:02.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:35:02.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:35:02.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:35:02.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:35:02.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:01 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:01.634+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:34.692337+0000 front 2024-09-21T03:34:34.692364+0000 (oldest deadline 2024-09-21T03:34:59.992476+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:02.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:02 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:02.305+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:03.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:03.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:03.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:03.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:03.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:03.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:02 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:02.679+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:48.586039+0000 front 2024-09-21T03:33:48.586074+0000 (oldest deadline 2024-09-21T03:34:09.685761+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.986895+0000 front 2024-09-21T03:33:54.986872+0000 (oldest deadline 2024-09-21T03:34:17.886856+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:03.787948+0000 front 2024-09-21T03:34:03.788020+0000 (oldest deadline 2024-09-21T03:34:27.887883+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:07.888314+0000 front 2024-09-21T03:34:07.888388+0000 (oldest deadline 2024-09-21T03:34:37.288882+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.791045+0000 front 2024-09-21T03:34:27.790918+0000 (oldest deadline 2024-09-21T03:34:52.490791+0000) 2024-09-21T03:35:03.123 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:02 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9[67501]: 2024-09-21T03:35:02.646+0000 7fa1183e7640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:34.692337+0000 front 2024-09-21T03:34:34.692364+0000 (oldest deadline 2024-09-21T03:34:59.992476+0000) 2024-09-21T03:35:03.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:03.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:03 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:03.276+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:04.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:04.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:04.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:04.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:04.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:04.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:03 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:03.717+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:04.317 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:04 smithi083 podman[115002]: 2024-09-21 03:35:04.061151583 +0000 UTC m=+5.944187448 container died 620715511de6abe1fc93fcfe9b4db789c3bc04330cbdb63d1e54a4c3d3c88140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:04.623 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:04 smithi083 podman[115002]: 2024-09-21 03:35:04.4735251 +0000 UTC m=+6.356560959 container cleanup 620715511de6abe1fc93fcfe9b4db789c3bc04330cbdb63d1e54a4c3d3c88140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:04.623 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:04 smithi083 bash[115002]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:04.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:04 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:04.241+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:05.035 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:04 smithi083 podman[115017]: 2024-09-21 03:35:04.64167173 +0000 UTC m=+0.570006900 container remove 620715511de6abe1fc93fcfe9b4db789c3bc04330cbdb63d1e54a4c3d3c88140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:05.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:04 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:04.726+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:05.337 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:05.034076878 +0000 UTC m=+0.100893654 container create 238e547e770d710a89035e6d8c884ce3e3bd38c2b3e762dbe2480617d24f8be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:35:05.337 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:04.948929063 +0000 UTC m=+0.015745837 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:35:05.338 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:05.261461196 +0000 UTC m=+0.328277975 container init 238e547e770d710a89035e6d8c884ce3e3bd38c2b3e762dbe2480617d24f8be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:35:05.338 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:05.264635628 +0000 UTC m=+0.331452406 container start 238e547e770d710a89035e6d8c884ce3e3bd38c2b3e762dbe2480617d24f8be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T03:35:05.338 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:05.336339935 +0000 UTC m=+0.403156715 container attach 238e547e770d710a89035e6d8c884ce3e3bd38c2b3e762dbe2480617d24f8be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:05.622 INFO:journalctl@ceph.osd.9.smithi083.stdout:Sep 21 03:35:05 smithi083 podman[115105]: 2024-09-21 03:35:05.459802264 +0000 UTC m=+0.526619038 container died 238e547e770d710a89035e6d8c884ce3e3bd38c2b3e762dbe2480617d24f8be5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:05.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:05 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:05.225+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:06.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:06.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:05.705+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:06.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:06.116 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:35:06.116 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:35:06.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 systemd[1]: Stopping Ceph osd.10 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:06.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:06.407 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:06.407 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.217+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:06.407 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.404+0000 7f2caf275640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:06.407 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.404+0000 7f2caf275640 -1 osd.10 102 *** Got signal Terminated *** 2024-09-21T03:35:06.407 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:06 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:06.404+0000 7f2caf275640 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:35:07.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:07.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:07.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:07.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:07.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:07.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:06 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:06.694+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:07.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:07.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:07 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:07.264+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:08.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:08.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:08.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:08.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:08.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:08.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:07 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:07.722+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:08.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:08 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:08.291+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:09.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:09.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:09.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:09.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:09.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:09.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:08 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:08.685+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:09.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:09 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:09.338+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:10.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:09 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:09.687+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:10.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:10 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:10.320+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:11.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:11.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:10 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:10.714+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.604497+0000 front 2024-09-21T03:33:46.604457+0000 (oldest deadline 2024-09-21T03:34:10.104390+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.705879+0000 front 2024-09-21T03:33:54.705903+0000 (oldest deadline 2024-09-21T03:34:19.405973+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:33:59.406635+0000 front 2024-09-21T03:33:59.406590+0000 (oldest deadline 2024-09-21T03:34:26.406944+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:09.908183+0000 front 2024-09-21T03:34:09.908048+0000 (oldest deadline 2024-09-21T03:34:34.607838+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:19.908571+0000 front 2024-09-21T03:34:19.908610+0000 (oldest deadline 2024-09-21T03:34:44.008673+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:27.410385+0000 front 2024-09-21T03:34:27.410325+0000 (oldest deadline 2024-09-21T03:34:52.710302+0000) 2024-09-21T03:35:11.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10[58089]: 2024-09-21T03:35:11.281+0000 7f2cab860640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-09-21T03:34:37.411502+0000 front 2024-09-21T03:34:37.411446+0000 (oldest deadline 2024-09-21T03:35:00.311196+0000) 2024-09-21T03:35:12.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:12.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:12.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:12.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:12.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:12.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:12.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:11 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:11.692+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:12.272 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:11 smithi142 podman[68443]: 2024-09-21 03:35:11.960483989 +0000 UTC m=+5.647731477 container died 7baf4d65d926937d045343ad11d3f18d22f2933e60d388ca3d8a910eb23799fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:35:12.677 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:12 smithi142 podman[68443]: 2024-09-21 03:35:12.270151274 +0000 UTC m=+5.957398760 container cleanup 7baf4d65d926937d045343ad11d3f18d22f2933e60d388ca3d8a910eb23799fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:35:12.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:12 smithi142 bash[68443]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:35:12.678 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:12 smithi142 podman[68458]: 2024-09-21 03:35:12.504487044 +0000 UTC m=+0.539219906 container remove 7baf4d65d926937d045343ad11d3f18d22f2933e60d388ca3d8a910eb23799fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:35:13.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:13.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:13.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:13.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:13.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:13.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:13.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:12 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:12.729+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:13.126 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:12 smithi142 podman[68545]: 2024-09-21 03:35:12.725066445 +0000 UTC m=+0.015379913 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:35:13.126 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:12 smithi142 podman[68545]: 2024-09-21 03:35:12.863756067 +0000 UTC m=+0.154069523 container create b3659eb2b057e5d2c051a6cbab13139863d220e7015dded55327cb3354aa5da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T03:35:13.126 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:13 smithi142 podman[68545]: 2024-09-21 03:35:13.049767869 +0000 UTC m=+0.340081322 container init b3659eb2b057e5d2c051a6cbab13139863d220e7015dded55327cb3354aa5da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:13.126 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:13 smithi142 podman[68545]: 2024-09-21 03:35:13.053024509 +0000 UTC m=+0.343337965 container start b3659eb2b057e5d2c051a6cbab13139863d220e7015dded55327cb3354aa5da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:35:13.126 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:13 smithi142 podman[68545]: 2024-09-21 03:35:13.124494445 +0000 UTC m=+0.414807902 container attach b3659eb2b057e5d2c051a6cbab13139863d220e7015dded55327cb3354aa5da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:35:13.427 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 21 03:35:13 smithi142 podman[68545]: 2024-09-21 03:35:13.251698112 +0000 UTC m=+0.542011566 container died b3659eb2b057e5d2c051a6cbab13139863d220e7015dded55327cb3354aa5da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:13.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:35:13.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:13.865 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:35:13.865 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:35:13.865 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:14.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:14.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:13.737+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:14.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:13 smithi193 systemd[1]: Stopping Ceph osd.11 for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:14.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.200+0000 7fd1f52c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:14.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.200+0000 7fd1f52c9640 -1 osd.11 102 *** Got signal Terminated *** 2024-09-21T03:35:14.565 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.200+0000 7fd1f52c9640 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:35:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:15.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:15.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:15.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:14 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:14.709+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:16.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:15 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:15.745+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:17.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:17.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:17.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:16 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:16.711+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-21T03:34:49.999116+0000 front 2024-09-21T03:34:49.999152+0000 (oldest deadline 2024-09-21T03:35:15.899131+0000) 2024-09-21T03:35:18.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.682+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:18.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.682+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:18.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.682+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:18.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.682+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:18.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.682+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:18.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.683+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:18.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.683+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:18.066 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:17 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:17.683+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-21T03:34:49.999116+0000 front 2024-09-21T03:34:49.999152+0000 (oldest deadline 2024-09-21T03:35:15.899131+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-09-21T03:33:46.890315+0000 front 2024-09-21T03:33:46.890261+0000 (oldest deadline 2024-09-21T03:34:11.590473+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-09-21T03:33:54.992391+0000 front 2024-09-21T03:33:54.992050+0000 (oldest deadline 2024-09-21T03:34:20.292109+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T03:34:04.393015+0000 front 2024-09-21T03:34:04.393095+0000 (oldest deadline 2024-09-21T03:34:29.093209+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-09-21T03:34:12.594234+0000 front 2024-09-21T03:34:12.594256+0000 (oldest deadline 2024-09-21T03:34:34.894090+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-09-21T03:34:18.295920+0000 front 2024-09-21T03:34:18.295967+0000 (oldest deadline 2024-09-21T03:34:44.195820+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T03:34:29.997167+0000 front 2024-09-21T03:34:29.997124+0000 (oldest deadline 2024-09-21T03:34:53.497280+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-09-21T03:34:44.099027+0000 front 2024-09-21T03:34:44.099101+0000 (oldest deadline 2024-09-21T03:35:09.998809+0000) 2024-09-21T03:35:18.967 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:18 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11[59788]: 2024-09-21T03:35:18.641+0000 7fd1f18b4640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-21T03:34:49.999116+0000 front 2024-09-21T03:34:49.999152+0000 (oldest deadline 2024-09-21T03:35:15.899131+0000) 2024-09-21T03:35:20.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:19 smithi193 podman[69754]: 2024-09-21 03:35:19.789337744 +0000 UTC m=+5.726255446 container died 6402f5d9319da9cb656f2f7ffaf7c4dd1811007db3a8b961bb47a2efb2d99e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:35:20.435 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 podman[69754]: 2024-09-21 03:35:20.166683432 +0000 UTC m=+6.103601165 container cleanup 6402f5d9319da9cb656f2f7ffaf7c4dd1811007db3a8b961bb47a2efb2d99e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:20.436 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 bash[69754]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:35:20.761 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 podman[69770]: 2024-09-21 03:35:20.434484516 +0000 UTC m=+0.645283254 container remove 6402f5d9319da9cb656f2f7ffaf7c4dd1811007db3a8b961bb47a2efb2d99e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:35:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 podman[69858]: 2024-09-21 03:35:20.760270442 +0000 UTC m=+0.106791185 container create 9d94520522ce5f807c491c866e87f618bc90573c31f9e634d767f8dfffb37880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:35:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 podman[69858]: 2024-09-21 03:35:20.669275736 +0000 UTC m=+0.015796451 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:35:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:20 smithi193 podman[69858]: 2024-09-21 03:35:20.996802956 +0000 UTC m=+0.343323661 container init 9d94520522ce5f807c491c866e87f618bc90573c31f9e634d767f8dfffb37880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:21.065 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:21 smithi193 podman[69858]: 2024-09-21 03:35:21.000106674 +0000 UTC m=+0.346627389 container start 9d94520522ce5f807c491c866e87f618bc90573c31f9e634d767f8dfffb37880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:21.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:21 smithi193 podman[69858]: 2024-09-21 03:35:21.084626771 +0000 UTC m=+0.431147483 container attach 9d94520522ce5f807c491c866e87f618bc90573c31f9e634d767f8dfffb37880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:35:21.483 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 03:35:21 smithi193 podman[69858]: 2024-09-21 03:35:21.186973467 +0000 UTC m=+0.533494169 container died 9d94520522ce5f807c491c866e87f618bc90573c31f9e634d767f8dfffb37880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:35:21.776 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:35:21.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:21.815 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:35:21.815 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:35:21.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:35:22.111 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:21 smithi083 systemd[1]: Stopping Ceph mds.a for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:22.368 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a[74408]: 2024-09-21T03:35:22.108+0000 7f8600d49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:22.368 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:22 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a[74408]: 2024-09-21T03:35:22.108+0000 7f8600d49640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:35:25.122 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:24 smithi083 podman[115204]: 2024-09-21 03:35:24.773598146 +0000 UTC m=+2.766114867 container died d57899cc9133eb766f6a36ddbfb9b73eefbf34038db8a504426308ce8d71c107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:25.494 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:25 smithi083 podman[115204]: 2024-09-21 03:35:25.216741594 +0000 UTC m=+3.209258306 container cleanup d57899cc9133eb766f6a36ddbfb9b73eefbf34038db8a504426308ce8d71c107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:35:25.494 INFO:journalctl@ceph.mds.a.smithi083.stdout:Sep 21 03:35:25 smithi083 bash[115204]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:35:25.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:35:25.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:25.656 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:35:25.657 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:35:25.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:35:25.872 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:25 smithi083 systemd[1]: Stopping Ceph mds.d for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:26.372 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d[73285]: 2024-09-21T03:35:25.979+0000 7f8f942f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:26.372 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:25 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d[73285]: 2024-09-21T03:35:25.979+0000 7f8f942f3640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:35:30.256 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:29 smithi083 podman[115339]: 2024-09-21 03:35:29.995655511 +0000 UTC m=+4.116972870 container died 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:35:30.623 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:30 smithi083 podman[115339]: 2024-09-21 03:35:30.3460496 +0000 UTC m=+4.467366902 container cleanup 41475ed58ad8ff1c46357f14af370da7a0ca0425d7bbed2c47e28ce6f7b1c69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:35:30.626 INFO:journalctl@ceph.mds.d.smithi083.stdout:Sep 21 03:35:30 smithi083 bash[115339]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:35:30.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:35:30.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:30.793 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:35:30.793 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:35:30.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:35:31.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:30 smithi083 systemd[1]: Stopping Ceph mds.g for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:31.373 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: 2024-09-21T03:35:31.083+0000 7f869dd83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:31.373 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:31 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g[74023]: 2024-09-21T03:35:31.083+0000 7f869dd83640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:35:33.052 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:32 smithi083 podman[115460]: 2024-09-21 03:35:32.772486213 +0000 UTC m=+1.765367597 container died e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:35:33.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:33 smithi083 podman[115460]: 2024-09-21 03:35:33.149899455 +0000 UTC m=+2.142780845 container cleanup e2060f161d42d69bfa346b2078058d98892311815b6d04bebb02f180c95e5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:33.372 INFO:journalctl@ceph.mds.g.smithi083.stdout:Sep 21 03:35:33 smithi083 bash[115460]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:35:33.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:35:33.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:33.603 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:35:33.603 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:35:33.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:35:33.921 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:33 smithi083 systemd[1]: Stopping Ceph mds.j for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:34.372 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j[72920]: 2024-09-21T03:35:33.920+0000 7fad42b9a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:34.372 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:33 smithi083 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j[72920]: 2024-09-21T03:35:33.920+0000 7fad42b9a640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:35:38.214 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:37 smithi083 podman[115580]: 2024-09-21 03:35:37.948027421 +0000 UTC m=+4.127029869 container died 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:35:38.498 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:38 smithi083 podman[115580]: 2024-09-21 03:35:38.287304942 +0000 UTC m=+4.466307446 container cleanup 3ac39b45586d8b7bfc27ed4ed4926030d14a1fd0d155d94973112a3f77d9704b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:35:38.498 INFO:journalctl@ceph.mds.j.smithi083.stdout:Sep 21 03:35:38 smithi083 bash[115580]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:35:38.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:35:38.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:38.754 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:35:38.754 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:35:38.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:35:39.054 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:38 smithi142 systemd[1]: Stopping Ceph mds.b for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:39.314 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:35:39.052+0000 7f49cd03e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:39.315 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:39 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b[62972]: 2024-09-21T03:35:39.052+0000 7f49cd03e640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:35:41.172 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:40 smithi142 podman[68647]: 2024-09-21 03:35:40.783144791 +0000 UTC m=+1.831705395 container died 144994b3b26d45bdce5bdc89e15e54c401a76ddab5f0b2e9fa0696fc78174580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:35:41.427 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:41 smithi142 podman[68647]: 2024-09-21 03:35:41.269312393 +0000 UTC m=+2.317872997 container cleanup 144994b3b26d45bdce5bdc89e15e54c401a76ddab5f0b2e9fa0696fc78174580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:35:41.428 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 21 03:35:41 smithi142 bash[68647]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:35:41.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:35:41.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:41.731 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:35:41.732 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:35:41.732 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:35:42.058 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:41 smithi142 systemd[1]: Stopping Ceph mds.e for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:42.427 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e[62428]: 2024-09-21T03:35:42.056+0000 7f961cc1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:42.428 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:42 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e[62428]: 2024-09-21T03:35:42.056+0000 7f961cc1a640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:35:45.042 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:44 smithi142 podman[68783]: 2024-09-21 03:35:44.78087814 +0000 UTC m=+2.831670536 container died fe46fd12cb07cf24e5f59a6d7032f0df4e302dbdadf87845e68c3487a635f54b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:35:45.409 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:45 smithi142 podman[68783]: 2024-09-21 03:35:45.139789613 +0000 UTC m=+3.190582010 container cleanup fe46fd12cb07cf24e5f59a6d7032f0df4e302dbdadf87845e68c3487a635f54b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:35:45.409 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 21 03:35:45 smithi142 bash[68783]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:35:45.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:35:45.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:45.600 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:35:45.600 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:35:45.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:35:45.927 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:45 smithi142 systemd[1]: Stopping Ceph mds.h for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:46.427 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h[62701]: 2024-09-21T03:35:45.926+0000 7fe353863640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:46.428 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:45 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h[62701]: 2024-09-21T03:35:45.926+0000 7fe353863640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:35:49.162 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:48 smithi142 podman[68903]: 2024-09-21 03:35:48.784946422 +0000 UTC m=+2.969111406 container died 6348f3961e6ef2e91df6da54498ae4517ad20d3e60a995e178b3b53c4900316b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:35:49.427 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:49 smithi142 podman[68903]: 2024-09-21 03:35:49.235100599 +0000 UTC m=+3.419265592 container cleanup 6348f3961e6ef2e91df6da54498ae4517ad20d3e60a995e178b3b53c4900316b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:35:49.462 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 21 03:35:49 smithi142 bash[68903]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:35:49.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:35:49.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:49.704 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:35:49.704 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:35:49.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:35:50.177 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:49 smithi142 systemd[1]: Stopping Ceph mds.k for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:50.177 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: 2024-09-21T03:35:50.029+0000 7fdb06500640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:50.178 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:50 smithi142 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k[62148]: 2024-09-21T03:35:50.029+0000 7fdb06500640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:35:53.074 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:52 smithi142 podman[69027]: 2024-09-21 03:35:52.777711183 +0000 UTC m=+2.854329915 container died ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:35:53.428 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:53 smithi142 podman[69027]: 2024-09-21 03:35:53.163651275 +0000 UTC m=+3.240270000 container cleanup ca0a5e702f669090c93896ddd431ddee75c7b5a0211a7435bc7e674b7940edb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:35:53.428 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 21 03:35:53 smithi142 bash[69027]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:35:53.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:35:53.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:53.629 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:35:53.630 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:35:53.630 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:35:53.934 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:53 smithi193 systemd[1]: Stopping Ceph mds.c for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:54.315 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c[64498]: 2024-09-21T03:35:53.932+0000 7fccba43b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:54.315 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:53 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c[64498]: 2024-09-21T03:35:53.932+0000 7fccba43b640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:35:57.065 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:56 smithi193 podman[69961]: 2024-09-21 03:35:56.788147568 +0000 UTC m=+2.960388686 container died 34f6a285446510b6822edce77b4694e2cef7066cab8817bc4d8e9ecfda77805a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:35:57.443 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:57 smithi193 podman[69961]: 2024-09-21 03:35:57.2235776 +0000 UTC m=+3.395818709 container cleanup 34f6a285446510b6822edce77b4694e2cef7066cab8817bc4d8e9ecfda77805a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:35:57.443 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:57 smithi193 bash[69961]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:35:57.702 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 03:35:57 smithi193 podman[69985]: 2024-09-21 03:35:57.441320966 +0000 UTC m=+0.656165462 container remove 34f6a285446510b6822edce77b4694e2cef7066cab8817bc4d8e9ecfda77805a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:35:57.706 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:35:57.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:35:57.743 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:35:57.743 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:35:57.743 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:35:58.065 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:35:57 smithi193 systemd[1]: Stopping Ceph mds.f for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:35:58.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:35:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: 2024-09-21T03:35:58.068+0000 7f1326498640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:35:58.565 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:35:58 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f[64215]: 2024-09-21T03:35:58.068+0000 7f1326498640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:36:01.054 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:36:00 smithi193 podman[70092]: 2024-09-21 03:36:00.784706238 +0000 UTC m=+2.815643138 container died 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:36:01.315 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:36:01 smithi193 podman[70092]: 2024-09-21 03:36:01.152577536 +0000 UTC m=+3.183514430 container cleanup 0568e50348aa37aaf8c116ca19167a20f93db48f92126bae7f62ad82a9305027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T03:36:01.315 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 03:36:01 smithi193 bash[70092]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:36:01.568 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:36:01.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:36:01.602 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:36:01.603 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:36:01.603 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:36:02.065 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:01 smithi193 systemd[1]: Stopping Ceph mds.i for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:36:02.065 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: 2024-09-21T03:36:01.931+0000 7f7262863640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:36:02.065 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:01 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i[64768]: 2024-09-21T03:36:01.931+0000 7f7262863640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:36:05.058 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:04 smithi193 podman[70215]: 2024-09-21 03:36:04.779191397 +0000 UTC m=+2.955168108 container died 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:36:05.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:05 smithi193 podman[70215]: 2024-09-21 03:36:05.156194425 +0000 UTC m=+3.332171134 container cleanup 507f099162a6e05bff52e8d31bb0f7d0fc0987b99df9a507027e11911e1447b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:36:05.315 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 03:36:05 smithi193 bash[70215]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:36:05.586 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:36:05.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:36:05.618 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:36:05.618 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:36:05.619 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:36:06.065 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:05 smithi193 systemd[1]: Stopping Ceph mds.l for 5108497e-77c6-11ef-baf5-efdc52797490... 2024-09-21T03:36:06.065 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: 2024-09-21T03:36:05.942+0000 7fa4a84b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:36:06.065 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:05 smithi193 ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l[63946]: 2024-09-21T03:36:05.942+0000 7fa4a84b3640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:36:09.061 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:08 smithi193 podman[70335]: 2024-09-21 03:36:08.790669069 +0000 UTC m=+2.952123731 container died 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:36:09.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 podman[70335]: 2024-09-21 03:36:09.134843545 +0000 UTC m=+3.296298206 container cleanup 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:36:09.315 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 bash[70335]: ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:36:09.592 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:36:09.613 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 podman[70347]: 2024-09-21 03:36:09.360944955 +0000 UTC m=+0.564873632 container remove 7e2a7e1d8a23e9596ce8c0b90608ed8f35b6dba08f06583ef4ec3a22640b815b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5108497e-77c6-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:36:09.613 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 systemd[1]: ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.l.service: Deactivated successfully. 2024-09-21T03:36:09.613 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 systemd[1]: Stopped Ceph mds.l for 5108497e-77c6-11ef-baf5-efdc52797490. 2024-09-21T03:36:09.613 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 03:36:09 smithi193 systemd[1]: ceph-5108497e-77c6-11ef-baf5-efdc52797490@mds.l.service: Consumed 2.970s CPU time. 2024-09-21T03:36:09.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:36:09.624 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:36:09.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:36:09.828 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:36:12.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:36:12.525 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:36:15.116 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:36:15.319 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:36:17.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:17.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:17.862 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:36:17.889 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:36:17.891 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi083/crash 2024-09-21T03:36:17.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:36:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:36:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:36:17.935 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi142/crash 2024-09-21T03:36:17.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:36:17.974 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:36:17.974 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:36:17.976 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi193/crash 2024-09-21T03:36:17.977 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:36:18.013 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/5108497e-77c6-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:36:18.013 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:36:18.014 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:36:18.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:36:18.094 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:36:18.094 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:36:18.097 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:36:18.099 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:36:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:36:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-09-21T03:36:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:36:18.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:36:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:36:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:36:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:36:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:36:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:36:18.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:36:18.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:36:18.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:36:18.131 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:36:18.131 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.8.log: /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:36:18.132 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:36:18.146 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T03:36:18.146 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:36:18.146 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:36:18.147 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:36:18.147 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:36:18.148 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:36:18.148 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:36:18.150 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:36:18.150 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:36:18.151 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:36:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:36:18.270 INFO:teuthology.orchestra.run.smithi083.stderr: 84.3% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:36:18.274 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:36:18.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 81.8% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:36:18.294 INFO:teuthology.orchestra.run.smithi193.stderr: 87.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:36:18.297 INFO:teuthology.orchestra.run.smithi083.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:36:18.312 INFO:teuthology.orchestra.run.smithi083.stderr: 95.0% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:36:18.313 INFO:teuthology.orchestra.run.smithi142.stderr: 81.8% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:36:18.314 INFO:teuthology.orchestra.run.smithi193.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:36:18.314 INFO:teuthology.orchestra.run.smithi142.stderr: 90.3% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:36:18.315 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:36:18.334 INFO:teuthology.orchestra.run.smithi142.stderr: 87.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:36:18.334 INFO:teuthology.orchestra.run.smithi142.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:36:18.339 INFO:teuthology.orchestra.run.smithi193.stderr: 93.0% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:36:18.349 INFO:teuthology.orchestra.run.smithi083.stderr: 95.0% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:36:18.375 INFO:teuthology.orchestra.run.smithi193.stderr: 90.4% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:36:18.452 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:36:18.669 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:36:18.713 INFO:teuthology.orchestra.run.smithi142.stderr: 93.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:36:18.812 INFO:teuthology.orchestra.run.smithi142.stderr: 91.0% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:36:18.855 INFO:teuthology.orchestra.run.smithi083.stderr: 93.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:36:18.929 INFO:teuthology.orchestra.run.smithi193.stderr: 90.9% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:36:19.140 INFO:teuthology.orchestra.run.smithi083.stderr: 90.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:36:19.204 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:36:19.509 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:36:19.650 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:36:20.492 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:36:21.726 INFO:teuthology.orchestra.run.smithi193.stderr: 91.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:36:25.634 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:36:26.088 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:36:26.840 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:36:30.338 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:36:32.767 INFO:teuthology.orchestra.run.smithi142.stderr: 93.8% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:37:30.247 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:37:30.310 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:37:37.982 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:37:40.473 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:37:43.851 INFO:teuthology.orchestra.run.smithi142.stderr: 91.6% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:37:48.745 INFO:teuthology.orchestra.run.smithi142.stderr: 91.3% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:37:53.995 INFO:teuthology.orchestra.run.smithi142.stderr: 91.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:37:57.596 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:38:09.378 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:38:09.380 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-09-21T03:38:09.380 INFO:teuthology.orchestra.run.smithi142.stderr:real 1m51.266s 2024-09-21T03:38:09.380 INFO:teuthology.orchestra.run.smithi142.stderr:user 3m50.956s 2024-09-21T03:38:09.380 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m7.678s 2024-09-21T03:38:10.837 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:38:10.839 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-21T03:38:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:real 1m52.729s 2024-09-21T03:38:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:user 3m49.695s 2024-09-21T03:38:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m7.965s 2024-09-21T03:38:29.703 INFO:teuthology.orchestra.run.smithi193.stderr: 91.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:38:35.272 INFO:teuthology.orchestra.run.smithi193.stderr: 91.2% -- replaced with /var/log/ceph/5108497e-77c6-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:38:35.275 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-21T03:38:35.275 INFO:teuthology.orchestra.run.smithi193.stderr:real 2m17.161s 2024-09-21T03:38:35.275 INFO:teuthology.orchestra.run.smithi193.stderr:user 4m4.473s 2024-09-21T03:38:35.275 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m8.521s 2024-09-21T03:38:35.275 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:38:35.275 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi083/log 2024-09-21T03:38:35.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:39:12.078 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi142/log 2024-09-21T03:39:12.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:39:50.164 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi193/log 2024-09-21T03:39:50.165 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:40:31.799 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:40:31.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force 2024-09-21T03:40:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:40:32.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force 2024-09-21T03:40:32.768 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:40:33.369 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5108497e-77c6-11ef-baf5-efdc52797490 --force 2024-09-21T03:40:33.572 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 5108497e-77c6-11ef-baf5-efdc52797490 2024-09-21T03:40:34.171 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:40:34.171 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:40:34.190 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:40:34.210 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:40:34.226 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:40:34.226 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:40:34.237 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:40:34.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:40:34.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:40:34.252 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:40:34.357 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:40:34.366 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:40:34.366 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:40:34.369 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:40:34.371 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:40:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:40:34.386 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:40:34.386 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:40:34.388 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:40:34.388 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-21T03:40:34.388 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 -335us[ -324us] +/- 54ms 2024-09-21T03:40:34.388 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 +619us[ +619us] +/- 80ms 2024-09-21T03:40:34.389 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -83us[ -83us] +/- 84ms 2024-09-21T03:40:34.389 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:40:34.391 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:40:34.391 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-21T03:40:34.392 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 -329us[ -290us] +/- 54ms 2024-09-21T03:40:34.392 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +621us[ +661us] +/- 80ms 2024-09-21T03:40:34.392 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -94us[ -55us] +/- 84ms 2024-09-21T03:40:34.392 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 -332us[ -330us] +/- 54ms 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 +617us[ +617us] +/- 80ms 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -104us[ -104us] +/- 84ms 2024-09-21T03:40:34.395 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:40:34.395 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:40:34.404 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:40:34.404 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:40:34.454 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:40:34.662 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:40:34.767 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:40:34.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:40:34.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:34.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:40:34.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:34.934 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:40:34.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:34.962 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:40:34.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:40:35.038 INFO:teuthology.task.internal:Duration was 2492.817277 seconds 2024-09-21T03:40:35.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:40:35.088 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:40:35.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:40:35.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:40:35.093 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:40:35.151 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:40:35.155 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:40:35.165 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:40:35.618 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:40:35.618 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-09-21T03:40:35.618 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:40:35.642 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-09-21T03:40:35.643 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:40:35.672 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T03:40:35.672 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:40:35.700 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:40:35.701 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:40:35.703 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:40:35.714 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:40:35.777 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:40:35.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:40:35.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:40:35.786 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:40:37.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:40:37.184 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:40:37.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:40:37.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:40:37.262 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:40:37.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:40:37.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.325 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.355 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-09-21T03:40:37.359 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-09-21T03:40:37.378 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-21T03:40:37.402 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:37.427 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:37.444 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:40:37.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:40:37.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:40:37.471 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:40:37.472 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi083 2024-09-21T03:40:37.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:40:37.534 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi142 2024-09-21T03:40:37.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:40:37.574 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913450/remote/smithi193 2024-09-21T03:40:37.574 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:40:37.613 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:40:37.613 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:40:37.616 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:40:37.618 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:40:37.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:40:37.676 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:40:37.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:40:37.705 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:40:37.705 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:40:37.708 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:40:37.710 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:40:37.720 INFO:teuthology.orchestra.run.smithi083.stdout: 266054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:40 /home/ubuntu/cephtest 2024-09-21T03:40:37.725 INFO:teuthology.orchestra.run.smithi142.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:40 /home/ubuntu/cephtest 2024-09-21T03:40:37.727 INFO:teuthology.orchestra.run.smithi193.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:40 /home/ubuntu/cephtest 2024-09-21T03:40:37.735 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:40:37.756 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:40:37.858 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 2492.817276954651 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T03:40:37.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:40:37.937 INFO:teuthology.run:pass